How Moonshot AI replaced a decade-old fixed-weight residual connection with learned depth-wise attention, unlocking 25% more compute efficiency in Kimi
Ig one should be reading this atleast 2 times
Ig one should be reading this atleast 2 times