yifanzhang-pro/recurrent-looped-tranformer

[](https://arxiv.org/abs/2610.07591) [](https://yifanzhang-pro.github.io/recurrent-looped-tranformer/) Recurrent Looped Transformer (RLT) feeds the previous token's final decoder state into the current token's decoder input through a gated merge, so the computation path grows with sequence length at a fixed per-token cost. A parallel causal encoder supplies token representations and global key–value (KV) memory, and the decoder keeps a sliding-window attention (SWA) cache at every layer. The same update runs over prompt and response tokens. Authors: Yifan Zhang¹, Jichen Feng², Shihan Qin², Sun-Yuan Kung¹, Mengdi Wang¹ · ¹Princeton University, ²University of Pennsylvania arXiv: 2610.07591 · Submitted: October 6, 2026 · Project updated: October 11, 2026 [arXiv] [Latest PDF] [arXiv PDF] [中文论文] [Project website] [Depth allocation] [Feedback frequency] [Complete six-task study] For token $xt$, let $et$ be its causal encoder representation and $M{\le t}$ the encoder-derived global KV memory. The decoder state includes both the recurrent output and layerwise SWA KV: Architecture PDF After $t$ tokens, the…