MemTensor/MemT-1.0

MemT-1.0 is a new generation of memory foundation models that makes persistent, dynamically evolving memory an intrinsic part of model computation. MemT-1.0 integrates dynamic local memory and trainable hyper memory to retain and leverage historical information without accessing original context at inference. Its persistent state remains constant in size, reducing storage by approximately 66× versus full-context KV cache at 128K history length. The project makes three main contributions: A Native Memory Block runs alongside the backbone attention and contains two components: During generation, memory attention reads from the current memory state, normalizes and gates the readout, and adds it to the backbone attention output. At the end of each turn or segment, stable GDN-2 performs token-level memory updates for subsequent steps. Across 11 memory benchmarks, MemT-1.0-Flash and MemT-1.0-Nano rank first and second among the evaluated parametric-memory methods, respectively. Strong textual-memory systems remain ahead overall, particularly on long histories that require…