Berkeley researchers teach language model agents to compress memory without losing the plot

8:00 AM PT · July 31, 2026

Researchers Jakob Bjorner, Aly Lidayan, Satvik Golechha, Kartik Goyal, and Alane Suhr published ABBEL to address a specific, practical problem: as language model agents take on tasks spanning hundreds or thousands of interactions, such as long software development sessions, keeping the full history in context becomes impractical, but existing summarization approaches consistently underperform models that simply keep everything in context, in part because there is little high quality training data for good summarization in interactive settings. ABBEL’s core idea is to treat a running summary as a belief state that updates periodically as new observations arrive, and to add an auxiliary training signal the authors call belief grading, inspired by autoencoders, which rewards a belief state for preserving enough information that the model can reconstruct the most recent observation from it alone. On CollabBench, a collaborative coding benchmark, ABBEL with reconstruction based belief grading closed about half the performance gap to a full context model, while needing half as many training steps, 50 rather than 100, and using far less peak context, about 6,010 tokens against 14,080 for the full context approach. On a Wordle like game called Combination Lock, adding domain specific belief grading let ABBEL match or exceed full context performance, while removing that grading slowed learning back down. The authors frame the result as evidence that what an agent chooses to remember, and how that memory is trained, matters as much as how much context a model can technically hold.