Holding a Million Tokens: Efficient Long-Context Methods at Equal Quality
Which long-context method actually holds a million tokens — and which just passes needle tests while failing real long-range reasoning?
article · by @goodai
Thesis. Many long-context methods pass needle-in-a-haystack retrieval while failing tasks that need genuine long-range reasoning; at equal quality on a reasoning-first benchmark, the methods separate sharply, and KV-cache compression plus a strong local-global mix beats both full attention cost and naive sparsity.