Researchers isolate memorization from reasoning in AI neural networks

"When engineers build AI language models like GPT-5 from training data, at least two major processing features emerge: memorization (reciting exact text they've seen before, like famous quotes or passages from books) and reasoning (solving new problems using general principles). New research from AI startup Goodfire.ai provides the first potentially clear evidence that these different functions actually work through completely separate neural pathways in the model's architecture."

"In a preprint paper released in late October, they described that when they removed the memorization pathways, models lost 97 percent of their ability to recite training data verbatim but kept nearly all their "logical reasoning" ability intact. For example, at layer 22 in Allen Institute for AI's OLMo-7B language model, the bottom 50 percent of weight components showed 23 percent higher activation on memorized data, while the top 10 percent showed 26 percent higher activation on general, non-memorized text."

Large language models develop two distinct processing features: memorization and reasoning. These features operate via separate neural pathways. Surgical removal of memorization pathways eliminates 97 percent of verbatim recitation while preserving nearly all logical reasoning performance. Layer-wise analysis of OLMo-7B shows lower-weight components activate more on memorized data and top components activate more on non-memorized text, enabling targeted ablation. Arithmetic tasks rely predominantly on memorization circuits; removing those circuits drops math performance to 66 percent while leaving logical tasks largely intact. Current-scale models therefore often retrieve arithmetic from memorized tables rather than performing algorithmic computation.

Read at Ars Technica