The Weekly Paperclip W4: Where Agent Experience Goes.

Executive Summary

Every organisation deploying a persistent-memory agent is accumulating a liability it has not measured. Not the model weights, which are audited to death, but the experience: the traces, the summaries, the distilled skills, the retrieval index that quietly grew for eight months. At some point a customer exercises a deletion right, or a regulator asks what the system still knows about a person, or an internal review asks whether a retracted document is genuinely gone. The honest answer in most deployments is that nobody knows, because nobody decided in advance where deletion was even possible.

Five papers from this window converge on that question from different directions. One measures deletion in a memory-externalising architecture and finds something genuinely surprising. One deliberately moves experience out of context and into weights, which is the same boundary crossed in the opposite direction and on purpose. One proposes a governance architecture that gates memory consolidation before it happens. One converts traces into callable skills that carry their own applicability boundaries and evidence links. And one tests the thing every team actually does, which is squash the context down, and finds it changes nothing about output quality while costing more.

Read together they draw a line I have started calling the Deletion Boundary: the line in an agent's architecture on one side of which forgetting is an administrative act, and on the other side of which it is a retraining problem. The boundary is not a property of your policy. It is a property of your architecture, it is fixed the day you choose where experience accumulates, and almost nobody chooses it deliberately.

TL;DR: Deletion in agent systems is an architectural decision made before deployment, not a compliance control bolted on after. Where your experience lives determines whether removing it is a database operation or an impossibility.

Who this is for: anyone accountable for what a persistent agent remembers, anyone who has signed a data-deletion commitment they have not architecturally verified, engineering leaders choosing where agent experience accumulates, and teams whose agents have been running long enough to have accumulated something worth deleting.

How to Read This Edition

N.I.C.E. is my editorial instrument, not the authors' own. No research team here scored their own work on these axes. Every number below is my assessment as a reader, and I give the one-line reason so you can disagree precisely.

• N, Novelty. Genuinely new, or a competent recombination?
• I, Industry-impact. How soon, and how directly, can a production team use this?
• C, Code-reproducibility. Can you run it? Four of the five papers in this edition mention no public repository, and I have capped each of them accordingly. That is a statement about what I could verify, not a judgement of the work. A conceptual framework ships no runnable artifact and is capped by design rather than as criticism.
• E, Elegance. Does the central mechanism feel inevitable once you see it?

Each pillar runs 1 to 5, total out of 20. Every arXiv identifier, title, author list and submission date below was checked against the primary record before this shipped. Two of the five papers predate this week and are dated honestly rather than presented as new. A target or a demo is not a result, and I mark the difference every time it matters.

The Countdown

#1: Auditing Forgetting in Limited Memory Language Models

Links: arXiv:2607.00605 (submitted 1 July 2026, cs.CL). Authors: Arya Raeesi, Hanna Roed. No public repository is mentioned in the abstract.

The Problem. Limited Memory Language Models externalise factual knowledge into a database precisely so that a fact can be deleted without retraining the model. That is the entire promise: unlearning becomes a delete statement. But nobody had rigorously audited whether the promise holds. If you delete a fact from the store and the model still answers correctly, where did the answer come from? The comfortable assumption in the field, and one I have repeated myself, is that the model has quietly absorbed the fact into its weights, and that a gradient cannot be deleted.

The Aha Solution. The authors build a causal audit rather than a benchmark. They decompose post-deletion behaviour into three separable components: parametric leakage, retrieval-mediated correctness, and retrieval artifact rate, then run three database interventions (FULL, DEL-ON, DEL-OFF) across 12,228 deletions to attribute surviving correctness to a cause instead of guessing at one.

The result overturns the comfortable assumption. Parametric leakage is, in their words, "near zero in every variant and every prompt style": strip retrieval away and the model rarely produces the deleted answer at all. What survives, survives in the index. Residual correctness ranges from 0.7 percent on the released database to 13.6 percent on the most adversarial variant, and the retrieval artifact rate matches retrieval-mediated correctness "within rounding everywhere." Their conclusion is the line worth carrying into any architecture review: post-deletion correctness is "predominantly reconstituted from near-neighbor retrieval," which means the unlearning boundary "is drawn primarily by the database administrator rather than by the model."

I need to correct something I have said. I have argued that the hard part of AI deletion is that you cannot delete a gradient. In this architecture that is measurably not where the problem lives. The weights forget approximately as advertised. The residual survives in the retrieval graph, reconstituted from near neighbours of the thing you removed. That is a far better problem to have, because it is a problem an administrator can actually act on, and a far worse one to discover late, because most teams are auditing the model and not the index.

Industry Application. If you have signed a deletion commitment for a retrieval-backed agent, your verification procedure is probably testing the wrong component. Testing that the model no longer emits a fact when retrieval is live tells you very little. The audit that matters is theirs: ablate retrieval and test, then restore retrieval and test again, and attribute the difference. And the deeper design lesson is that deleting a record is not the same as deleting what the record's neighbours collectively imply. If five near-duplicate passages support a fact, removing one is theatre.

N.I.C.E. Scorecard

Key takeaway

• Builder: Audit the index, not the model. Run the ablation: delete, disable retrieval, test; then re-enable retrieval and test again. The gap is your real residual.
• Enterprise / regulated: Your deletion commitment is an architectural claim. If you cannot produce this attribution for your own stack, you have made a promise you have not verified, and near-neighbour reconstitution is the mechanism most likely to break it.

#2: Sample-Efficient Learning from Agent Experience

Links: arXiv:2607.21051 (submitted 23 July 2026, cs.CL). Authors: Chenhui Gou, Haoqin Tu, Yunhao Fang, Jianfei Cai, Hamid Rezatofighi. No public repository is mentioned in the abstract.

The Problem. Agents learn well from their own interaction histories through in-context learning, and the gains vanish the moment that history leaves the context window. Environment interactions are expensive: real experiments, real human feedback, real money. So the field wants to internalise experience into weights without paying for the experience twice.

The Aha Solution. The authors name the problem Experience Distillation and implement it with no further environment interaction beyond the experience already collected. On 749 curated software-engineering tasks and six text-adventure games, it retains at least 64.8 percent of the in-context learning gains, against 3.8 percent for direct supervised fine-tuning on the same collected experience. Against classical reinforcement-learning baselines it matches performance with at least 9.6 times fewer environment samples. Those figures are the authors' measurements, not mine.

Why it belongs in this edition. This is the Deletion Boundary being crossed deliberately, and in the direction that makes deletion hard. Paper #1 found that in a retrieval-externalised architecture the residual lives in the index, where an administrator can reach it. Experience Distillation takes experience that was sitting safely in context, where it is trivially removable, and moves it into weights, where it is not. That is a real and often correct engineering trade, and it is made almost universally without anyone recording that a deletion property was traded away in exchange for sample efficiency.

Industry Application. Excellent economics, and the number to carry into a planning meeting is 9.6 times fewer environment samples. Then write the other half down. Once an interaction history is distilled into weights, removing that experience is a retraining problem, and any deletion commitment covering it needs to say so before the distillation runs rather than after somebody asks. If the experience distilled includes anything about identifiable people, treat the distillation step as the point where your deletion boundary moved.

N.I.C.E. Scorecard

Key takeaway

• Builder: If in-context gains keep evaporating when the history rolls out of the window, distillation recovers most of them for no new environment cost.
• Enterprise / regulated: Record the trade. Distillation converts a deletable context into a non-deletable weight update. That is a governance event, and the moment to document it is before the run.

#3: From Memory to Skills: Evidence-Grounded Co-Evolution Governance for Long-Horizon LLM Agents

Links: arXiv:2607.16621 (submitted 18 July 2026, cs.CL). Authors: Bo Tang, Yang Zhang, Guomian Zhuang, Wenqiang Wei, Gaoyang Zheng, Lindong Xie, Yanchao Tan, Feiyu Xiong, Qingyu Yang, Edward Chung, Zhiyu Li. No public repository is mentioned in the abstract.

The Problem. Most memory systems retrieve past traces as passive context. The agent reads what it did before and hopes the resemblance is useful. Experience never becomes capability; it stays as recollection the model must reinterpret from scratch every time.

The Aha Solution. MSCE is a training-free framework that organises experience into three tiers: grounded step traces, reusable procedural policies, and declarative environmental cognition. Its sharpest move, and the reason it belongs in a governance edition, is what a crystallised skill is required to carry. When an evidence-backed policy with positive estimated gain becomes a callable skill, it retains evidence links, applicability boundaries, decision guidance, verification rules, and reliability estimates. It also introduces reflection-weighted value backfilling, propagating sparse terminal feedback through dense local self-reflections to produce evidence-calibrated trace values.

That list is a governance artifact wearing engineering clothes. A capability that knows what evidence produced it, where it does not apply, and how reliable it is, is a capability you can audit, scope and revoke. Compare that with the ordinary case, where an agent has got better at something for reasons nobody recorded.

Industry Application. The liftable idea does not require adopting the framework. It is the requirement that a learned capability carry its provenance. If your agent accumulates skills, make each one record what evidence justified it and where it stops applying. That single constraint turns an opaque behavioural drift into something a reviewer can inspect and, when the underlying evidence is retracted, something you can find and remove. Evidence links are also the closest thing in this edition to a deletion handle on distilled capability.

N.I.C.E. Scorecard

Key takeaway

• Builder: Make every learned skill carry the evidence that justified it and the boundary where it stops applying. You will need both the first time one misfires.
• Enterprise / regulated: Evidence links are what let you answer "why does the system do this" about a behaviour nobody explicitly programmed, and what let you remove a capability when its source is retracted.

#4: Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the SSGM Framework

Links: arXiv:2603.11768 (submitted 12 March 2026, cs.AI). Authors: Chingkwun Lam, Jiaxin Li, Lingfei Zhang, Kuo Zhao. Conceptual framework; no public repository is mentioned. Dated honestly: this is a March paper, included because it is the only work here that treats memory governance as an architecture rather than a control.

The Problem. As memory moves from static retrieval databases to dynamic agentic mechanisms, new failure modes appear that the retrieval-efficiency literature does not address: memory corruption, semantic drift, and privacy exposure created by the memory system itself rather than by the model.

The Aha Solution. SSGM's organising move is to decouple memory evolution from execution, enforcing consistency verification, temporal decay modeling and dynamic access control prior to any memory consolidation. The gate sits before the write, not after it. The paper names two failure modes precisely enough to design against: topology-induced knowledge leakage, where sensitive context is solidified into long-term storage as a side effect of how memory is linked, and semantic drift, where knowledge degrades through iterative summarisation.

Both are Deletion Boundary problems in disguise. Topology-induced leakage is how something ends up permanently stored that nobody decided to store. Semantic drift is how the thing you eventually try to delete no longer resembles what you are searching for.

Industry Application. This is a conceptual architecture, so treat it as a design checklist rather than a component. The transferable principle is the gate placement: govern the write, not the read. Most teams filter what comes out of memory, which is the expensive place to do it and the place where the record already exists. If consolidation is gated, the sensitive context never becomes durable in the first place, and deletion never becomes necessary. Every hour spent on retrieval-side filtering is an hour spent guarding a door in a wall you already built around the data.

N.I.C.E. Scorecard

Key takeaway

• Builder: Gate consolidation, not retrieval. Filtering on the way out means the sensitive thing is already stored.
• Enterprise / regulated: Topology-induced leakage is the mechanism by which your system retains something no one approved retaining. Semantic drift is why your deletion search will not find it later.

#5: Evaluating Memory Condensation Strategies for Coding Agents in Data-Driven Scientific Discovery

Links: arXiv:2605.18854 (submitted 13 May 2026, cs.LG). Authors: Renuka Chintalapati, Sid Raskar, Anurag Acharya, Jared Willard, Patrick Emami, Sameera Horawalavithana. No public repository is mentioned. Dated honestly: a May paper, included for a negative result that has not travelled as far as it should.

The Problem. Long-running coding agents overflow their context windows, and practitioners choose between truncation and failure. Many condensation strategies exist, from sliding windows to model-generated summaries, and there was no systematic comparison to guide the choice.

The Aha Solution. Eight condensation strategies, GPT-4o, sixty DiscoveryBench tasks across six scientific domains, 480 evaluations. The headline is a negative result and it is the most immediately bankable finding in this edition: no condenser significantly alters hypothesis quality, while model-based condensers raise token costs by 24 to 94 percent. The cheapest effective move was mechanical rather than clever: masking tool-call outputs produced an 8.6 percent net saving. The optimal choice also varies by domain and task length, which is a polite way of saying there is no default worth defending.

Why it belongs here. Summarisation is the most common way experience is compressed, and it is a lossy, irreversible transformation applied continuously and usually unlogged. It is also the mechanism behind the semantic drift that SSGM names. This paper says the expensive version of that transformation buys you nothing in quality. If you are going to degrade your own audit trail, at least stop paying a premium for the privilege.

Industry Application. Check what your agent framework does by default, because several condense with a model call. If yours does, this is a direct cost line with no measured quality cost to removing it, and masking tool-call outputs is the first thing to try. The governance note is separate and unglamorous: every summarisation pass rewrites the record of what happened. Keep the raw trace if the record has to survive a question later.

N.I.C.E. Scorecard

Key takeaway

• Builder: Check whether your framework condenses with a model call. Evidence says it costs 24 to 94 percent more and does not improve output. Try masking tool-call outputs instead.
• Enterprise / regulated: Every summarisation pass is a lossy rewrite of your evidence. If the trace may be asked about later, retain the raw one.

The Framework: the Deletion Boundary

One line, drawn through your architecture.

On the administrative side, experience lives in a store an operator can reach: retrieval indexes, trace logs, memory records. Deletion here is an operation with a receipt. Paper #1 shows this side is larger than the field assumed, because parametric leakage was near zero and the residual lived in the index. It also shows the failure mode: deleting a record does not delete what its near neighbours collectively imply.

On the parametric side, experience has been folded into weights or into distilled capability. Deletion here is a retraining problem, which for most organisations means it is not a deletion procedure at all. Paper #2 crosses this line deliberately and profitably, and almost nobody logs the crossing.

Three consequences worth taking into an architecture review:

1. The boundary is set by architecture, before deployment. It is not a policy you can adopt later. Where you let experience accumulate is where your deletion rights end.

2. Every crossing should be a recorded event. Distillation, consolidation, skill crystallisation and summarisation each move something from the administrative side toward the parametric side, or destroy the record of it. Those are governance events, and they currently appear in almost no change log.

3. Verify the boundary rather than asserting it. Paper #1 provides the procedure: ablate retrieval, test, restore, test, attribute the difference. A deletion commitment you have not tested this way is a claim about a system you have not measured.

The uncomfortable version, and the one worth saying plainly: most teams deploying persistent-memory agents have already fixed their deletion boundary, months ago, without discussing it. The architecture decided. Nobody wrote it down.

Every arXiv identifier, title, author list and submission date in this edition was verified against the primary record before publication. Four of the five papers mention no public repository and are capped on Code-reproducibility accordingly, which reflects what I could verify rather than the quality of the work. N.I.C.E. scores are my editorial assessment and are not the authors' own.

© 2026 Nabeel Khan. The Weekly Paperclip W4 — Where Agent Experience Goes is published under CC BY-NC-ND 4.0. Quote it, cite it, do not repackage it.