2/6 · Diagnose and repair context drift

The trap	A CLAUDE.md that works on day one degrades silently. Adherence slides from near-total to mediocre over months
The measure	Adherence is real and observable. It falls with file size and rule count
The diagnostic	Six failure states, from L0 amnesia to L5 infrastructure. They locate a control gap rather than award a score
The loop	Observe a failure, locate its context cause, repair the source, then replay the scenario
The proof	A load signal and repeated behavioral checks show whether the repair held

Three symptoms recur in teams that have run an AI coding agent for more than a few weeks.

After a long session, Claude starts hallucinating imports and people blame the model for having a bad day. Accumulated tool output, compaction, stale facts or poor adherence are possible causes. Elapsed time alone does not show which instructions loaded or remained effective.

Three sprints after an API migration, Claude still generates against the old API. CLAUDE.md is lying about the project’s current state because nobody updated it, and the model faithfully follows that lie.

Rule adherence can slide from near-total to middling over a couple of months, without a signal, alert, or loud break. It gets a little worse each week until someone notices the output has been mediocre for a while.

A good CLAUDE.md on day one is easy. Keeping it from lying about your project three months later demands ongoing work. The first part of this series (the science of why context degrades) explained the pressure. This article is the diagnostic layer: it identifies drift, locates the missing control, and defines the maintenance loop. The L0-to-L5 guide remains the implementation playbook for changing files, adding checks, and moving between levels.

---

Adherence is measurable

CLAUDE.md quality is observable, and the results are unkind to the “dump every rule you can think of” approach.

HumanLayer, running agents in production, landed on a blunt rule from watching adherence degrade: keep instruction files short, a few hundred lines at most, and ideally far less. The pattern is the one everyone feels. A short file gets followed closely. The longer it grows, the more rules the model quietly drops, and you don’t get to choose which ones. The mechanism is the same context rot from part one: a model spreading attention across hundreds of loosely relevant constraints spends more of its capacity filtering than reasoning. This article does not establish a universal rule-count threshold; measure adherence against the tasks and models you actually use.

Position is worth testing too. Lost in the middle motivates checking where important context appears; it does not provide a universal violation-reduction percentage for your CLAUDE.md.

The source of the file matters more than people expect. Gloaguen et al. at ETH Zürich (2026, across 138 benchmarks and 12 repos) compared agent config files written by hand with ones generated by an LLM. The hand-written files improved task success by about 4%. The LLM-generated ones reduced success by about 2% while adding roughly 20% to inference cost. Those results concern the configurations and tasks tested in that study; they do not establish that every generated instruction file is worse than no file.

---

L0 to L5 is a diagnostic, not a destination

After ten months building a production SaaS on Claude Code with a team of four, the same failure states kept recurring. Over that period, 445 non-merge commits of 2,811 total touched AI configuration files, roughly 16% of the activity. When one commit in six touches AI config, the context layer is maintained infrastructure. L0 to L5 names the control present at each state. It does not say that every repository should reach L5, or that a higher label compensates for incorrect instructions. The guide’s context engineering reference carries the operational detail for each transition.

• L0, amnesia.
  • No CLAUDE.md. Claude starts from zero every session, re-learns the project every morning, and forgets it every night.
  • Resembles the METR setup from part one: a capable agent dropped into a cold context each time.
• L1, accumulation.
  • A CLAUDE.md exists and grows. It’s the team wiki where everyone adds a rule and nobody removes one.
  • Useful and well-intentioned, but vulnerable to obsolete or conflicting rules.
• L2, organization.
  • Sections appear. Global rules get separated from project rules. The file is readable.
  • But it’s still one undifferentiated config loaded in full for every task, whether you’re touching the database or editing CSS.
• L3, relevance.
  • Path-scoping arrives. Rules activate only when the files of their domain are in scope. The backend rules don’t exist in the window while you’re in the frontend.
  • Relevance controls reduce unnecessary context; their effect still needs checking.
• L4, verification.
  • You stop assuming the config loaded and start proving it.
  • A load trace and behavioral probes provide different evidence about the active instructions.
• L5, infrastructure.
  • Profiles in YAML generate the configs. A sync command keeps them aligned across tools.
  • CI fails if they drift. Context configuration is now versioned, reviewed, and enforced like any other part of the codebase.

Use the ladder against a concrete failure. A stale API reference points to maintenance debt at L1 or L2. Irrelevant backend rules loaded during frontend work point to missing relevance controls at L2. A rule that may not have loaded points to missing verification at L3. My reading is that many teams operate at L1, with a few good intentions toward L2, but that is not a measured population distribution. The popularity of claude-token-efficient, a terse instruction file with more than 5,700 stars, shows interest in short configurations rather than proving where teams sit.

---

Four signals locate the drift

An oversized root file signals missing relevance controls. At Méthode Aristote, path-scoping reduced always-on context from roughly 8K tokens to 4K, a 40 to 50% reduction with no coverage loss in that setup. The operational directory layout and migration steps belong in the L0-to-L5 guide.

A procedure inside CLAUDE.md signals a category error. Always-on constraints belong in rules, while occasional procedures belong in skills or another on-demand mechanism. A 50-line onboarding sequence loaded during every CSS task adds context without helping that task.

Conflicting implementations across sessions can signal that architectural decisions are stored in the wrong place. CLAUDE.md carries recurring constraints. Decision stores retain choices and their reasons. Retrieval tools such as Serena, grepai, and Context7 supply code or documentation when a task needs them. The memory systems reference describes that separation.

A correct rule that is ignored can signal a load or adherence failure. That case needs observation before another rewrite: confirm which instructions loaded, reproduce the behavior, and compare the result against a named expectation.

---

Load evidence is not behavior evidence

L4 is where this stops being configuration and starts being engineering, because it introduces proof.

A canary check asks for an exact response when a particular instruction applies. An echo is an observed behavior in that session. If the echo is absent, investigate loading, competing instructions and the response itself; silence does not establish that the file never loaded.

I think of L3 as putting on the seatbelt and L4 as the dashboard light confirming that it is buckled. L4 gives you a signal.

When path-scoped rules, subdirectory configs or chained @imports make loading hard to inspect, a configured InstructionsLoaded hook can record instruction-load events. Correlate the file, source and session, including loads that happen after startup. The trace supports a load claim; the canary probes behavior.

A load check narrows one failure mode: whether the runtime received the intended instructions. It does not prove that the model followed them, that the instructions were correct, or that the result satisfies the task. The maintenance loop below separates those claims.

L5 makes the maintenance loop shared. An observed failure becomes a reproducible case. Investigate whether its cause lies in the context, then repair the authoritative source where appropriate and replay the case. Profiles make this scale. One YAML file per role or developer (the guide breaks down what those role profiles contain), and a sync command (pnpm ai:sync) regenerates CLAUDE.md, the Cursor rules, and the Copilot instructions from that single source. The convention lives in one place. The three configs regenerate in seconds. CI fails the build if they drift apart.

The payoff is concrete. When that loop ran a deliberate context cleanup in production, a process we called the Context Diet, the always-on config went from 2,518 lines to 646, a 74% cut, and our internal context quality score (a /120 rubric) went from 90 to 100. The shorter configuration scored better on that local rubric; this before-and-after observation does not establish the cause of every improvement.

The same discipline made a 243-file React migration trivial: 24 agents, around ten files each, 240 files merged automatically and 3 sent to human review, a 1.2% exception rate, finished in about an hour against a three-day projection. The mechanism is worth copying. An Opus planner ran read-only, no writes and no execution, which forced it to produce a complete brief.md spec before anything moved. Each of the 24 agents then ran in its own git worktree, so a broken agent could not touch the others, and the shared brief carried the context none of them would otherwise have had. None of that works without a context the agents can trust.

---

Where context discipline stops helping

This discipline has failure modes, and pretending otherwise is how people over-build.

Over-engineering is a real failure mode. A solo developer on a small project can stop at L2 with a little path-scoping. Profiles, sync, and CI gates create their own waste when added too early.

Context poisoning is the subtle one. A small hallucination works its way into CLAUDE.md, a fake convention or a wrong reference, and from then on it drifts every session silently. It’s more insidious than ordinary context rot because the output looks correct. Generated instructions need review against source evidence; otherwise a hallucination can become a persistent instruction.

Distinguish adherence from loading limits. The memory reference describes CLAUDE.md loading separately from the startup limit for the auto-memory index: 200 lines or 25 KB, whichever comes first. A missing instruction needs investigation of the actual source and load path; neither a long file nor a silent canary identifies the cause by itself.

And there’s a ceiling on the whole thing. As Birgitta Böckeler put it in her context engineering for coding agents piece on martinfowler.com, there are no unit tests for context engineering. You can measure adherence, you can run canary checks, you can score your config, but you cannot prove correctness the way you prove a function correct. This is a discipline of reducing uncertainty, not eliminating it, and treating it as exact is its own trap.

For the factual layer of that gap, automated drift detection gets you partway. ctxharness scans your CLAUDE.md for verifiable claims (file paths, npm scripts, version numbers) and checks each against the actual codebase. It won’t catch a hallucinated convention, but it will catch the path that moved and the script that was renamed, two classes of drift that compound silently in any long-lived project.

---

Maintain adherence as a loop

The diagnostic becomes useful when every observed failure feeds the same maintenance loop.

1. Observe: record the failed behavior, affected files, active task, and instruction you expected the agent to follow.
2. Locate: distinguish a stale fact, irrelevant context, missing instruction, load failure, or behavior that the context layer cannot guarantee.
3. Repair: change the smallest authoritative source. Do not add a global rule for a local exception.
4. Replay: run the same scenario again, then keep the case as a regression check when the failure matters.

A correlated InstructionsLoaded log can establish a file-load event. It cannot establish that every instruction inside the file will be followed. Behavioral checks cover the second claim, and non-deterministic cases need repeated runs with a success-rate threshold. Zineb Bendhiba described continuous non-regression tests with explicit pass thresholds (IFTTD episode 326), while Frédéric Barthelet described replaying key scenarios ten to a hundred times rather than treating one run as conclusive (episode 329).

The six-level context engineering guide turns each diagnosis into concrete file changes and checks. This article defines what to look for and how to keep a repair from decaying again.

The third article in this series leaves the configuration question and goes after cost: the four layers of token reduction, the tools that live in each one, and the blind spot that hides spending from everyone on a Max or Pro subscription. It covers the same discipline from the cost side, once you’ve handled the quality side here.

From the field, via IFTTD episode transcripts: Zineb Bendhiba, ep 326 on behavioral non-regression testing; Frédéric Barthelet, ep 329 on statistical CI/CD for non-deterministic systems.

If your CLAUDE.md has ever lied to you about the state of your own project, I’d like to hear how you caught it, and how long it took.

---

YSNK

(You should now know)

• There is no universal rule-count threshold established here; use named cases to measure adherence
• The cited study compares particular instruction files and tasks; generated content still needs evidence, review and behavioral evaluation
• L0 to L5 locates a missing context control; it is not a score and not every repository needs L5
• Context poisoning is more dangerous than plain rot: a hallucinated convention baked into CLAUDE.md drifts every session and the output still looks correct
• A repair is incomplete until the failing scenario has been replayed and retained as a regression check when its impact warrants one