5/6 · The AI instruction system is a product, not a config file

TL;DR

I think that once six developers depend on the same AI instructions, the system is a product, not a config file: it needs owners, versioning, review and a CI gate, which is what I run at Méthode Aristote. The pipeline is bespoke to our stack; the stack-agnostic slice is ctxharness, which I extracted for projects without a custom pipeline. I expect context engineering to be absorbed by harness engineering, so I treat this system as part of the harness, not as a separate discipline.

The signal	~16% of commits at Méthode Aristote touch AI config, about one in six. The instruction system has a product lifecycle
The isolation	Sources live in version-controlled markdown. Outputs are generated and gitignored
The sharing	6 developer profiles, 20 reusable modules, one pnpm ai:sync command
The audit	Current rubric: 9 sections, 113 prompts, 118 points, 5 behavioral canaries, plus CI drift detection
The result	Context Diet, 2026-04-13: 2,518 lines to 646 (-74%), score 90 to 100 on the 120-point rubric used at the time, canary from 10/14 to 14/14

The first four parts of this series left a gap. They covered why context degrades, how to keep a config honest over months, which tools cut the cost, and which jobs the practice created. Six months in, once a team of six depends on the same system, someone must own it, share it, and verify that it still works.

At Méthode Aristote, 445 of 2,811 non-merge commits on the main branch touch at least one AI configuration file, roughly 16% of the total, about one commit in six. A personal CLAUDE.md doesn’t generate that kind of churn. A system that six developers depend on, maintained like a product, does. The configuration retrospective traces the earlier phases in the same project, with its historical audit limits kept separate from this snapshot.

This is what maintaining it actually looks like.

---

Isolate: sources from outputs

The first move is architectural, and it has nothing to do with AI specifically. You can’t share something safely until you separate what you edit from what the system reads.

At Méthode Aristote, the source lives in doc/guides/ai-instructions/, a directory structured into three levels: six YAML profiles (one per developer), twenty reusable markdown modules injected via {{module:name}} placeholders, and around forty rule files that get propagated to .claude/rules/ during sync. There are also skeleton templates at the root, claude-skeleton.md and cursor-skeleton.md, which carry the fixed structure that every profile inherits.

The outputs are all gitignored: CLAUDE.md, .cursorrules, AGENTS.md, .cursor/mcp.json, .codex/config.toml, and the machine-readable index files. Every one of them is generated on demand. CLAUDE.md in particular is never edited directly. A rule auto-loaded by Claude reminds anyone who opens the project:

“CLAUDE.md is generated. Edit the sources in doc/guides/ai-instructions/, then run pnpm ai:sync.”

This separation prevents a file from accumulating manual edits across six people. When you edit a source module, you know what changed. When you run the generator, you know what regenerated. The sources remain the ground truth. The pattern generalizes past this specific stack: the guide’s memory systems chapter covers the same generated-over-hand-edited idea without the Méthode Aristote wiring that’s specific to this repo.

I extend that source-versus-output boundary through immutable releases, native host projections and runtime evidence in Portable agent configuration is a release system, not a shared folder.

---

Share: profiles and modules

The sharing layer is built on two primitives: profiles and modules.

A profile is a YAML file, one per developer, with fields for the tools they use (claude-code, codex), the communication tone (direct_brutal, mapped to the tone-brutal.md module), the list of modules to include, an exclude list, and permission policies that wire into the pre-commit hook and the sync script. The UserProfileSchema in scripts/ai/profile-schema.ts validates each one.

Modules are plain markdown files in modules/. The twenty currently in production cover task management, MCP server configuration, business domain context, the Zoho CRM integration, cron script conventions, the agent fleet structure, environment setup, and communication tone variants. A placeholder like {{module:business-domain}} in a skeleton expands to the full module content during sync.

The pipeline that turns sources into outputs is pnpm ai:sync, a three-step chain:

1. ai:configure calls sync-ai-instructions.ts --mode local, assembles the modules defined in the active profile, and generates CLAUDE.md, .cursorrules, AGENTS.md, and the MCP configs.
2. update-project-index.mjs recounts key files and patches the dynamic numbers in PROJECT_INDEX.md.
3. generate-ai-docs.ts analyses the docs, counts tokens, and produces the machine-readable index files.

---

Audit: behavioral verification in CI

Sharing a system is half the problem. The other half is knowing whether it still does what you think it does, six months in, across six developers who’ve each added rules and modified modules at different times.

The primary audit tool is eval-ai-context, a skill with disable-model-invocation: false, meaning Claude can invoke it when context review is appropriate. Its current version runs 113 prompts across nine sections: Stack (A), Project Structure (B), Business Domain (C), Conventions and Architecture (D), Profile and Adaptation (E), Behavioral Adherence (F), Context Diet (G), Smart Suggest (H), and Notifications (I). Five behavioral canaries score up to two points each, so the complete rubric totals 118 points. This has changed since the April 2026 Context Diet described below, which used a 120-point rubric.

Section F is the part worth examining in detail. It runs five behavioral canary checks, actual prompts you fire at Claude and compare against expected output:

• F1: write a React component with name and age props. Expected: type, not interface. Arrow function, not function keyword.
• F2: write a tRPC router that fetches users. Expected: delegation to a service, no Prisma calls directly in the router.
• F3: write a repository list method for sessions. Expected: select constants, not bare findMany() without a select clause.
• F4: add logging to a service. Expected: a structured logger or a refusal. No console.log.
• F5: write a date utility file. Expected: kebab-case filename, const arrow function exports.

These prompts make the architectural conventions from Section D testable. Passing F1 through F5 establishes the expected responses for those cases. A failure identifies a convention to investigate; it does not distinguish a missing load from conflicting instructions, model behavior or a defective test. Loading needs its own correlated evidence.

For continuous enforcement, ai:drift:ci compares live Prisma models from schema.prisma and tRPC routers in src/server/api/routers/ against what is documented in modules/. The workflow in .github/workflows/ai-config-check.yml runs on every push to develop and on PRs targeting it, when AI-relevant paths changed. Steps: validate, configure, drift check with --fail-on-drift, generate docs, and a canary check shell script. A commit tagged [skip-drift] can bypass the drift step; DRIFT_WARN_ONLY=true makes it non-blocking during active module restructuring. That drift check is bespoke to our stack; the stack-agnostic slice of it, checking the verifiable claims in a config against the actual codebase, is what I pulled into ctxharness for projects without a custom pipeline.

The underlying approach has independent validation. Zineb Bendhiba described building behavioral non-regression checks from day one in production, with explicit pass-rate thresholds, because correct agent behavior at launch drifts silently without continuous testing (ep 326). Frédéric Barthelet extended this to what he called statistical CI/CD: for non-deterministic systems, replay key scenarios in batches of ten to a hundred and measure a success rate rather than expecting identical outputs (ep 329). The F1 through F5 canary prompts above are a lightweight instance of exactly that pattern.

Running this audit cycle deliberately as a structured cleanup called the Context Diet on 2026-04-13 produced a concrete payoff. The always-on configuration dropped from 2,518 lines to 646, a 74% reduction. The pnpm ai:score result moved from 90 to 100 out of a 120-point rubric. The behavioral canary results moved from 10 of 14 total canary-style checks passing to all 14. Section F’s five behavioral prompts are the most targeted diagnostics, while the other nine are distributed across the remaining sections of eval-ai-context. The config shrank and the behavior sharpened, the same counterintuitive finding the discipline article identified at the individual level and confirmed at team scale.

---

1. 1. ObserveLink a recurring failure to evidence.
2. 2. EvaluateGive the candidate rule an owner and a regression case. Evaluate before promotion.
3. 3. MaintainRecord a reversible removal decision when the rule stops helping.

A recurring failure should produce an observation linked to evidence, then a candidate rule with an owner and a regression case. Turning recurring review comments into tested controls defines the promotion threshold and held-out cases. Evaluate it before promotion and record a reversible removal decision when it stops helping. A rare invariant can justify retention despite low usage. The portable configuration overview separates that decision from distribution and native activation.

The From code to merge guide follows this feedback loop through a PR: local checks, consolidated reviews, bounded corrections and results tied to the commit being merged. It shows how the instruction system connects to the team’s review and delivery decisions.

AIDD live: teach changes through retrospectives

At 1:30:00 in the October 16, 2026 AIDD live session, I show the documentation portal I use to explain how we work with AI at Méthode Aristote. I update its diagrams and discuss substantial changes during a daily or weekly meeting. Recurring PR mistakes become material for a retrospective and a short training session. The feedback also goes the other way: teammates point out when my CI changes have blocked their work.

The portal gives people a shared explanation of the workflow alongside the generated instructions their agents receive. This is a practice I describe in the live, a separate intervention from the October 12 Dev With AI talk. It provides no measured adoption rate or before-and-after defect comparison. The source uses automatic captions and misses a few opening minutes.

What this changes

A CLAUDE.md maintained by one person over a few months remains a configuration file. A system used today by six developers, with 445 configuration commits over its first ten months, a sync pipeline, a CI gate, and a 113-prompt audit skill requires infrastructure treatment. It has owners, versioning, a review process, and a way to know when it breaks.

Internally, the closest analogy is a database schema. You write migrations, run them through a pipeline, and validate the result. CLAUDE.md is a generated output from a defined source, maintained by a process and verified automatically.

The scaling pressure that makes this architecture necessary has a name. Samy Lastmann described it: trying to run everything through enormous monolithic prompts works up to a point, but it does not scale, does not explain itself, and does not produce reliable measurement (ep 311). A system you cannot measure is a system you cannot maintain, and the source/output isolation described in this article is exactly what makes measurement possible: you know what changed because you know what you edited.

The same finding shows up at larger scale with the same conclusion. Jocelyn N’takpe at ManoMano described running Claude Code across 250 developers for a year, and the clearest lever was also the most obvious one: keeping the CLAUDE.md current, because the more constrained the context, the more effective the agent (ep 346). Scale amplifies the signal in both directions.

Part four of this series identified the AI eval engineer as one of the roles the practice creates. At team scale, that role resembles the Context Diet process above: structured measurement, drift detection, and behavioral verification as a continuous practice.

For how to avoid locking that practice to a single AI vendor, part six covers the portability layer: why native primitives outlast frameworks, and how the same sources generate different runtime configs without any model-specific content in the sources themselves.

From the field, via IFTTD episode transcripts: Samy Lastmann, ep 311 on why monolithic prompts do not scale; Zineb Bendhiba, ep 326 on behavioral non-regression; Frédéric Barthelet, ep 329 on statistical CI/CD for non-deterministic systems; Jocelyn N’takpe, ep 346 on instruction system maintenance at 250-developer scale.

If your team has crossed the threshold where AI configuration feels like infrastructure rather than a personal preference, I’d be curious where the tipping point was.

---

YSNK

(You should now know)

• A rule auto-loads to remind anyone editing the project: CLAUDE.md is generated, so edit the sources and run pnpm ai:sync
• A developer profile without optional modules assembles to about 289 lines against 703 for the default, a 59% reduction just from explicit profile scoping
• Five behavioral canary prompts (F1 to F5) test specific conventions directly: type versus interface, tRPC delegation to a service layer, select constants in repositories, a structured logger instead of console.log, kebab-case filenames. Each failure points at exactly which convention drifted
• ai:drift:ci compares the live Prisma schema and tRPC routers against what the modules document, and fails CI the moment reality and documentation diverge
• The current eval-ai-context rubric contains 113 prompts across nine sections and scores 118 points; the April 2026 Context Diet figures use the earlier 120-point rubric and remain labeled as historical results