asa’s notes
← index
harness-engineering · May 12, 2026

Lecture 03. Turning the Repository Into the Single Source of Truth

6 min read

Your team's architectural decisions are scattered across Confluence, Slack, Jira, and a handful of senior engineers' heads. For humans, this works, awkwardly — you can ask a colleague, search chat history, dig through docs. If that fails, you can corner someone in the cafeteria. But for an AI agent, information that isn't in the repository simply doesn't exist.

This isn't an exaggeration. Think about what the agent's actual inputs are: the system prompt and task description, file contents from the repository, and tool execution results. That's it. Slack history, Jira tickets, Confluence pages, and the architectural decision you discussed with a colleague over Friday afternoon coffee — the agent can't see any of it. It can't "go ask someone" or "search chat history." It's an engineer locked inside the repository — anything outside it, it knows nothing about.

So the question becomes: are you going to give this engineer a good map?

What Belongs on the Map

OpenAI says it plainly: information that doesn't exist in the repo doesn't exist for the agent. They call this the "repo is the spec" principle — the repository itself is the highest-authority specification document.

Anthropic's documentation on long-running agents reflects this: persistent state is a necessary condition for long-task continuity. The ability to recover knowledge across sessions directly determines task success rate. And this state must live in the repository — because that's the only stable, accessible store the agent has.

You might think: "My team is small, knowledge lives in people's heads, and it still works fine." Sure, for humans. But if you're using agents, accept this fact: an agent can't ask a person. Everything it needs to know must be written down and placed where it can find it.

This isn't about "writing more documentation." It's about "putting decisive information in the right place." A 50-line ARCHITECTURE.md in the src/api/ folder is ten thousand times more useful than a 500-page design document in Confluence that nobody maintains. It's like a hand-drawn office map taped to your desk versus a beautiful architectural blueprint locked in a filing cabinet — the former is available the instant you need it; the latter is technically superior but useless in the moment.

Knowledge Visibility

How do you check if your map is good enough? Run a "cold-start test": open a brand-new agent session using only the repo's contents, and see if it can answer five basic questions.

If it can't answer, the map has gaps. Where the map is blank, the agent guesses — a wrong guess becomes a bug, an over-cautious guess wastes context. And every new session guesses again. The cost of guessing is always higher than the cost of mapping things correctly in the first place.

Core Concepts

  • Knowledge Visibility Gap: The proportion of total project knowledge NOT present in the repository. The bigger the gap, the higher the agent's failure rate. How much hidden knowledge about this project lives in your head? Count it all, then see how much has made it into the repo — the difference is your visibility gap.
  • System of Record: The code repository is the authoritative source for project decisions, architectural constraints, execution state, and verification standards. The repo has the final say, nowhere else counts. It's like a map marked "road closed" — you won't take that road. But if that information only lives in Nam's head, you have to ask Nam every time.
  • Cold-Start Test: The five questions above. However many it can answer is how complete your map is.
  • Discovery Cost: The context budget an agent burns finding a piece of critical information in the repo. The more hidden the information, the higher the discovery cost, and the less budget remains for the actual task. Burying critical information in a README ten directories deep is like locking a fire extinguisher in a basement safe — it exists, but you can't find it when you need it.
  • Knowledge Decay Rate: The rate at which knowledge entries become stale per unit of time. Documentation that's out of sync with the code is the biggest enemy — worse than no documentation at all.
  • ACID Analogy: Applying database transaction principles (Atomicity, Consistency, Isolation, Durability) to agent state management. We'll expand on this below.

How to Draw a Good Map

Principle 1: Knowledge lives close to the code. A rule about API endpoint authentication belongs near the API code, not buried in one giant global document. Place a short document in each module folder explaining that module's responsibilities, interfaces, and special constraints. It's like library shelf labels — want a history book, go straight to the shelf labeled "History." No need to search the whole library.

Principle 2: Use standardized entry files. AGENTS.md (or CLAUDE.md) is the agent's "landing page." It doesn't need to contain everything, but it must let the agent quickly answer three questions: "What is this project," "How do I run it," and "How do I verify it." 50-100 lines is enough.

Principle 3: Minimal but complete. Every piece of knowledge should have a clear use case. If removing a rule doesn't affect the quality of the agent's decisions, that rule shouldn't exist. But every question from the cold-start test must have an answer. This is a delicate balance — not too much, not too little, just enough.

Principle 4: Update alongside the code. Tie knowledge updates to code changes. The simplest approach: place architecture documentation in the corresponding module folder. When you modify the code, you naturally see the documentation. After a code change, CI can remind you to check whether the documentation needs updating.

Concrete repo structure:

project/
├── AGENTS.md # Entry point: project overview, run commands, hard constraints
├── src/
│ ├── api/
│ │ ├── ARCHITECTURE.md # API layer architectural decisions
│ │ └── ...
│ ├── db/
│ │ ├── CONSTRAINTS.md # Hard constraints on database operations
│ │ └── ...
│ └── ...
├── PROGRESS.md # Current progress: done, in progress, blocked
└── Makefile # Standardized commands: setup, test, lint, check

Managing Agent State With ACID Principles

This analogy comes from database transaction management — you might think it's overcomplicating things, but it actually gives you a very practical framework:

  • Atomicity: Every "logical operation" (e.g., "add a new endpoint and update tests") gets one git commit. If it fails partway through, git stash to roll back. All or nothing — no "half-done."
  • Consistency: Define verification predicates for a "consistent state" — all tests pass, lint reports no errors. The agent runs verification after every operation; inconsistent intermediate states never get committed. It's like a bank transfer — you can't debit without crediting.
  • Isolation: When multiple agents work concurrently, design state files to avoid race conditions. Simplest approach: each agent uses its own progress file, or use git branches for isolation. Two chefs can't season the same pot at the same time — who's responsible when it's too salty?
  • Durability: Critical project knowledge lives in files tracked by git. Temporary state can stay in session memory, but cross-session knowledge must be persisted to files. What's in your head doesn't count — only what's on paper counts.

A Real Transformation Story

A team maintains an e-commerce platform with ~30 microservices. Architectural decisions (inter-service communication protocols, data consistency strategy, API versioning rules) were scattered across: Confluence (partly stale), Slack (hard to search), a few senior engineers' heads (doesn't scale), and scattered code comments (no system).

After introducing AI agents, 70% of tasks required human intervention. Almost every failure involved the agent violating some hidden constraint "everyone knows but nobody wrote down." It's like a new hire that nobody told "you need to post lunch orders in the group chat" — they guess wrong, get scolded, but even after being scolded no one tells them the rule.

The team carried out a transformation:

  • Created AGENTS.md at the repo root with project overview, tech stack versions, and global hard constraints
  • Added ARCHITECTURE.md in each microservice folder describing responsibilities, interfaces, and dependencies
  • Created a centralized CONSTRAINTS.md with hard constraints in clear "MUST/MUST NOT" language
  • Added PROGRESS.md in each service folder tracking current work status

After the transformation: the same agent could answer all critical project questions cold-start, and task completion quality improved significantly.

Key Takeaways

  • Knowledge not in the repo doesn't exist for the agent. Putting critical decisions into the repo is the most fundamental harness investment — draw a good map so you don't get lost.
  • Use the "cold-start test" to assess repo quality: can a fresh session answer five basic questions using only the repo's contents?
  • Knowledge must live close to the code, be minimal but complete, and update alongside the code. It's not about writing more documentation — it's about putting information in the right place.
  • Apply ACID principles to agent state: atomic commits, consistency verification, concurrency isolation, durable critical knowledge.
  • Knowledge decay is the biggest enemy. Documentation out of sync with the code is more dangerous than no documentation — it leads the agent in the wrong direction while it thinks it's right.

Further Reading

  • OpenAI: Harness Engineering
  • Anthropic: Effective Harnesses for Long-Running Agents
  • Infrastructure as Code — Martin Fowler
  • ADR: Architecture Decision Records
  • The Twelve-Factor App

Exercises

Cold-start test: Open a brand-new agent session in your project (no verbal context, only repo contents). Ask it five questions: What is this system? How is it organized? How do I run it? How do I verify it? What's the current progress? Record what it can't answer, then improve the repo until it can.

Quantify knowledge externalization: List all the important decisions and constraints for development work in your project. Mark each as inside or outside the repo. Calculate your knowledge visibility gap (the proportion outside the repo). Plan to bring it below 10%.

ACID assessment: Assess your project's state management using this lecture's ACID analogy. Atomicity — can agent operations be cleanly rolled back? Consistency — is there verification of a "consistent state"? Isolation — do concurrent agents step on each other? Durability — is all cross-session knowledge persisted?