Lecture 04. Splitting Instructions Across Multiple Files

You've gotten serious about harness engineering — good for you. You created AGENTS.md and stuffed every rule, constraint, and lesson learned you could think of into it. A month later, the file has swollen to 300 lines, two months in it's 450 lines, three months in it's 600 lines. Then you notice the agent's performance is actually getting worse — on a simple bug fix, the agent burns a ton of context wading through unrelated deployment instructions; a critical security constraint buried at line 300 gets ignored entirely; three conflicting code style rules mean the agent picks one at random each time.

This is the "giant instruction file" trap. It's like overpacking a suitcase — everything seems useful, so you cram it all in until the zipper is about to burst. Finding your underwear means dumping the entire bag out. You're carrying a full suitcase, but you're really only using about a third of what's inside.

The Vicious Loop at the Root

The most common vicious loop goes like this: the agent makes a mistake, you say "add a rule to prevent this," you add it to AGENTS.md, it works temporarily, the agent makes another mistake, you add another rule, repeat, and the file balloons out of control.

This isn't your fault. It's a very natural response — "add a rule" every time something goes wrong feels reasonable, just like stuffing one more thing into your bag every time you leave the house "just in case." But the cumulative effect is disastrous. Let's look at exactly what goes wrong.

Context budget gets eroded. An agent's context window is finite. Say your agent has a 200K token window (Claude's standard). A bloated instruction file can eat 10-20K tokens. Still seems like plenty of room? But a complex task might need to read dozens of source files, tool execution results also consume context, and conversation history accumulates. By the time the agent needs to understand the code, the budget is already cramped — just like an overpacked suitcase with no room left for your laptop.

Lost in the middle. The "Lost in the Middle" paper (Liu et al., 2023) clearly demonstrated that LLMs use information in the middle of long texts significantly less effectively than information at the beginning or end. Your AGENTS.md has 600 lines, and line 300 says "all database queries must use parameterized queries" — that's a hard security constraint. But it's buried in the middle, and the agent will almost certainly overlook it. It's like the sunscreen bottle at the bottom of your overpacked suitcase — you know it's there, you dig three times, can't find it, and end up buying another one.

Priority conflicts. The file mixes non-negotiable hard constraints ("never use eval()"), important design guidance ("prefer functional style"), and a specific historical lesson ("fixed a WebSocket memory leak last week, watch for similar patterns"). These three rules carry completely different levels of importance, but they look identical in the file. The agent has no reliable signal to distinguish them — just like a passport and a charging cable jumbled together in a suitcase, with no way to know which is more urgent.

Maintenance decay. Large files are inherently hard to maintain. Outdated instructions are rarely removed — because the consequence of removal is uncertain ("maybe something else depends on this rule?"), while adding a new instruction feels free. The result: the file only ever grows, never shrinks, and the signal-to-noise ratio keeps declining. This is exactly like accumulating technical debt in software.

Contradiction accumulation. Instructions added at different times start to contradict each other — one says "use TypeScript strict mode," another says "some legacy files allow any types." The agent picks one at random to follow each time. It's like your mother saying "dress warmer" and your father saying "don't overdress," and you're standing at the door not knowing who to listen to.

Core Concepts

Instruction Bloat: When an instruction file occupies more than 10-15% of the context window, it starts to crowd out the budget for reading code and reasoning about the task. A 600-line AGENTS.md can consume 10,000-20,000 tokens — that's 8-15% of a 128K window eaten up before the agent even starts.
Lost in the Middle Effect: Liu et al.'s 2023 research demonstrated that LLMs use information in the middle of long texts significantly less effectively than information at the beginning or end. A critical constraint buried at line 300 of a 600-line file has a very high probability of being effectively ignored.
Instruction SNR: The ratio of instructions in a file relevant to the current task. Being forced to read 50 lines of deployment instructions while fixing a bug — that's low SNR.
Routing File: A short entry file whose core function is to point the agent to more detailed documents, not to contain everything itself. 50-200 lines is sufficient.
Progressive Disclosure: Give overview information first, detailed information on demand. Good harness design is like good UI design — don't dump all the options on the user at once.
Priority Ambiguity: When all instructions appear in the same format and location, the agent cannot distinguish non-negotiable hard constraints from soft, suggestive guidance.

Instruction Architecture

How to Split

Core principle: keep frequently needed information within arm's reach, store occasionally needed information further away, and leave behind what you'll never use.

The AGENTS.md entry file stays at 50-200 lines, containing only the most frequently used items — project overview (one or two sentences), the first command to run (make setup && make test), global hard constraints (no more than 15 non-negotiable rules), and links to topic documents (one-line description + when it applies).

# AGENTS.md

## Project Overview
FastAPI Python 3.11 backend, PostgreSQL 15 database.

## Quick Start
- Install: `make setup`
- Test: `make test`
- Full verification: `make check`

## Hard Constraints
- All APIs must use OAuth 2.0 authentication
- All database queries must use SQLAlchemy 2.0 syntax
- All PRs must pass pytest + mypy --strict + ruff check

## Topic Documents
- [API Design Patterns](docs/api-patterns.md) — Required reading when adding an endpoint
- [Database Rules](docs/database-rules.md) — Required when modifying database operations
- [Testing Standards](docs/testing-standards.md) — Reference when writing tests

Each topic document runs 50-150 lines, organized by subject in the docs/ directory or alongside the relevant module. The agent only reads them when needed. It's like packing cubes in a suitcase — one for underwear, one for toiletries, one for cables. Finding something doesn't require dumping out the whole bag.

Some information is better placed directly in the code — type definitions, interface annotations, explanations in config files. The agent naturally sees these while reading the code, no need to duplicate them in instructions.

Every instruction should have a provenance ("why was this rule added?"), a scope condition ("when does this rule apply?"), and an expiration condition ("under what circumstances can this rule be removed?"). Audit regularly, remove outdated, redundant, and contradictory entries. Manage your instructions the way you manage code dependencies — unused dependencies should be removed, or they just slow the system down.

If an instruction absolutely must be in the entry file, put it at the beginning or the end — never in the middle. The "lost in the middle" effect tells us LLMs use information at the extremes significantly better than the center. But the better approach is to move the instruction to a topic document to be loaded on demand.

Both OpenAI and Anthropic implicitly support the split-file approach. OpenAI says entry files should be "short and routing-oriented," Anthropic says information controlling long-running agents should be "concise and high-priority." Both are saying the same thing: don't cram everything into one file. The suitcase needs to be organized, not brutally overstuffed.

Real-World Example

A SaaS team had their AGENTS.md balloon from 50 lines to 600. The content mixed tech stack versions, coding standards, historical bug-fix notes, API usage guides, deployment processes, and individual team members' preferences — the whole suitcase about to burst.

Agent performance began to visibly decline: while fixing simple bugs, the agent spent a lot of context wading through unrelated deployment instructions; the security constraint "all database queries must use parameterized queries" was buried at line 300 and frequently ignored; three conflicting code style rules caused random agent behavior.

The team performed a "suitcase reorganization":

AGENTS.md was trimmed down to 80 lines: only project overview, run commands, and 15 global hard constraints
Created topic documents: docs/api-patterns.md (120 lines), docs/database-rules.md (60 lines), docs/testing-standards.md (80 lines)
Added topic document links in the routing file
Historical notes were converted into test cases or deleted

After the restructuring: the success rate on the same task set rose from 45% to 72%. Compliance with the security constraint rose from 60% to 95% — because it moved from the middle of the file to the top of the routing file, no longer "lost in the middle."

Key Takeaways

"Add a rule" is short-term relief, long-term poison. Before adding a rule, ask: would this rule be better in a topic document? Don't keep stuffing the suitcase.
The entry file is a router, not an encyclopedia. 50-200 lines with just overview, hard constraints, and links.
Take advantage of the "lost in the middle" effect: put important information at the beginning or end; move unimportant information to topic documents.
Manage instruction bloat like technical debt. Audit regularly, every instruction needs a provenance, scope condition, and expiration condition.
After splitting, SNR improves and the agent devotes more of its context budget to actual tasks instead of wading through irrelevant instructions.

Exercises

SNR audit: Take your current entry instruction file and list all instruction items. Pick 5 different common task types and mark whether each instruction is relevant to that task. Calculate the SNR for each task type. Instructions that are noise for most tasks should move to topic documents.

Progressive disclosure restructuring: If you have an instruction file over 300 lines, split it into: (a) a routing file under 100 lines, (b) 3-5 topic documents. Run the same task set (at least 5) before and after, compare success rates.

Verify the lost-in-the-middle effect: In a long instruction file, place an important constraint at the beginning, middle, and end in turn, running the same task set each time (at least 5 runs per position). See if compliance rates differ. You may be surprised how strong the positional effect is.