Lecture 08. Using Feature Lists to Constrain What an Agent Does

You ask an agent to build an e-commerce website. After it finishes, it tells you "done." You look at the code — user authentication works, but the checkout button in the cart does nothing, and the payment flow isn't connected at all. The problem: you never told it what "done" means, so it used its own standard — "I wrote a lot of code and it looks pretty complete."

In many people's eyes, a feature list is just a reminder note — write everything down so you don't forget, then set it aside. But in the world of harnesses, a feature list isn't a note for humans — it's the backbone of the entire harness. The scheduler depends on it to pick tasks, the verifier depends on it to judge completion, the handoff reporter depends on it to generate summaries. Break the backbone and the whole body is paralyzed.

Both Anthropic and OpenAI emphasize: artifacts must be externalized. Feature state must live in a machine-readable file in the repo, not in unstructured conversation text.

The Agent Doesn't Know What "Done" Means

Neither Claude Code nor Codex automatically knows what you mean by "done." You say "add a shopping cart feature," and the model's interpretation might be "write a Cart component and an addToCart method." But what you meant was "users can browse products, add to cart, and complete checkout end-to-end." This understanding gap persists forever without a feature list. The agent uses its own hidden standard — usually "the code has no obvious syntax errors." What you need is end-to-end behavior verification. It's like asking someone to buy some fruit — you say "grab some fruit" and they come back with lemons. Their fruit and your fruit aren't the same kind of fruit.

Look at this common progress note:

Did user auth, cart is mostly done, still need checkout

Can a new agent session answer these questions from this note? What does "mostly done" mean? Which tests has the cart passed? What's blocking checkout? The answer to all of them is "nobody knows." It's like telling a doctor "my stomach hurts, it's been a bit better lately" — what can they prescribe from that?

Result: the new session spends 20 minutes inferring project state, and may re-implement features that are already done. Anthropic's engineering data shows that good progress notes cut session-startup diagnosis time by 60-80%.

The Feature State Machine

Core Concepts

The feature list is a harness primitive: Not an "optional planning tool," but the foundational data structure that every other harness component depends on. Like a database table schema — you can't say "let's just skip the primary key."
The three-part structure: Each feature item is a triple of (behavior description, verification command, current state). Missing any element makes the item incomplete.
The state machine model: Each feature item has four states — not_started, active, blocked, passing. State transitions are controlled by the harness, not freely changed by the agent.
Pass-state gating: The only way a feature moves from active to passing is by successfully executing the verification command. This is irreversible — once passing, it can't go back. Like passing an exam means you passed — you can't retroactively change the grade.
Single source of truth: All information about "what needs to be done" must originate from one feature list. No contradictions between the feature list and conversation history.
Back-pressure: The number of not-yet-passing features is the pressure the harness exerts on the agent. Zero pressure = project complete.

Why the Feature List Must Be a "Primitive"

Documentation is for humans to read; a primitive is for the system to enforce. Documentation can be ignored; a primitive cannot be bypassed.

Think of it like a database trigger constraint vs. an application-layer check: the former is enforced by the database engine, no SQL can skip it; the latter depends on application code correctness and can be inadvertently bypassed. The feature list as a harness primitive serves four specific harness components:

Scheduler: Reads state, picks the next not_started feature. Like a factory production planning system.
Verifier: Executes the verification command, decides whether to allow the state transition. Like quality inspection.
Handoff Reporter: Automatically generates a session handoff summary from the feature list. Like an automated shift report.
Progress Tracker: Counts state distribution, provides project health metrics. Like a dashboard.

How to Do It Right

1. Define a Minimal Feature List Format

You don't need a complex system — a structured Markdown or JSON file is enough. What matters is that every item has the triple:

{
 "id": "F03",
 "behavior": "POST /cart/items with {product_id, quantity} returns 201",
 "verification": "curl -X POST http://localhost:3000/api/cart/items -H 'Content-Type: application/json' -d '{\"product_id\":1,\"quantity\":2}' | jq .status == 201",
 "state": "passing",
 "evidence": "commit abc123, test output log"
}

2. Let the Harness Control State Transitions

The agent cannot directly change a feature's state to passing. It can only submit a verification request; the harness executes the verification command and decides whether to allow the transition. This is "pass-state gating."

3. Write the Rules in CLAUDE.md

## Feature List Rules
- Feature list file: /docs/features.md
- Only one feature active at a time
- Verification command must pass before marking as passing
- Don't modify feature list state yourself — the verification script updates it automatically

4. Calibrate Granularity

Every feature item should have a scope "completable in one session." Too broad and it won't finish; too narrow and management overhead increases. "Users can add items to the cart" is good granularity. "Implement the shopping cart" is too broad. "Create a name field on the Cart model" is too narrow. It's like cutting a steak — not the whole cut, and not ground meat either.

Real-World Case

An e-commerce platform with 10 features. Two tracking approaches compared:

Reminder-note mode: The agent uses unstructured notes. After 3 sessions, the notes become "did user auth and product listing, cart is mostly done but has bugs, checkout hasn't started." A new session needs 20 minutes to infer state, eventually re-implementing already-completed features. It's like your shopping list saying "milk, bread, and that thing" — at the store, you still don't know what to buy.

Backbone mode: Every feature has a clear state and verification command. A new session reads the feature list and in 3 minutes knows: F01-F05 are passing, F06 is active, F07-F10 are not_started. Continues directly from F06, no rework.

Quantitative result: projects using a structured feature list show a 45% higher feature completion rate than free-form tracking, with zero duplicate implementations.

Key Takeaways

The feature list is the harness's backbone, not a note for humans. The scheduler, verifier, and handoff reporter all depend on it.
Every feature item must have the triple: behavior description + verification command + current state. Missing one element makes it incomplete — like a three-legged chair missing a leg.
State transitions are controlled by the harness — the agent cannot change state on its own. Passing verification = the only upgrade path.
The feature list is the project's single source of truth — all "what to do" information originates from one list.
Calibrate granularity to "completable in one session."

Exercises

Design a Feature List: Define a minimal feature list JSON schema. Include: id, behavior description, verification command, current state, evidence reference. Use it to describe a real project with 5 features.

Compare Verification Rigor: Pick 3 features and design both a "loose" verification (e.g., "code has no syntax errors") and a "strict" verification (e.g., "end-to-end test passes"). Compare the false-positive rate under each approach.

Single Source of Truth Audit: Review an existing agent project and check for scope information that conflicts with the feature list (hidden requirements in conversation, TODO comments in code, etc.). Design a plan to consolidate all information into the feature list.