Lecture 09. Preventing Agents from Declaring Victory Too Early

You ask an agent to implement a "password reset" feature. It modifies the database schema, writes the API endpoint, adds an email template, runs unit tests (all pass), and then confidently tells you it's "done." But when you actually try to run it—the password reset link can't be sent (missing email service configuration), the database migration failed partway through (schema inconsistency), and the entire flow has never actually been executed even once.

This feeling is nothing new—like filling out an entire exam, confidently turning it in first, only to fail once the grades come out. Just because the exam is filled in completely doesn't mean the answers are all correct.

This isn't an isolated incident. The classic 2017 ICML paper by Guo et al. demonstrated: modern neural networks are often systematically overconfident—the confidence models report is significantly higher than their actual accuracy. This is also true of AI coding agents: they "feel" they're done, but in reality, they're far from it. Your harness must replace the agent's "feeling" with verification based on external execution.

The Slippery Slope

Premature completion claims almost always follow the same pattern: the code looks fine—syntax is correct, the logic seems reasonable, and static analysis shows no obvious errors. But the harness doesn't mandate comprehensive execution verification, so the agent skips actually running the code or only runs partial tests. It runs unit tests but skips integration tests; it runs tests but doesn't check coverage. In the end, "the code looks fine" is taken as evidence for "the feature is complete." And the exam gets turned in.

Information is lost at every step. From the task spec to code implementation to runtime behavior, every transition can introduce drift, and every skipped verification step compounds the information asymmetry.

The Three-Layer Completion Check

Core Concepts

Premature Completion Claim: The agent asserts a task is complete, but unmet specifications still exist. The core problem: the agent judges based on local, code-level confidence, while system-level correctness requires holistic verification.
Confidence Calibration Bias: The systematic gap between an agent's self-reported completion confidence and its actual completion quality. For complex, multi-file tasks, this bias is strongly positive—the agent is always more confident than its actual capability warrants. Like a student who always overestimates their exam score.
Completion Criteria: A set of explicit, executable evaluation conditions defined in the harness. The agent must satisfy all conditions before claiming completion. "Done" shifts from a subjective judgment to an objective determination.
Verification-Validation Double Gate: The first, verification layer checks "does the code correctly implement the specified behavior"; the second, validation layer checks "does system-level behavior meet end-to-end requirements." Both must pass to be considered complete.
Runtime Feedback Signal: Logs, process state, and health checks from actually running the program. This is the objective basis the harness uses to judge completion quality.
Completion-First Constraint: Verify functional correctness first, then handle performance, and finally address style. Refactoring is forbidden until core functionality is verified.

Passing Unit Tests ≠ Task Complete

This is the most common and also the most dangerous trap. The agent writes the code, runs unit tests, all green, and says "done." But the design philosophy of unit tests—isolating the unit under test and mocking dependencies—is exactly what makes them incapable of catching cross-component issues:

Interface Mismatch: The file path passed by the render process to the preload script is a relative path, but the preload script expects an absolute path. Both of their respective unit tests use mocks and pass. The problem is only caught during end-to-end testing. It's like every musician in a band practicing perfectly alone, only to realize they're playing in different keys when they play together.

State Propagation Errors: A database migration changes a table schema, but the ORM's cache layer still holds cached entries for the old schema. Unit tests provide a fresh mock environment each time, which won't expose this cross-layer state inconsistency.

Environment Dependency: Code works correctly in the test environment (where everything is mocked) but fails in the real environment due to configuration differences, network latency, or unavailable services. Like singing perfectly in the rehearsal room, but running into sound equipment failure on stage.

"Let Me Also Refactor While I'm At It" Is Poison for Completion Assessment

Claude Code has a common behavior pattern: it starts refactoring code, optimizing performance, and improving style before core functionality has passed verification. Knuth's saying, "Premature optimization is the root of all evil," takes on new meaning in the agent scenario—refactoring shifts the boundary between verified and unverified code, potentially breaking earlier code paths that were implicitly assumed correct. It's like recopying your multiple-choice answers into a nicer format before you've finished the math essay questions—not only is it a waste of time, you might also copy something wrong.

Systematic Bias in Self-Assessment

Anthropic discovered a deeper failure pattern in their 2026 research: when an agent is asked to evaluate its own work, it systematically gives overly positive assessments—even when a human observer would clearly consider the quality substandard. This is like asking a student to grade their own exam—they'll always be especially lenient with their own answers.

This problem is especially severe in subjective tasks (such as design aesthetics)—whether a "layout is elegant" is a judgment call, and agents tend to reliably lean positive. Even for tasks with verifiable outcomes, an agent's performance can still be hampered by poor judgment.

The solution isn't to make the agent "more objective"—the same model that generates also evaluates, and inherently tends to be generous with itself. The solution is to separate the "doer" from the "checker." Like a student shouldn't grade their own exam—you need an independent grader.

An independent evaluation agent, specifically tuned to be "picky," is far more effective than letting the agent generate its own self-assessment. Empirical data from Anthropic: Architecture, Runtime, Cost, Does Core Functionality Work? Single Agent (bare run), 20 minutes, $9, No (game entities don't respond to input); Three Agents (planner + generator + evaluator), 6 hours, $200, Yes (the game is fully playable)

This is the same model (Opus 4.5) with the same prompt ("build a retro 2D game editor"). The only difference is the harness—from "bare run" to "planner expands the requirement → generator implements feature by feature → evaluator performs actual click-testing with Playwright."

Source: Anthropic: Harness design for long-running application development

How to Prevent Premature Submission

1. Externalize Completion Assessment

Completion assessment should not be performed by the agent itself. The harness must perform completion validation independently, using runtime signals as input rather than the agent's confidence. Write this explicitly in CLAUDE.md:

## Definition of Done
- Feature complete = end-to-end verification passed, not "code was written"
- Required verification levels:
 1. Unit tests pass
 2. Integration tests pass
 3. End-to-end flow verification passes
- Do not proceed to level 2 if level 1 fails
- Do not proceed to level 3 if level 2 fails

2. Build a Three-Layer Completion Check

Layer 1: Syntax and Static Analysis. Lowest cost, least informative, but must pass. This is the minimum bar—you have to spell the words correctly before we look at anything else.
Layer 2: Runtime Behavior Verification. Execute tests, check application startup, confirm critical paths. This is the core evidence of completion. Writing it isn't enough; it has to actually run.
Layer 3: System-Level Validation. End-to-end testing, integration validation, simulating user scenarios. The last line of defense against premature claims. It doesn't just have to run; it has to run correctly.

3. Design a Good "Red Pen" for Agents

OpenAI introduced a particularly effective pattern in their Codex practices: error messages for agents should include repair instructions. Don't just draw a big red X like a lazy grader; be like a good teacher and write "here's how you should change this" in the margin. Instead of "Test failed", use "Test failed: POST /api/reset-password returned 500. Check that the email service config exists in environment variables. The template file should be at templates/reset-email.html." This kind of specific, actionable feedback lets the agent self-correct without human intervention.

4. Capture Runtime Signals

Effective runtime signals include:

Does the application start successfully and reach a ready state?
Do critical feature paths execute successfully at runtime?
Are database writes, file operations, and other side effects correct?
Are temporary resources cleaned up?

Real-World Case

Task: Implement a user password reset feature. Includes database operations, sending email, and modifying an API endpoint.

Premature submission path: The agent modifies the database schema, writes the API endpoint, adds the email template, runs unit tests (pass), and claims completion. The exam is fully filled out.

Actual deductions: (1) The end-to-end flow was never tested—actually sending and verifying the reset link was never confirmed. (2) The database migration failed after partial execution, causing schema inconsistency. (3) Email service configuration was missing in the target environment.

Harness intervention: Completion validation was enforced—(1) Start the full application to verify the reset endpoint is reachable; (2) Execute the entire reset flow; (3) Verify database state consistency. All defects were found within the session, saving 5-10x the cost of later fixes. The independent grader found the real problems.

Key Takeaways

Agents are often systematically overconfident—confidence calibration bias is an objective fact. Filling out the exam completely doesn't mean you got it right.
Completion assessment must be externalized—the harness verifies independently; don't trust the agent's "feeling." A student can't grade their own exam.
All three validation layers are essential—pass syntax, pass behavior, pass system, progressing one layer at a time.
Error messages should be like a good teacher's red pen—include specific repair steps so the agent can self-correct.
No refactoring until core functionality is verified—the completion-first constraint is key to preventing premature optimization.

Exercises

Design a Completion Validation Function: Design a complete completion-validation process for a task involving database migration and API modification. List the required runtime signals and pass/fail criteria for each. Run it on a real task and record any potential issues it finds.

Measure Calibration Bias: Pick 10 different types of programming tasks and record the agent's self-reported completion confidence versus actual completion quality. Calculate the bias value and analyze its relationship with task complexity.

Multi-Layer Defense Experiment: Run three configurations on the same task set—(a) static analysis only, (b) adding unit testing, (c) full three-layer verification. Compare the rate of premature completion claims and the number of undetected defects.