The Verification Gap#

The core challenge of agentic development isn’t getting the agent to write code. It’s the translation problem: how do you get what’s in your head — the domain knowledge, the business rules, the implicit “everyone knows that” — into a form the agent can act on?

Agents write tests now. They’re good at it. Given a function, they’ll generate assertions that cover branches, check return values, handle edge cases. Tests pass. CI is green. But passing tests don’t tell you whether you’ve built the right thing.

This is the verification gap: the space between syntactic correctness (does the code do what it says) and semantic correctness (does the system accomplish the business goal). Agents can reason about the first. The second requires domain knowledge that has to come from you.

Syntactic vs semantic#

A syntactic test verifies code behavior. Does this function return the expected value? Does this endpoint return 200? Does this component render without crashing? An agent can derive these from the code itself — it reads the implementation, infers the contract, and writes assertions against it.

A semantic test verifies a business invariant. Referral credits should only apply after the return window closes. A tenant’s default workspace must exist before any user can log in. Permissions inherit from parent to child, but explicit denies override inherited allows.

These rules don’t live in the code. They live in your head, in Notion docs, in conversations with your PM. The agent has no access to them. It can’t decide that the return window matters, or that permission inheritance has an override rule. It doesn’t know what “correct” means for your domain.

This isn’t a tooling gap. Integration tests have the same problem — you can hit real databases and real APIs and still test the wrong thing if you don’t know which invariants matter. It’s an encoding problem: how do you take the business semantics in your head and make them executable?

The human’s irreducible job#

Your role shifted. You’re not writing tests anymore — the agent handles that. You’re defining what verifiable means for your domain.

This is harder than writing tests. It requires you to articulate the business rules that are so obvious to you that you’ve never written them down. The ones that live in tribal knowledge, in “everyone knows that,” in the gap between the spec and reality.

The work is making the semantic layer explicit:

  • Name your invariants. “Onboarding is complete” means: tenant exists, schema migrated, welcome email queued, default permissions set, workspace created. Not “signup returns 200.” The strongest invariants don’t need a runtime check at all; they can be encoded directly in the types.
  • Define the boundaries. What’s the difference between “working” and “not broken”? A test that checks the API response is syntactic. A check that the downstream effects actually happened is semantic.
  • Encode, don’t assume. If a business rule isn’t expressed as a runnable check somewhere, it doesn’t exist as far as the agent is concerned. And it will drift.

The bottleneck isn’t the agent’s ability to write code or tests. It’s whether you’ve made the semantic layer explicit enough that anyone — human or agent — can verify against it.

Building a verifiability stack#

Once you’ve named your invariants, you need a way to check them. This is where the harness comes in — not a test suite, but a collection of executable checks that verify business semantics against the running system.

The shape of this stack:

  1. Define invariants — what must be true for this process to be “done”
  2. Encode as runnable checks — scripts, CLI recipes, or API calls that verify real state
  3. Expose as an interface — give the agent (and yourself) a way to run them

A justfile works well here. It becomes an API for your project — a set of named operations the agent can discover, execute, and iterate against. just verify-onboarding encodes your semantic definition of “onboarding works.” The agent doesn’t need to understand why those checks matter. It just needs to run them, see failures, and fix the code until they pass.

Structured logs fit into this stack too. When critical business events happen, emit them as structured data. Then your harness can verify not just that an API returned 200, but that the downstream effects — the job ran, the email queued, the permissions propagated — actually occurred.

The harness is how you close the verification gap. You encode the semantics; the agent does the iteration.

The shift#

The common advice for working with AI agents is “write better prompts” or “give more context.” That’s solving the wrong problem.

The real work of agentic development is translation. The agent is an excellent pair programmer — but only if you can close the gap between the domain knowledge in your head and the executable artifacts the agent operates on. Better prompts don’t do that. Externalizing your semantics into verifiable checks does.

The agent can write the code. It can write the tests. It can run them and iterate. What it can’t do is decide which invariants matter — which business rules constitute “working correctly” for your domain. That’s your job now: not writing code, but translating domain knowledge into something verifiable. The better you are at that translation, the more effective the agent becomes.