Feedback Loops¶
If you remember one pattern, make it this one.
An agent with a goal will produce something plausible. An agent with a goal and a loop will produce something correct — because plausible-but-wrong gets caught by the loop and fed back, automatically, without you in the middle.
The shape¶
Compare:
> Write code to fit the resonance peak. Test it on the synthetic dataset
> in data/synthetic/ where the true parameters are known (truth.json) —
> iterate until you recover all parameters within 2% . Show me the
> residuals plot when you're done. # goal + loop
The first gives you code to review. The second gives you a result to check — the agent already ran the review loop itself, many times, faster than you could once.
Loops you already own¶
Research projects are full of ready-made loops; the skill is noticing them:
| You have | The loop is |
|---|---|
| Unit tests | "Run pytest; iterate until green" |
| Synthetic data with known truth | "Recover injected parameters to N%" |
| A known limiting case | "Reproduce Schwarzschild when a→0" |
| A published figure | "Match Fig. 3 of the paper before extending it" |
| A conserved quantity | "Energy drift < 1e-8 over the integration" |
| A second method | "Symbolic result must match numerics to tolerance" |
The last two are the physicist's superpower: nature hands you invariants, and invariants are free test oracles. Closure tests — inject, recover, compare — turn any simulation into a verification loop.
The "build the loop first" move¶
For anything nontrivial, the professional move is to make the agent build the measuring device before the thing being measured:
> We're going to implement the perturbation solver. FIRST: write the test
> harness — Schwarzschild limit, convergence test under grid refinement,
> and comparison against the tabulated QNM values in tests/data/. Only
> then implement, and iterate until the harness passes.
This is test-driven development wearing a lab coat, and it's the single highest-leverage habit in agentic research: it converts your physics knowledge into an automated judge, and the agent's speed into convergence instead of noise.
Goodhart's demon
Agents optimize what you measure — including by cheating (hardcoding the expected answer, weakening the test). Make the loop hard to game: held-out cases, randomized injections, "do not modify tests/" in CLAUDE.md. Then spot-check. Trust the loop, audit the loop.