Generating code isn't the hard part. Verifying it is.

An AI agent can scaffold a feature, wire up the API, and write the tests before you’ve finished reading the ticket. What it can’t do is tell you whether any of it actually works. The bottleneck in AI-assisted development has already moved: the hard part isn’t generating code — it’s verifying it.

That gap is easy to miss, because a coding agent is very good at producing a convincing sense of completion — green checkmarks, a passing suite, a tidy diff — without any real proof the thing runs. A test can be green and still fail to catch a regression.

Where the gap actually is

Traditional development earns its confidence the slow way: someone writes the test, clicks the button, thinks about the edge case, watches it fail, and fixes it. An agent skips straight to the artifact. The result is tests that exist without verifying — a check that confirms a button is on the page but never clicks it, an assertion that passes because it asserts nothing, a case that only goes green because the test before it left the right state behind.

The framing that keeps us honest: QA is about reducing risk. A test that doesn’t reduce risk doesn’t just add nothing — it adds noise, one more green light you now have to trust and maintain. More tests is not more quality. Quality comes from the engineering judgment behind each check, and that judgment is exactly the part generation quietly skips.

Why off-the-shelf generators fall short

We tried the existing “AI writes your tests” tools before building our own. The same three limits showed up every time:

• No project context. They don’t know your architecture, your conventions, or what your data really looks like — so they write plausible tests for a codebase that isn’t yours.
• An incomplete cycle. Most stop at generating the test. They don’t run it, read the failure, or ask whether the result made sense — the loop ends exactly where verification begins.
• Shallow validation. They can’t tell a test that reduces risk from one that merely avoids failing. Everything green looks equally useful, and none of it is weighed.

Put together, they automate the one step that was never the bottleneck — writing the test — and skip the ones that were: running it, judging it, refining it.

Eval is the missing mechanism

This is where our whole platform starts. Plune treats eval as the mechanism of truth — the loop is Generate → Evaluate → Gate, and the middle word is the one that does the work. Our CLI and eval-action exist to make evaluation cheap enough to run on every change, so a regression turns something red before it ships.

Cairn takes that same idea and moves it inside the loop. Eval isn’t a final grade stapled onto the end — it’s the working method: run the check, analyze the failure, confirm that the tests which pass are passing for a real reason, consider the edge case, isolate the state. The principle underneath is simple: don’t trust the first answer. Each pass gets a little closer to the truth by testing, learning, and refining — not by generating more.

What Cairn actually does

Cairn is our open-source autonomous QA agent — not a replacement for a QA engineer, but the tireless assistant that takes the repetitive verification work off their plate. Instead of stopping at “here are some tests,” it:

• explores the running application to learn the stack and the real flows,
• grounds every locator against the live DOM, so it tests real elements instead of hallucinated selectors,
• generates checks, runs them, and reads what actually happened,
• analyzes the failures and refines its scenarios, iteration after iteration,
• and scores its own run — grounding, locator quality, and a holistic verdict, not just a raw pass count.

The name is the point. A cairn is a stack of stones marking a trail: it isn’t the road itself, but it keeps you from getting lost. The agent doesn’t replace your judgment — it marks the path so your judgment has somewhere to stand.

Why this matters for teams

AI has made features cheap to produce. It has not made them cheap to trust. As generation speeds up, the verification debt behind it piles up — and manual QA can’t stretch to cover that gap on its own. The answer isn’t ten times more tests; it’s a verification layer that lives in the loop and can run, judge, and refine them.

That’s the line Cairn is built to draw — between “looks ready” and “actually works.”

Try it on your own app

npm i -g @plune-ai/cairn
cairn explore --url https://your-app.example.com

Docs: getting started · GitHub.