Evaluate and gate your LLM app.
Plune is the open-source eval platform for teams building with LLMs. Write assertion suites once, run them locally with the CLI, and block regressions on every pull request with the GitHub Action.
One product, two surfaces.
Write your evals once in plune.yaml —
run them locally with the CLI, then gate every pull request in CI.
Test your LLM app from the terminal
The Plune CLI runs an assertion suite against your provider and
reports pass/fail with token cost — 10 assertion types, caching, and a regression
diff between runs.
- uses: plune-ai/eval-action@v1 with: config: plune.yaml fail-on-regression: true
Block regressions on every pull request
eval-action re-runs the same evals on each PR, posts a sticky diff
comment of what changed, and turns the check red on a passed → failed
regression. Mock provider by default — zero cost.
Let an agent write your tests
Cairn is our open-source QA agent. It logs in, explores your app,
designs methodology-based test cases, and generates self-repairing
@playwright/test specs — the front door into the
Plune quality stack.
What’s new at Plune
-
Generating code isn't the hard part. Verifying it is.
AI writes the feature in seconds, then leaves you the real work — proving it actually runs. Why we built Cairn as an autonomous verification layer on top of eval, not one more test generator.
Read post -
We pointed our own QA agent at our own site
Cairn explored plune.ai and caught our product-updates signup silently broken in prod — three config layers deep — plus an honest look at where an agent over-asserts.
Read post - @plune-ai/cli
Introducing Plune CLI
Assertion testing for LLM apps — describe your checks in one plune.yaml, run them locally or in CI, and catch regressions with a diff.
Read post