Six steps of one real run, with the file each one actually produced. Everything below is copied verbatim from examples/expense-demo/qaia-journey/, which is kept in the repository precisely so this page cannot be a mockup.
simulated: <default applied>, so nobody actually arbitrated anything here. And the acting model had read the file's sequestered judge section beforehand. All three are stated in the run's own journey file. What follows shows the shape of the output. It is not evidence that it works on your ticket.
us-ingest โ writes state/US-004/00-source.md
The source is captured verbatim, with the checks logged rather than assumed: redaction scan, control-character sanitisation, and an untrusted-input check โ because a requirement document is data, not instructions.
- **Redaction scan**: no direct personal/sensitive data found (no real names, SSNs, cards, health data, addresses). No masking applied. - **Untrusted-input check**: the source text contains no directive aimed at the assistant โ it is a plain requirement description. Treated as data throughout. > **As an** employee, > **I want** to submit an expense report and have it approved through the right chain, > **so that** I get reimbursed correctly and the company keeps an auditable trail.
need-understanding โ writes state/US-004/02-understanding.md
This is the step most generators skip, and the one that decides whether the suite is honest. Nine questions came out of eight acceptance criteria. Here is one, unedited:
Q1 โ AC2/AC6: threshold boundary at exactly โฌ500 and exactly โฌ5000.
Why it matters: routes every report; an off-by-one here silently under- or
over-approves.
A close literal reading is self-consistent ("under โฌ500" excludes 500 from band A;
the closed range "โฌ500โโฌ5000" includes both ends; "above โฌ5000" excludes 5000 from
band C) โ so band B would be `[500, 5000]` inclusive on both ends. But this reading
depends on treating two different phrasings ("under X" vs "XโY") as precisely
complementary, which is exactly the kind of boundary phrasing that real
implementations get wrong.
istqb-design โ writes state/US-004/03-design.md
One line per acceptance criterion. The justification is the deliverable โ a technique named without a reason is decoration.
| AC2 | Boundary value analysis + Decision table | Three amount bands with two literal boundaries (โฌ500, โฌ5000) โ BVA; the resulting required-role set per band is a decision table (amount band ร role). | | AC5 | Boundary value analysis | A strict amount threshold (โฌ25) on receipt requirement โ BVA at value/value-1. | | (cross-cutting) | Error guessing / checklist | Authorization and IDOR conditions, anchored on the systematic-expansion checklist (not named per-AC in the source, but a reflex expansion for any authenticated multi-actor workflow). |
That last row is the one worth noticing: nothing in the story mentions authorization. It comes from a systematic expansion pass, not from the text โ and it is the row that later produced a real IDOR finding in the demo application.
prioritize โ writes state/US-004/04-priorities.md
Probability ร impact per condition, proposed. The tool ranks; a human decides. Every ambiguity is carried through with the question that produced it, so a P1 flagged low (Q1) stays visibly unresolved rather than quietly becoming a fact.
testbook-generate โ writes testbooks/US-004/*.feature and coverage-matrix.md
38 scenarios from one story. Stable IDs so a regeneration diffs instead of overwriting, the technique on the tag line, the condition in a comment.
@QAIA-US-004-019 @AC4 @P2 @negative @boundary
# condition: AC4-C3 [req-neg] โ priority P2
Scenario: A line dated 91 days ago is blocked with an explanatory message
Given "employee@demo" has a draft report with one EUR line "supplies" of 10.00
dated 91 days ago, receipt attached
When "employee@demo" submits the report
Then the attempt is refused with a message mentioning "90 days"
| AC | Condition | Scenario ID | Prio | Rationale | Confidence | | AC2 | AC2-C1 | @QAIA-US-004-008 | P1 | Financial-control boundary. | normal | | AC2 | AC2-C2 | @QAIA-US-004-009 | P1 | Q1-flagged; exact-โฌ500 planted- | low (Q1) | | | | | | ambiguity boundary. | | | AC3 | AC3-C1 [req-neg] | @QAIA-US-004-013 | P1 | Classic internal-control defect class. | normal |
Eleven of the 38 rows carry a named open question like that one.
automate โ writes Playwright specs carrying the same IDs
Native Playwright, page objects as fixtures, the scenario ID on the test so a failure points back at the requirement.
test('@QAIA-US-004-021 @AC5 @P1 @negative @boundary a line at exactly the receipt
threshold without a receipt is refused', async ({ request }) => {
const t = await apiLogin(request, B, 'employee@demo');
const { id } = await apiCreateDraft(request, B, t, {
lines: [{ category: 'gear', amount: 25.00, date: todayISO(), receipt: false }],
});
const r = await apiSubmit(request, B, t, id);
expect(r.status()).toBe(422);
});
testbook-score, automation-score, aptitude-gate โ a separate, read-only plugin
No producer grades its own output. That rule has repeatedly caught defects the producing skill could not see in itself โ including one where the generated code was perfectly faithful to a test book whose Then was too weak, so the tests passed against the very defect the requirement existed to forbid. A judge looking only at code-vs-book would have missed it too.
Nothing above tells you whether this is worth your time on your ticket โ no QA engineer has ever run it end to end on real work, and that is the project's largest unknown. If you would consider being the first, there is a 15-minute pilot kit built for exactly that, and the only thing asked in return is where it went wrong.
โ Back to the overview ยท Read the full run, every file โ