Here is an exercise I now run with every training cohort that has a real pipeline to look after: we audit their test suite the way we would audit a production service, because that is what it has become. The exercise takes forty minutes, needs no special tooling, and reliably surprises people who believed their pipeline was fine. TestOps, the operating discipline behind automated checks — owners, budgets, measured trustworthiness for the whole verdict stream — did not exist as a named thing when most of us learned testing, so nobody taught us to audit for it. This checklist fixes that. Run it against your own suite, score honestly, and you will know within the hour whether your green builds deserve the confidence you give them.

Why Audit a Suite at All?

Because a suite is infrastructure with opinions. It tells your team what may ship, and if its opinions are noisy, slow or expensive, every other investment in delivery quality inherits the damage. Auditing for TestOps also changes the tone of the conversation: instead of blaming whoever’s merge went red, the team starts asking which part of the verdict service misfired, and that question has an answer you can fix. The clearest framing of the discipline — what it covers, what it deliberately leaves to release management, where the evidence sits — is the Ops Atlas’s TestOps page, and it is worth reading before you score, so the questions below have context. Ready? Pen and the CI dashboard, that’s all you need.

The Ten-Point TestOps Audit

Score one point for every honest “yes”. Anything short of a ten is a to-do list, not a failure.

1. Can you state, from data, how often a red result is wrong? Not a guess — a number someone computed from recorded outcomes.
2. Do you know the usual wait for a verdict? The typical case and the slow tail, separately, in minutes.
3. Is every attempt recorded somewhere queryable? One place where you can pull any check’s history against any commit.
4. Does every new check serve a probation period before it gains the right to block a merge?
5. Does every parked check carry an owner, a ticket and a deadline — and has the parked list actually shrunk this quarter?
6. Does the pre-merge stage finish inside ten minutes on a bad day, not just a good one?
7. Can your team price a single verdict? Runner spend divided by verdicts delivered, computed within the last month.
8. Does a gate run even when the jobs feeding it fail? If a broken pipeline can merge by silence, score zero — this one is a safety item.
9. Is a human accountable for every deletion and every gate rule? Automated healing may be logged, but people own those decisions.
10. Has anybody celebrated a coverage percentage or a raw test count this quarter? Score yes only if the celebration was challenged — those numbers describe effort, not trust.

Scoring It, Honestly

Seven or above, and your suite is already run like a service; the remaining gaps are refinement, and you should say so publicly — teams rarely hear that their testing is in good shape, so the credit lands twice. Three to six is the common band, and it usually comes with a specific texture: some team historically owned the numbers, that person left, and the practice decayed into ritual. Below three, stop optimising and start rebuilding — and be honest about whether operations is even your problem. When a suite is small and quick, when failures are genuine defects, or when the real gap is that too few checks exist, an audit is the wrong tool; the team needs writing help, not process help. And when two or more of the warning signs line up — slow pre-merge stages, reflex re-runs, a trunk that goes red on noise, several teams stacked on one suite, a CI bill under finance’s microscope — the audit will write your business case for you.

The First Ninety Days of TestOps Fixes

If the score shocked you, resist the urge to buy anything. The standard sequence works in three monthly moves:

• Days 1-30: stand up the record — attempt-level results in a queryable store, keyed by check and commit — and compute your baseline for wrong-reds, wait times and spend.
• Days 31-60: build the containment — a proper quarantine registry with owners and expiry dates, plus a probation period so newborn checks cannot block anyone.
• Days 61-90: buy speed and set targets — shard the suite, prune what the baseline proves is dead weight, and publish a service level for verdicts.

Three months, one visible metric each month, and the score moves by itself the next time you run the audit — which, by the way, should go on the calendar as a quarterly habit.

Bringing the Audit Into Your Training Plan

For trainers and team leads, this exercise slots neatly beside the reliability curriculum, and it works best when you have run it on your own suite first — the empathy gap between scoring and being scored is worth the hour it costs: the monitoring and observability training guide covers the measurement instincts the checklist assumes, and the session design in that piece on teaching failure inside DevOps training pairs well with scoring real suites together, because nothing starts a conversation like a shared, honest number. If the audit reveals your team needs deeper reliability skills than a course can give, the advice in finding an SRE trainer to build reliability skills applies almost word for word to finding TestOps help. Audit quarterly, score in the open, and treat a rising score the way it should be treated — as proof that the verdicts your team ships on have earned their authority back.