Mutation Testing

What it is

Mutation testing measures the quality of your tests, not their coverage. The tool deliberately injects small faults (mutants) into your source — flip a comparison, remove a call, change a constant — then runs your test suite.

If the tests fail, the mutant is killed (good). If they still pass, the mutant survived, and that line is a hole in your suite.

1
   original code ──► mutate ──► mutant
2
        │                          │
3
        │                          ▼
4
        │                    run test suite
5
        │                          │
6
        │              ┌───────────┴───────────┐
7
        │              ▼                       ▼
8
        │          tests fail              tests pass
9
        │          (mutant killed ✓)       (mutant survived ✗)
10
        │
11
        └── mutation score = killed / total mutants

A mutation score near 100% means your assertions actually detect change; line coverage can be 100% while the mutation score is low — that is exactly the gap coverage hides.

When to use it / when to avoid it

Use it for

Critical modules: pricing, auth, money, permissions, parsers
Auditing tests you inherited and do not trust
Finding assertions that run but never check anything
A periodic quality gate, not an every-commit one

Avoid it when

The suite is slow — mutation testing multiplies runtime by the number of mutants
Code is trivial glue, generated, or has no branching to mutate
You run it on the whole repo every commit instead of a scoped subset
You chase a 100% score and start writing tests that only kill noise

Frameworks & tools

Tool	Ecosystem	Notes
Stryker	JS/TS, C#, Scala	Rich reporter, incremental mode, dashboard
PIT (pitest)	Java/JVM	Fast, mature, plugin ecosystem
mutmut	Python	Simple CLI; parallel mutation
cosmic-ray	Python	Pluggable mutators, SQLite session store
cargo-mutants	Rust	Zero-config, respects `cargo test`
go-mutesting	Go	Mutation for Go packages
Mutmut / mutmut2	Python	Community continuation of mutmut

Quick setup

1
npm install -D @stryker-mutator/core @stryker-mutator/vitest-runner
2
npx stryker run

1
pip install mutmut
2
mutmut run
3
mutmut results

1
go install github.com/zimmski/go-mutesting/cmd/go-mutesting@latest
2
go-mutesting ./...

1
<!-- Maven: PIT plugin -->
2
<plugin>
3
  <groupId>org.pitest</groupId>
4
  <artifactId>pitest-maven</artifactId>
5
  <version>1.17.2</version>
6
</plugin>

1
dotnet tool install -g dotnet-stryker
2
dotnet stryker

Example

1
{
2
  "$schema": "./node_modules/@stryker-mutator/core/schema/stryker-schema.json",
3
  "testRunner": "vitest",
4
  "mutate": ["src/**/*.ts", "!src/**/*.test.ts"],
5
  "thresholds": { "high": 90, "low": 80, "break": 70 },
6
  "reporters": ["html", "clear-text", "progress"]
7
}

1
# survive only on code the tests actually exercise
2
mutmut run --paths-to-mutate src/discount.py --tests-dir tests
3
mutmut show      # inspect surviving mutants

1
go-mutesting --exec "go test ./discount/..." ./discount/...
2
# reports mutants that survived the suite

1
<plugin>
2
  <groupId>org.pitest</groupId>
3
  <artifactId>pitest-maven</artifactId>
4
  <version>1.17.2</version>
5
  <configuration>
6
    <targetClasses><param>com.example.discount.*</param></targetClasses>
7
    <mutationThreshold>80</mutationThreshold>
8
  </configuration>
9
</plugin>

1
{
2
  "stryker-config": {
3
    "project": "Discount.csproj",
4
    "thresholds": { "high": 90, "low": 80, "break": 70 }
5
  }
6
}

Anti-patterns & best practices

Do not run it everywhere, every time. Scope it to critical modules and run it nightly or pre-release; it multiplies your test runtime.
Use incremental mode. Stryker and PIT can cache unchanged mutants so repeat runs are cheap.
Prioritise survived mutants with real impact. A surviving mutant in a boundary check matters more than one in a log message.
Do not chase 100%. Equivalent mutants (same behaviour, different code) can never be killed; treat the score as a signal, not a target.
Fix tests, not the tool. If a mutant survives, add or tighten an assertion — never suppress it to raise the number.
Pair with coverage. Coverage tells you what ran; mutation testing tells you what was actually verified.