Skip to content

Realistic Test Data Variance ​

Test datasets should mirror production variance across every dimension a feature filters, sorts, groups, or branches on. Uniform seeds hide bugs.

Why ​

A bug that only shows in varied data is caught before production.

How To Apply ​

  • Vary every dimension a feature acts on: role, status, organization, owner, dates, counts.
  • Factories spread these dimensions across the dataset by design, the same on every run. Tests pin specific values only when the assertion requires them.
  • For any feature that filters or branches on a property, the dataset must contain at least one matching and one non-matching record - otherwise the test can't distinguish "works" from "is a no-op".

Examples ​

  • A role filter that does nothing still looks fine when every seeded user is an admin.
  • A status sort gives the right order when every record has the same status.
  • A "show mine" toggle passes when every record has the same owner.