Property Based Testing Examples

The order that breaks a JSON round trip is small: an empty SKU, a quantity of zero, a price of zero, and a one-character order ID. Nobody writes that fixture by hand, which is how it survives review. A property test reaches inputs like it on the first run, then reduces any failure around them to the shortest input that still reproduces, so the bug report is one line of data instead of a stack trace full of random strings.

A Working Hypothesis Example

Take a serializer that has to survive a round trip. The property is one assertion: encode then decode returns the original value. The generator mirrors the dataclass constructor, so every generated order is a legal order.

# pip install hypothesis==6.170.15 pytest
import json
from dataclasses import dataclass, asdict
from hypothesis import given, settings, strategies as st

@dataclass(frozen=True)
class Order:
 order_id: str
 sku: str
 quantity: int
 unit_price_cents: int

def encode(order: Order) -> str:
 return json.dumps(asdict(order), separators=(",", ":"))

def decode(payload: str) -> Order:
 raw = json.loads(payload)
 return Order(**raw)

order_strategy = st.builds(
 Order,
 order_id=st.uuids().map(str),
 sku=st.text(alphabet="ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-", min_size=1, max_size=24),
 quantity=st.integers(min_value=1, max_value=10_000),
 unit_price_cents=st.integers(min_value=0, max_value=50_000_000),
)

@settings(max_examples=200, deadline=None)
@given(order_strategy)
def test_encode_decode_roundtrip(order: Order):
 assert decode(encode(order)) == order

# Expected output: 1 passed
# Note: asdict() fixes key order for this test. A production decoder also
# needs unknown-field handling, schema versioning, and a size limit on payloads.

Nothing in that test enumerates cases. st.builds wires the keyword arguments into Order, the SKU alphabet is restricted to characters the warehouse system actually issues, and the price range stops at what the billing table allows. When the assertion fails, Hypothesis reports the smallest order that breaks it and stores the case in its example database, controlled by the database setting in the same profile, so later runs start from the failure instead of hunting for it again.

Generators: The Engine Behind Every Property

A generator produces values of a type, and a property test can only find bugs its generator can express. Hypothesis exposes generators as hypothesis.strategies, imported as st. The primitives cover backend payloads: integers, text, floats, lists, dictionaries, datetimes, uuids. Composition is where they pay off.

st.builds(Cls, **kwargs) constructs objects, which suits dataclasses and plain constructors.
st.lists(item, min_size=, max_size=) bounds collections so no example grows enormous.
st.sampled_from([...]) picks from an enum or a fixed vocabulary of status codes.
st.one_of(a, b) unions domains, which fits discriminated unions in API payloads.
st.map(fn) derives one field from another so two related values stay consistent.

Two rules keep generators honest. Generate valid values only: a generator that emits malformed orders turns a round-trip test into a parser test and buries the real failure in noise. Bound everything: st.lists(st.text()) with no max_size will eventually produce an input large enough to time out, and the timeout says nothing about correctness.

Go gets the same model from Rapid, which documents an imperative API with type-safe data generation through generics and generation biased toward small values and edge cases. That bias changes failure quality: small inputs shrink faster and hit boundary conditions that uniform sampling walks past. gopter splits the same job across packages, with gen for generators and shrinkers, prop for building properties from a condition function, arbitrary for composite types, and commands for stateful tests.

Shrinking: Turning Random Failures into Small Ones

Random inputs produce unreadable failures unless something reduces them. The shrinker searches for a simpler input that still fails, repeating until no smaller value reproduces the bug. Rapid documents that a failure is minimized before it is presented. Hypothesis runs shrinking as a phase of the default profile, and because phases is a settings attribute you can reorder or disable it when you need to inspect the shrinker itself.

from hypothesis import given, strategies as st

def paginate(items: list[str], page_size: int) -> list[list[str]]:
 pages = []
 # Bug: the +1 adds a trailing page when len(items) is a multiple of page_size.
 for start in range(0, len(items) + 1, page_size):
 pages.append(items[start:start + page_size])
 return pages

@given(
 items=st.lists(st.text(min_size=1, max_size=8), max_size=20),
 page_size=st.integers(min_value=1, max_value=8),
)
def test_page_count_matches_requests(items, page_size):
 pages = paginate(items, page_size)
 expected = (len(items) + page_size - 1) // page_size
 assert len(pages) == expected

# Falsifying example:
# test_page_count_matches_requests(
# items=['0'],
# page_size=1,
# )
# Note: this property checks pagination shape only. A production suite should
# also assert that the flattened pages equal the input, in order, exactly once.

The output is the return on the whole exercise. A failure that began as a twenty-item list with an eight-slot page size comes back as one item and a page size of one, small enough to read and small enough to paste into a regression test. Replay matters for the same reason: RapidCheck, the C++ library in the same family, encodes a failure as an opaque string that reproduces the minimal case exactly, while Hypothesis offers its own replay path through the example database and the derandomize setting.

Shrinking has limits. Values with no natural ordering do not get smaller, so an opaque session token survives at full length. Properties that read wall-clock time or global state can shrink into something that no longer reproduces. A generator that maps through a lossy function blocks the process too: st.map(lambda n: n * 0 + 7) collapses every integer to 7, and the shrinker has nothing left to simplify.

Comparing the Libraries

Library	Language	Generation API	Shrinking	Stateful testing
Hypothesis 6.170.15	Python	Composable `strategies` module, profiles configured through `settings`	Automatic, runs as a settings phase	Supported; `stateful_step_count` defaults to 50
Rapid	Go	Imperative API with generics, generation biased toward small values and edge cases	Automatic, minimizes before reporting	State machine and model-based testing
gopter	Go	`gen` for generators and shrinkers, `arbitrary` for composite types	Shrinkers ship in `gen` next to generators	`commands` package for command sequences

The trade-off shows up in the shrinking column. Hypothesis and Rapid handle minimization inside the runner, so a five-line property gets a minimized counterexample for free, at the cost of a settings surface you have to learn. gopter keeps shrinkers beside generators, which means more code but exact control over what “simpler” means for a domain type. A team with a house full of custom generators may prefer that control; a team writing its first property will not.

The method predates all three. QuickCheck introduced the pattern of stating a property as executable code and letting a harness generate inputs to refute it, and a 2024 ICSE paper on property-based testing in practice still defines it the same way: executable formal specifications checked by an automated harness. TypeScript projects reach for fast-check to do the same job. Across every option, the questions that decide adoption are narrow: does the library shrink automatically, and can you replay a failure exactly? Read the shrinking section of the docs before the feature list.

Pitfalls That Bite in Production

Four mistakes account for most complaints about property tests, and none of them are about the library.

Testing an oracle instead of a property. Reimplementing the function inside the test proves the two implementations agree, not that either is correct. Properties that earn their place: invariants (a total is never negative), round trips (decode(encode(x)) equals x), idempotence (running the migration twice equals running it once), and equivalence against a slow reference implementation you trust.

Unbounded generators. Collections without max_size and text without a length cap produce inputs that time out. Set both, and keep a deadline configured. deadline=None is reasonable only after you have measured the runtime and decided the slow example is worth it.

Non-deterministic systems inside the property. Real HTTP calls, sleeping threads, and datetime.now() break shrinking, because the failing case stops reproducing. Inject the clock and the client as parameters, then keep the property pure.

Disabling randomness to get a green build. The default profile sets derandomize=False, so each run explores different inputs. That is the feature. Turning it on to stabilize CI removes the coverage you paid for and postpones the failure to a worse moment.

Keeping CI Time Under Control

A large max_examples on every property turns a commit into a long queue. Split the budget by feedback loop: a small profile for pull requests, a deep one for the scheduled build. Hypothesis makes that a configuration file rather than a code change.

# conftest.py, with hypothesis==6.170.15 installed
from hypothesis import settings, HealthCheck

settings.register_profile(
 "ci_fast",
 max_examples=50,
 deadline=2000, # milliseconds; fail when a single example is that slow
 suppress_health_check=[HealthCheck.too_slow],
)
settings.register_profile(
 "nightly_deep",
 max_examples=1000,
 deadline=None,
 stateful_step_count=100,
)

# Select a profile when running the suite:
# pytest --hypothesis-profile=nightly_deep
# Note: profiles do not cap total suite runtime. Track per-test durations and
# move the slowest properties into the nightly job rather than lowering the
# example count for everything.

Run pytest with --hypothesis-profile=ci_fast locally and with --hypothesis-profile=nightly_deep in the scheduled job. In Go, the same split is usually a build tag or an environment variable that selects a different example count before the test binary runs, since the count belongs to each check call rather than to a global profile.

Where Property Tests Fit in a Real Test Suite

Property tests do not replace example-based tests. Examples document intent and lock in regressions you already found; properties cover the input space around them. The practical pattern is to keep the examples, add properties around serialization, money arithmetic, permission checks, and anything with a state machine, then let each shrunk failure become a new example test.

Stateful testing is the highest-value entry point for backend engineers because it exercises what you actually ship: a sequence of operations. Rapid supports state machine and model-based testing directly. gopter builds stateful tests from arbitrary command sequences through its commands package. Hypothesis generates sequences of steps and allows 50 by default before the invariant is checked, which is enough to catch an order that can be cancelled twice or a balance that goes negative after a refund plus a retry.

A first week that works: one round-trip property for the wire format, one invariant for money or quantities, one stateful property for the order or account lifecycle. Run a small example count on pull requests and a large one nightly. The first property takes an afternoon, mostly to learn the strategy API; by the tenth, the generator is the only part that needs thought, and generators get reused across tests.

The libraries are mature against their own design goals. Hypothesis documents its defaults, Rapid advertises automatic shrinking and stateful testing, gopter separates generators, shrinkers, and commands, and RapidCheck keeps replay strings for exact reproduction. None of that supplies the one thing only you can write: a property that is true of your system, stated in code a reviewer can read.

More in-depth coverage from this blog on closely related topics:

Sources and References

Sources cited while researching and writing this article: