Zum Hauptinhalt springen

Chapter 28 — Professional Testing with pytest, Fakes, Mocks, and Test Architecture

1. Opening problem​

A test suite reports 98% coverage.

Production still fails because:

  • database rollback is broken;
  • retries duplicate payments;
  • time-zone handling differs on the server;
  • a mock accepts a nonexistent method;
  • tests import source code instead of the built wheel;
  • asynchronous background tasks fail after tests finish;
  • test order changes results.

Coverage does not prove confidence.

Professional testing asks:

  • Which risks are being tested?
  • At what boundary?
  • With which environment?
  • What contract is protected?
  • Can the test fail for the right reason?
  • Does it test behavior or implementation detail?
  • Is the suite deterministic?
  • Are failure and cleanup paths covered?

2. Test layers​

A practical portfolio can include:

Unit tests​

Small scope, fast, no real external infrastructure.

Component tests​

Several application classes integrated with in-memory or local adapters.

Integration tests​

Real database, filesystem, broker, HTTP client, or framework integration.

Contract tests​

Verify assumptions between independently changing components.

Acceptance tests​

Exercise user-visible behavior through a public boundary.

End-to-end tests​

Exercise a deployed or production-like system across major infrastructure.

No single layer is sufficient.

The pyramid, trophy, or other shape matters less than risk coverage and feedback speed.


3. Test behavior, not private implementation​

Fragile:

def test_service_calls_private_helper():
service._normalize.assert_called_once()

Behavioral:

def test_registration_normalizes_email():
user = service.register(
" Steve@Example.com "
)

assert (
user.email
== "steve@example.com"
)

Private refactoring should not break tests when public behavior remains correct.

Interaction assertions are appropriate when the interaction is itself part of the contract, such as:

  • payment charged once;
  • event published after commit;
  • audit entry written;
  • retry count;
  • forbidden external call.

4. Arrange, Act, Assert​

def test_withdraw_reduces_balance():
account = Account(
balance_cents=10_000
)

account.withdraw(
2_500
)

assert (
account.balance_cents
== 7_500
)

Keep the act clear.

Avoid a test that performs many unrelated operations and contains dozens of assertions.

A test can have multiple assertions when they verify one coherent outcome.


5. Test names​

Good:

def test_register_rejects_duplicate_email():
...
def test_timeout_rolls_back_transaction():
...

Poor:

def test_user_1():
...

A name should communicate:

  • situation;
  • action;
  • expected outcome.

Readable names become living documentation.


6. pytest discovery​

Typical files:

tests/
├── test_accounts.py
└── integration/
└── test_postgres_repository.py

pytest discovers tests through configured naming conventions.

Keep configuration in pyproject.toml or another supported configuration file.

Example:

[tool.pytest.ini_options]
testpaths = [
"tests",
]
addopts = [
"--strict-markers",
"--strict-config",
]

Strict configuration helps catch misspelled markers and invalid settings.


7. Assertions​

pytest rewrites assertions to produce rich failure output.

assert actual == expected

Prefer direct assertions over generic boolean messages when pytest can show the difference.

For floating point:

import pytest


assert actual == pytest.approx(
expected,
rel=1e-9,
)

For exceptions:

with pytest.raises(
ValueError,
match="positive",
):
withdraw(-1)

Assert structured exception fields when possible, not only message text.


8. Fixtures​

import pytest


@pytest.fixture
def account() -> Account:
return Account(
balance_cents=10_000
)

Test:

def test_withdraw(
account: Account,
) -> None:
account.withdraw(
2_500
)

assert (
account.balance_cents
== 7_500
)

A fixture provides a defined context.

Good fixtures are:

  • explicit;
  • focused;
  • composable;
  • scoped correctly;
  • safe to tear down.

Do not hide the entire test scenario behind one enormous fixture.


9. Fixture scopes​

Common scopes:

  • function;
  • class;
  • module;
  • package;
  • session.

Default function scope gives strong isolation.

Broader scope can reduce setup time but increases shared-state risk.

Before widening scope, ask:

  • Is the object immutable?
  • Can tests mutate it?
  • Does cleanup fully reset state?
  • Can tests run in parallel?
  • Does ordering affect behavior?
  • Is setup truly expensive?

Performance is not worth nondeterministic tests.


10. Yield fixtures​

@pytest.fixture
def database():
connection = connect_test_database()

try:
yield connection
finally:
connection.close()

Code before yield is setup.

Code after yield is teardown.

The fixture must yield exactly once.

For several independent resources, use safe teardown patterns so a later setup failure does not skip cleanup of earlier resources.

ExitStack can help.


11. Fixture dependency graph​

@pytest.fixture
def repository(
database,
):
return PostgresRepository(
database
)
@pytest.fixture
def service(
repository,
clock,
):
return UserService(
repository=repository,
clock=clock,
)

pytest resolves fixtures as a dependency graph.

Keep dependencies visible in test signatures where practical.

Excessive autouse fixtures create hidden behavior.


12. conftest.py​

Fixtures in conftest.py can be discovered for tests in its directory scope without explicit import.

Use it for shared testing infrastructure.

Avoid:

  • unrelated global fixtures;
  • hidden autouse mutations;
  • business data factories far from tests;
  • plugin-like magic that developers cannot locate.

Local fixtures near tests improve readability when reuse is limited.


13. Factory fixtures​

Instead of one fixed user:

@pytest.fixture
def user_factory():
def create(
*,
email: str = (
"steve@example.com"
),
active: bool = True,
) -> User:
return User(
email=email,
active=active,
)

return create

Test:

def test_inactive_user_is_skipped(
user_factory,
):
user = user_factory(
active=False
)
...

Factories keep test-specific details visible.

Do not create a universal factory with fifty parameters.


14. Parametrization​

import pytest


@pytest.mark.parametrize(
(
"value",
"expected",
),
[
(" A ", "a"),
("B", "b"),
(" c", "c"),
],
)
def test_normalize(
value: str,
expected: str,
) -> None:
assert (
normalize(value)
== expected
)

Parametrization is useful when one behavior should hold for several input-output cases.

Avoid a giant table containing unrelated scenarios and hidden branching.

Use descriptive IDs:

pytest.param(
"",
None,
id="empty-is-missing",
)

15. Property-oriented testing​

Example-based tests check selected cases.

Property-oriented tests check general invariants:

  • normalization is idempotent;
  • serialization round-trips;
  • sorting output is ordered;
  • adding zero preserves money;
  • parser never returns invalid domain values.

Even without a dedicated property-testing library, write loops over generated or boundary cases carefully.

A specialist library can generate broader cases, but the property itself must be meaningful.


16. Temporary paths​

def test_export(
tmp_path: Path,
) -> None:
output = (
tmp_path
/ "report.csv"
)

export_report(
output
)

assert output.read_text(
encoding="utf-8"
).startswith(
"id,"
)

tmp_path provides an isolated pathlib.Path.

Do not write tests into repository directories or use fixed global filenames.


17. Monkeypatching environment and attributes​

def test_load_timeout(
monkeypatch,
) -> None:
monkeypatch.setenv(
"TIMEOUT_SECONDS",
"10",
)

settings = load_settings()

assert (
settings.timeout_seconds
== 10
)

pytest restores the environment change after the test.

Other operations include:

  • setting attributes;
  • deleting attributes;
  • setting mapping entries;
  • changing directory;
  • changing import path.

Use monkeypatch for controlled global-boundary changes, not as the default design.

Dependency injection often produces simpler tests.


18. Patch where looked up​

Suppose:

# service.py
from gateway import send


def notify() -> None:
send()

Patching:

patch(
"gateway.send"
)

may not affect the already-bound service.send.

Patch:

patch(
"service.send"
)

because that is where the function is looked up.

Better design:

def notify(
sender: Sender,
) -> None:
sender.send()

Now no import patch is needed.


19. unittest.mock​

from unittest.mock import (
Mock,
)


sender = Mock()
sender.send.return_value = None

Mocks can:

  • return values;
  • raise exceptions;
  • record calls;
  • assert interactions;
  • expose configured attributes.

Risks:

  • accepting methods that real objects do not have;
  • tests coupled to call sequence;
  • unrealistic behavior;
  • complex setup;
  • false confidence.

Use mocks at narrow boundaries.


20. Specs and autospec​

sender = Mock(
spec=MessageSender
)

or:

from unittest.mock import (
create_autospec,
)


sender = create_autospec(
EmailSender,
instance=True,
)

Specs restrict available attributes.

Autospec can also enforce callable signatures more closely.

This reduces tests passing against nonexistent APIs.

It still does not reproduce semantic behavior.


21. Fakes​

class FakeUserRepository:
def __init__(self) -> None:
self.users: dict[
int,
User,
] = {}

def find(
self,
user_id: int,
) -> User | None:
return self.users.get(
user_id
)

def save(
self,
user: User,
) -> None:
self.users[
user.user_id
] = user

A fake has a lightweight working implementation.

Benefits:

  • readable state;
  • realistic repeated calls;
  • fewer interaction assertions;
  • reusable component tests.

Risks:

  • fake diverges from real infrastructure;
  • transaction semantics may differ;
  • query behavior may be oversimplified;
  • constraints may be missing.

Use contract tests against both fake and real adapter where valuable.


22. Stubs and spies​

A stub returns configured values:

class FixedClock:
def now(
self,
) -> datetime:
return fixed_time

A spy records interactions while performing some behavior.

Terminology varies, but design intent matters:

  • stub → control input;
  • fake → lightweight implementation;
  • mock → interaction-oriented programmable object;
  • spy → observe calls.

Prefer the simplest test double.


23. Time testing​

Bad:

time.sleep(1)

Better:

class Clock(
Protocol
):
def now(
self,
) -> datetime:
...

Test clock:

@dataclass
class FakeClock:
current: datetime

def now(
self,
) -> datetime:
return self.current

Inject time.

This makes tests fast and deterministic.

Also inject:

  • random generators;
  • ID generators;
  • retry delay;
  • external clients;
  • filesystem roots;
  • environment readers.

24. Database tests​

Unit tests with a fake repository do not prove SQL correctness.

Integration tests should verify:

  • migrations;
  • constraints;
  • transaction behavior;
  • isolation;
  • query mapping;
  • time zones;
  • encoding;
  • concurrency;
  • rollback;
  • indexes where performance matters.

Use a real compatible database, not a different in-memory database with different semantics, when database behavior is the risk.


25. Transaction isolation per test​

Common strategy:

  1. begin transaction;
  2. run test;
  3. roll back.

This can be fast.

But it may hide behavior involving:

  • commit hooks;
  • separate connections;
  • transaction boundaries;
  • background workers;
  • database-generated events.

Some tests need committed data and explicit cleanup.

Choose according to the behavior under test.


26. HTTP tests​

Different scopes:

  • service unit test with fake gateway;
  • client adapter test against a mock server;
  • API route test in-process;
  • contract test against provider schema;
  • end-to-end test against deployed environment.

Mocking an HTTP library call does not prove:

  • URL construction;
  • headers;
  • timeout;
  • serialization;
  • retry safety;
  • TLS behavior;
  • response parsing.

Test at the appropriate boundary.


27. Async tests​

Async code needs tests that await completion and own task lifetime.

Questions:

  • Are background tasks awaited or cancelled?
  • Are pending tasks left after the test?
  • Is cancellation tested?
  • Does cleanup run?
  • Are timeouts deterministic?
  • Is event-loop global state shared?

Use the async testing support selected by the project and document plugin requirements.

Avoid real sleep where a controllable clock or event can synchronize behavior.


28. Concurrency tests​

Concurrency tests are difficult because scheduling varies.

Use:

  • barriers;
  • events;
  • controlled fakes;
  • bounded timeouts;
  • repeated stress tests;
  • deterministic ownership;
  • invariant assertions;
  • race detectors or runtime modes where available.

Avoid:

sleep(
0.1
)

as the only synchronization mechanism.

A test that usually passes is not deterministic.


29. Failure-path testing​

Test:

  • parser rejection;
  • dependency timeout;
  • retry exhaustion;
  • rollback failure;
  • partial batch failure;
  • cancellation;
  • cleanup after exception;
  • duplicate events;
  • invalid state transition;
  • missing configuration;
  • corrupted file;
  • empty result;
  • boundary maximums.

Happy-path-only suites create fragile systems.


30. Exception assertions​

with pytest.raises(
InventoryShortageError
) as captured:
inventory.reserve(
product_id="A",
quantity=10,
)
error = captured.value

assert (
error.available
== 3
)

assert (
error.requested
== 10
)

Structured fields are more stable than full message matching.

Use message matching for user-visible or contractually important text.


31. Warning tests​

with pytest.warns(
DeprecationWarning,
match="use new_method",
):
old_method()

Warnings allow migration before removal.

In CI, configure important warnings as errors where appropriate.

Do not globally ignore all warnings.

Third-party warnings may require targeted filters and upgrade plans.


32. Markers​

@pytest.mark.integration
def test_postgres_repository():
...

Register markers in configuration:

[tool.pytest.ini_options]
markers = [
"integration: uses real infrastructure",
"slow: takes significant time",
]

Run:

pytest -m \
"not integration"

Markers support selection, not a substitute for clear directory structure and CI jobs.


33. Test order and isolation​

A test must not rely on another test running first.

Symptoms:

  • suite passes, individual test fails;
  • random order exposes failures;
  • shared singleton state;
  • global cache contamination;
  • reused database rows;
  • environment not restored.

Fix ownership.

Do not solve isolation problems by forcing one order.


34. Test the built artifact​

A source-tree test can pass even when the wheel is missing:

  • modules;
  • templates;
  • metadata;
  • type information;
  • CLI entry points.

Release pipeline:

  1. run fast source tests;
  2. build wheel;
  3. install wheel into clean environment;
  4. run smoke and acceptance tests;
  5. publish immutable artifact.

Packaging correctness is a test concern.


35. Contract tests for fakes​

If FakeUserRepository is used widely, create shared behavioral tests:

def repository_contract(
repository,
) -> None:
user = create_user()

repository.save(
user
)

assert (
repository.find(
user.user_id
)
== user
)

Run against:

  • fake;
  • Postgres adapter;
  • another implementation.

This does not prove every infrastructure detail, but it reduces semantic drift.


36. Snapshot tests​

Snapshot tests compare output with stored expected data.

Useful for:

  • stable serialization;
  • CLI output;
  • generated documents;
  • error reports;
  • schemas.

Risks:

  • blindly accepting large updates;
  • hidden unstable timestamps;
  • unreadable snapshots;
  • implementation-detail coupling;
  • secret leakage.

Review snapshots as carefully as code.


37. Coverage​

Coverage can reveal unexecuted code.

It cannot prove:

  • assertions are meaningful;
  • edge cases are covered;
  • concurrency is safe;
  • dependencies are integrated correctly;
  • behavior matches requirements;
  • mocks are realistic.

Use coverage as a diagnostic, not a quality score.

Branch coverage can expose missing alternatives better than statement coverage alone.


38. Mutation testing​

Mutation testing changes code deliberately and checks whether tests fail.

Examples:

  • invert comparison;
  • remove statement;
  • replace return;
  • alter constant.

Surviving mutations indicate weak assertions or untested behavior.

Mutation testing can be expensive and noisy.

Use it selectively for critical pure logic.


39. Flaky tests​

Common causes:

  • real time;
  • random data without recorded seed;
  • shared infrastructure;
  • async tasks left running;
  • network;
  • test order;
  • fixed ports;
  • weak cleanup;
  • race conditions;
  • environment differences.

Do not normalize retries for flaky tests without investigation.

A retry can hide defects.

Quarantine only with owner, issue, and repair deadline.


40. CI test strategy​

A possible pipeline:

Pull request​

  • formatting or lint;
  • static typing;
  • unit tests;
  • fast component tests;
  • build wheel;
  • wheel smoke test.

Main branch​

  • database integration;
  • API contract tests;
  • supported Python matrix;
  • security scans;
  • selected concurrency tests.

Scheduled​

  • full end-to-end;
  • dependency-upgrade tests;
  • performance trends;
  • stress;
  • mutation tests;
  • less common platform combinations.

Optimize feedback without abandoning risk coverage.


41. TypeScript comparison​

Comparable tools and concepts:

TypeScript ecosystemPython ecosystem
Jest/Vitest fixtures and hookspytest fixtures
parameterized testspytest.mark.parametrize
mocks/spiesunittest.mock or pytest integrations
temporary directoriestmp_path
module mockingpatch or monkeypatch at lookup location
coveragecoverage integrations
contract testsadapter and provider contracts
fake timersinjected clocks or tool-specific support

Python's fixture dependency graph is especially powerful, but hidden fixture magic can become difficult to maintain.


42. Common mistakes​

Coverage target as strategy​

Test risks.

Mock every dependency​

Use fakes and real integrations appropriately.

Patch definition site​

Patch lookup site.

Broad fixture scope​

Preserve isolation.

Autouse everything​

Keep setup visible.

Real sleeps​

Use synchronization or injected time.

Fake database as only database test​

Verify the real adapter.

Tests only against source​

Install the wheel.

Interaction assertions for pure outcomes​

Prefer state and return behavior.

Retry flaky tests silently​

Fix root causes.


43. English vocabulary​

TermMeaning
fixturecontrolled test context or dependency
parametrizationrunning one test with several cases
test doubleobject replacing a collaborator in a test
fakelightweight working implementation
stubobject returning controlled values
mockprogrammable interaction-focused double
isolationindependence from other tests and state
contract testtest shared behavioral expectations
flakinessnondeterministic test behavior
mutation testingevaluating tests by changing production code

Useful sentences:

  • “The mock accepts a method that the real adapter does not expose.”
  • “The fixture scope trades setup cost for shared-state risk.”
  • “The test patches the definition site instead of the lookup site.”
  • “The fake repository requires contract tests against the real adapter.”
  • “The assertion protects a private implementation detail rather than behavior.”
  • “The suite passes against the source tree but not the built wheel.”

44. Speaking task​

Explain for fifteen minutes:

How do you design a Python test suite that gives real confidence rather than only high coverage?

Include test layers, fixtures, fakes, mocks, integrations, concurrency, and artifact testing.


45. Writing task​

Write a 650-word test-strategy proposal for a Python SaaS backend using PostgreSQL, HTTP integrations, asynchronous jobs, and object storage.


46. Exercises​

Exercise 1​

Write unit tests for a money value object.

Exercise 2​

Create a fixture factory for users.

Exercise 3​

Parametrize parser boundary cases.

Exercise 4​

Replace an import patch with dependency injection.

Exercise 5​

Create a fake repository and run a shared contract against it.

Exercise 6​

Test transaction rollback after an infrastructure exception.

Exercise 7​

Design a wheel smoke-test job.


47. Complete solutions​

Solution 1​

def test_money_adds_same_currency():
left = Money(
amount_cents=500,
currency="USD",
)

right = Money(
amount_cents=250,
currency="USD",
)

assert (
left + right
== Money(
amount_cents=750,
currency="USD",
)
)


def test_money_rejects_mixed_currency():
with pytest.raises(
ValueError,
match="Currency mismatch",
):
Money(
500,
"USD",
) + Money(
250,
"EUR",
)

Solution 2​

@pytest.fixture
def user_factory():
sequence = 0

def create(
*,
email: str | None = None,
active: bool = True,
) -> User:
nonlocal sequence
sequence += 1

resolved_email = (
email
or (
f"user-{sequence}"
"@example.com"
)
)

return User(
user_id=sequence,
email=resolved_email,
active=active,
)

return create

Solution 3​

@pytest.mark.parametrize(
(
"raw",
"expected",
),
[
(
" Steve@Example.com ",
"steve@example.com",
),
(
"a@b.de",
"a@b.de",
),
],
)
def test_parse_email_accepts_valid(
raw: str,
expected: str,
) -> None:
assert (
str(parse_email(raw))
== expected
)


@pytest.mark.parametrize(
"raw",
[
"",
"missing-at",
"@domain.com",
"name@",
42,
None,
],
)
def test_parse_email_rejects_invalid(
raw: object,
) -> None:
with pytest.raises(
ValueError
):
parse_email(raw)

Solution 4​

Before:

from payment_gateway import (
charge,
)


def pay(order):
return charge(
order.total
)

After:

class PaymentGateway(
Protocol
):
def charge(
self,
amount_cents: int,
) -> str:
...


def pay(
order: Order,
gateway: PaymentGateway,
) -> str:
return gateway.charge(
order.total_cents
)

Test with a fake gateway instead of patching the module binding.

Solution 5​

class FakeUserRepository:
def __init__(self) -> None:
self._users: dict[
int,
User,
] = {}

def save(
self,
user: User,
) -> None:
self._users[
user.user_id
] = user

def find(
self,
user_id: int,
) -> User | None:
return self._users.get(
user_id
)


def assert_user_repository_contract(
repository,
) -> None:
user = User(
user_id=42,
email=(
"steve@example.com"
),
active=True,
)

repository.save(user)

assert (
repository.find(42)
== user
)

assert (
repository.find(999)
is None
)

Run the contract against the fake and real database adapter.

Solution 6​

def test_failure_rolls_back(
database,
repository,
failing_gateway,
) -> None:
with pytest.raises(
GatewayTimeoutError
):
service = OrderService(
repository=repository,
gateway=failing_gateway,
)

service.create_order(
command()
)

assert (
repository.count_orders()
== 0
)

This should use a real transaction-capable integration environment when rollback behavior is the risk.

Solution 7​

1. Create clean virtual environment.
2. Install build frontend.
3. Build wheel.
4. Create second clean environment.
5. Install only the wheel.
6. Run `python -m package --help`.
7. Run generated CLI command.
8. Import public modules.
9. Load required package resources.
10. Execute a small acceptance test.
11. Verify installed metadata and version.
12. Preserve wheel as the deployable artifact.

48. Chapter checkpoint​

You should now be able to explain:

  1. testing layers;
  2. behavioral tests;
  3. pytest discovery and assertions;
  4. fixtures and scopes;
  5. safe teardown;
  6. fixture graphs and factories;
  7. parametrization;
  8. temporary paths;
  9. monkeypatch;
  10. patch lookup location;
  11. mocks, specs, and autospec;
  12. fakes, stubs, and spies;
  13. time and randomness injection;
  14. database and HTTP integration tests;
  15. async and concurrency testing;
  16. failure-path testing;
  17. warnings and markers;
  18. isolation;
  19. artifact tests;
  20. contract tests;
  21. coverage and mutation testing;
  22. flaky-test management;
  23. CI test architecture.