Chapter 12 — The Python Data Model and Special Methods
1. Opening problem
Why does this work?
len(team)
Why can this work?
for member in team:
...
Why can this work?
team_a == team_b
Python operations are connected to object protocols through special methods, often called dunder methods because their names begin and end with double underscores.
Examples:
len(value)→value.__len__();repr(value)→value.__repr__();- iteration →
__iter__and__next__; - equality →
__eq__; - hashing →
__hash__; - call syntax →
__call__; - context management →
__enter__and__exit__.
The Python data model allows user-defined objects to participate naturally in the language.
2. Protocol-oriented design
Python often asks:
What behaviour does this object support?
rather than:
Which framework base class does this object inherit from?
A class with __len__ works with len.
A class with __iter__ works in for.
A class with __enter__ and __exit__ works in with.
This is runtime protocol integration.
Static protocols can document the same behavioural contracts.
3. __repr__
repr() should provide a developer-oriented representation.
class User:
def __init__(
self,
user_id: int,
email: str,
) -> None:
self.user_id = user_id
self.email = email
def __repr__(self) -> str:
return (
"User("
f"user_id={self.user_id!r}, "
f"email={self.email!r}"
")"
)
Use:
print(repr(user))
The !r conversion applies repr to field values.
A good representation is:
- unambiguous;
- concise;
- useful during debugging;
- safe regarding secrets and personal data.
Do not include passwords, tokens, or sensitive identifiers.
4. __str__
str() is a user-oriented representation.
class User:
def __str__(self) -> str:
return self.email
If __str__ is absent, Python usually falls back to __repr__.
Use __str__ when a natural human-facing representation exists.
Do not make it the only detailed debugging representation.
5. Equality with __eq__
class UserId:
def __init__(
self,
value: int,
) -> None:
self.value = value
def __eq__(
self,
other: object,
) -> bool:
if not isinstance(
other,
UserId,
):
return NotImplemented
return self.value == other.value
Why return NotImplemented instead of False for unrelated types?
It allows Python to try the reflected comparison or another appropriate mechanism.
NotImplemented is a special singleton for unsupported operations. It is not the same as raising NotImplementedError.
6. Equality design questions
Before implementing equality, decide:
- Is this an entity or a value object?
- Does identity depend on one ID or every field?
- Can values change after insertion into a set?
- Should subclasses compare equal?
- Is comparison with primitives allowed?
Example value object:
EmailAddress(
"steve@example.com"
)
Two normalized addresses can compare by value.
Example entity:
User(
user_id=42,
email="..."
)
Two user objects may compare by user_id, even if other fields differ.
Equality is domain design, not only syntax.
7. Hashing
Hashable objects can be:
- dictionary keys;
- set elements;
- members of
frozenset.
Rule:
Equal objects must have equal hashes.
class UserId:
def __hash__(self) -> int:
return hash(self.value)
Never base hashing on mutable fields.
Dangerous:
class User:
def __hash__(self) -> int:
return hash(self.email)
If email changes after insertion into a set, lookup behaviour becomes broken.
Value objects used as keys should normally be immutable.
A frozen dataclass is often appropriate:
from dataclasses import dataclass
@dataclass(frozen=True)
class UserId:
value: int
8. Ordering
Implementing only __lt__ may support sorting:
class Priority:
def __init__(
self,
value: int,
) -> None:
self.value = value
def __lt__(
self,
other: object,
) -> bool:
if not isinstance(
other,
Priority,
):
return NotImplemented
return self.value < other.value
Usage:
sorted(priorities)
Use functools.total_ordering carefully:
from functools import total_ordering
@total_ordering
class Priority:
...
It can derive missing ordering methods from __eq__ and one ordering method.
Explicit methods may be clearer in performance-sensitive or subtle domains.
9. __len__
class Team:
def __init__(
self,
members: list[str],
) -> None:
self._members = list(members)
def __len__(self) -> int:
return len(self._members)
Now:
len(team)
works.
__len__ must return a non-negative integer.
Truthiness uses __bool__ first, then __len__ if __bool__ is absent.
10. Iteration with __iter__
from collections.abc import Iterator
class Team:
def __iter__(
self,
) -> Iterator[str]:
return iter(self._members)
Now:
for member in team:
...
Returning the internal list iterator allows read traversal without exposing the list itself.
If iteration order is part of the API, document it.
11. Containment with __contains__
class Team:
def __contains__(
self,
member: object,
) -> bool:
return member in self._members
Now:
"Mika" in team
works.
Without __contains__, Python may fall back to iteration.
Implement __contains__ when a more efficient or semantically precise check is possible.
12. Indexing with __getitem__
class Team:
def __getitem__(
self,
index: int,
) -> str:
return self._members[index]
Now:
team[0]
works.
Supporting slices requires accepting slice:
from typing import overload
class Team:
@overload
def __getitem__(
self,
index: int,
) -> str:
...
@overload
def __getitem__(
self,
index: slice,
) -> tuple[str, ...]:
...
def __getitem__(
self,
index: int | slice,
) -> str | tuple[str, ...]:
result = self._members[index]
if isinstance(index, slice):
return tuple(result)
return result
Do not implement sequence behaviour unless it matches the domain.
13. Callability with __call__
class TaxCalculator:
def __init__(
self,
rate: float,
) -> None:
self._rate = rate
def __call__(
self,
subtotal: float,
) -> float:
return subtotal * (
1 + self._rate
)
Usage:
calculate_total = TaxCalculator(
0.19
)
print(calculate_total(100))
Callable objects combine function syntax with stored configuration.
14. Context managers
class Transaction:
def __enter__(
self,
) -> "Transaction":
self.begin()
return self
def __exit__(
self,
exception_type,
exception,
traceback,
) -> bool:
if exception is None:
self.commit()
else:
self.rollback()
return False
Usage:
with Transaction() as transaction:
update_database()
Returning False means exceptions propagate.
Returning True suppresses the exception.
Suppress only when the context manager fully handles the failure and hiding it is semantically correct.
15. Numeric operators
class Money:
def __init__(
self,
amount: int,
currency: str,
) -> None:
self.amount = amount
self.currency = currency
def __add__(
self,
other: object,
) -> "Money":
if not isinstance(
other,
Money,
):
return NotImplemented
if self.currency != other.currency:
raise ValueError(
"Currency mismatch"
)
return Money(
self.amount + other.amount,
self.currency,
)
Operator overloading should preserve intuitive meaning.
Do not overload operators merely to appear elegant.
Questions:
- Is the operation mathematically or conceptually natural?
- Are errors clear?
- Is mutability surprising?
- Does the result type make sense?
- Does ordering have a stable business meaning?
16. Reflected operations
For:
left + right
Python may try:
left.__add__(right)
If that returns NotImplemented, it may try:
right.__radd__(left)
This allows cooperation between types.
Example:
class Vector:
def __radd__(
self,
other: object,
):
...
Return NotImplemented when the operation is unsupported.
Do not raise TypeError immediately unless the combination is recognized but invalid.
17. In-place operations
+= may call __iadd__.
If unavailable, Python may fall back to normal addition and rebinding.
Mutable class:
def __iadd__(
self,
value: int,
):
self.total += value
return self
Immutable value object:
def __add__(
self,
value: int,
):
return Counter(
self.total + value
)
In-place syntax does not guarantee mutation. The type controls behaviour.
Document mutability expectations.
18. Attribute hooks
__getattr__ runs when normal lookup fails:
class Settings:
def __init__(
self,
values: dict[str, object],
) -> None:
self._values = values
def __getattr__(
self,
name: str,
) -> object:
try:
return self._values[name]
except KeyError as error:
raise AttributeError(
name
) from error
Now:
settings.timeout
can map to dictionary values.
Risks:
- typos become runtime failures;
- static analysis becomes weaker;
- introspection may be confusing;
- internal attribute access can recurse.
__getattribute__ intercepts every attribute lookup and is much more dangerous. Use only with deep understanding.
19. __setattr__
__setattr__ intercepts assignment.
class FrozenAfterInit:
def __setattr__(
self,
name: str,
value: object,
) -> None:
...
This can implement validation or immutability, but properties, dataclasses, and descriptors are often clearer.
Incorrect implementations can cause infinite recursion:
def __setattr__(self, name, value):
self.name = value
This calls __setattr__ again.
Use:
object.__setattr__(
self,
name,
value,
)
when bypassing the override intentionally.
20. Copy protocols
Custom classes may define:
__copy__
__deepcopy__
Use when normal copying cannot preserve semantics.
However, copying complex domain entities is often a design smell.
Questions:
- Should identity be duplicated?
- Should nested resources remain shared?
- Should database sessions be copied?
- Should event history be copied?
- Is a named clone operation clearer?
Prefer explicit domain operations when copying has business meaning.
21. Pickling
Python's serialization protocol can use methods such as:
__getstate__
__setstate__
Pickle is Python-specific and unsafe for untrusted data.
Never unpickle arbitrary external input.
For public data boundaries, prefer explicit formats and schemas such as JSON, message formats, or validated DTOs.
22. Dataclasses and the data model
from dataclasses import dataclass
@dataclass(
frozen=True,
order=True,
)
class Version:
major: int
minor: int
patch: int
The dataclass can generate:
__init__;__repr__;__eq__;- ordering methods;
- hashing depending on configuration.
Generated behaviour must still match domain semantics.
Do not use order=True if versions require special pre-release rules not represented by simple field ordering.
23. TypeScript comparison
JavaScript and TypeScript use symbols and methods for protocols, such as:
[Symbol.iterator]()
Python integrates protocols through special method names.
Python operator and language integration is broader:
- arithmetic;
- callability;
- context management;
- representation;
- hashing;
- containment;
- iteration;
- attribute access.
TypeScript compile-time interfaces describe shape. Python's data model defines runtime behaviour.
24. Common mistakes
__repr__ leaks secrets
Redact sensitive data.
Equality without hashing thought
Equal mutable objects should not be hash keys.
Hash based on mutable data
This breaks sets and dictionaries.
Operators with surprising meaning
Prefer named methods.
Suppressing exceptions in __exit__
Return False unless handling is complete.
Dynamic attribute magic
It weakens tooling and clarity.
Implementing every protocol
Only support operations that match the domain.
Confusing NotImplemented and NotImplementedError
They serve different purposes.
25. English vocabulary
| Term | Meaning |
|---|---|
| data model | rules connecting objects to language behaviour |
| special method | method with a double-underscore protocol name |
| representation | textual form of an object |
| hash | stable integer used by hash collections |
| containment | membership testing |
| reflected operation | operation attempted on the right operand |
| in-place operation | operator intended to update or reuse an object |
| context manager | object controlling setup and cleanup |
| suppression | preventing an exception from propagating |
| descriptor | object controlling attribute access |
Useful sentences:
- “The object participates in the sequence protocol.”
- “Equal values must produce equal hashes.”
- “The representation must not expose secrets.”
- “Returning
NotImplementedallows reflected dispatch.” - “The context manager rolls back and re-raises the exception.”
- “This operator overload does not match user expectations.”
26. Speaking task
Explain for nine minutes:
How the Python data model lets custom objects behave like built-in objects.
Include representations, equality, hashing, iteration, callability, and context management.
27. Writing task
Write a 400-word design review of a mutable Money class that is hashable, allows cross-currency addition, and prints full internal transaction metadata in __repr__.
28. Exercises
Exercise 1
Create an immutable EmailAddress with normalized equality, hashing, safe repr, and user-facing str.
Exercise 2
Create a reusable Team container supporting:
- length;
- iteration;
- membership;
- integer indexing;
- slicing.
Exercise 3
Create a Timer context manager that records elapsed time and never suppresses errors.
Exercise 4
Implement a Vector2D with addition, equality, and developer representation.
Exercise 5
Explain why a mutable object used as a dictionary key is dangerous.
29. Complete solutions
Solution 1
class EmailAddress:
__slots__ = ("_value",)
def __init__(
self,
value: str,
) -> None:
normalized = (
value.strip().lower()
)
if (
"@" not in normalized
or normalized.startswith("@")
or normalized.endswith("@")
):
raise ValueError(
"Invalid email address"
)
self._value = normalized
def __str__(self) -> str:
return self._value
def __repr__(self) -> str:
local, _, domain = (
self._value.partition("@")
)
redacted = (
local[:1]
+ "***@"
+ domain
)
return (
"EmailAddress("
f"{redacted!r}"
")"
)
def __eq__(
self,
other: object,
) -> bool:
if not isinstance(
other,
EmailAddress,
):
return NotImplemented
return self._value == other._value
def __hash__(self) -> int:
return hash(self._value)
The object has no public mutation API, so its hash remains stable.
Solution 2
from collections.abc import Iterator
from typing import overload
class Team:
def __init__(
self,
members: list[str],
) -> None:
self._members = list(members)
def __len__(self) -> int:
return len(self._members)
def __iter__(
self,
) -> Iterator[str]:
return iter(self._members)
def __contains__(
self,
member: object,
) -> bool:
return member in self._members
@overload
def __getitem__(
self,
index: int,
) -> str:
...
@overload
def __getitem__(
self,
index: slice,
) -> tuple[str, ...]:
...
def __getitem__(
self,
index: int | slice,
) -> str | tuple[str, ...]:
result = self._members[index]
if isinstance(index, slice):
return tuple(result)
return result
Solution 3
from time import perf_counter
class Timer:
def __enter__(
self,
) -> "Timer":
self._started = perf_counter()
self.elapsed = 0.0
return self
def __exit__(
self,
exception_type,
exception,
traceback,
) -> bool:
self.elapsed = (
perf_counter()
- self._started
)
return False
Usage:
with Timer() as timer:
perform_work()
print(timer.elapsed)
Solution 4
class Vector2D:
__slots__ = (
"x",
"y",
)
def __init__(
self,
x: float,
y: float,
) -> None:
self.x = float(x)
self.y = float(y)
def __repr__(self) -> str:
return (
"Vector2D("
f"x={self.x!r}, "
f"y={self.y!r}"
")"
)
def __eq__(
self,
other: object,
) -> bool:
if not isinstance(
other,
Vector2D,
):
return NotImplemented
return (
self.x == other.x
and self.y == other.y
)
def __add__(
self,
other: object,
) -> "Vector2D":
if not isinstance(
other,
Vector2D,
):
return NotImplemented
return Vector2D(
self.x + other.x,
self.y + other.y,
)
Solution 5
A dictionary calculates a key's hash to choose a storage location. If the fields used by the hash change, the key may remain stored in the old location while producing a new hash during lookup.
The dictionary may no longer find the key reliably.
Therefore, dictionary keys should have stable equality and hashing for their lifetime.
30. Chapter checkpoint
You should now be able to explain:
- runtime protocols;
reprversusstr;- equality and
NotImplemented; - hash invariants;
- ordering;
- iteration and containment;
- indexing and slicing;
- callability;
- context managers;
- operator overloading;
- dynamic attribute hooks;
- when not to implement a special method.