Chapter 24 — Threads, Processes, Asyncio, and Structured Concurrency

1. Opening problem

A developer says:

Node.js is asynchronous, while Python is synchronous.

Another says:

Python threads are useless because of the GIL.

Both statements are misleading.

Python supports several concurrency models:

• operating-system threads;
• separate processes;
• asyncio tasks and event loops;
• thread and process executors;
• subprocesses;
• optional free-threaded CPython builds.

The correct choice depends on:

• whether work is CPU-bound or I/O-bound;
• library support;
• cancellation requirements;
• shared state;
• fault isolation;
• deployment model;
• serialization cost;
• observability;
• latency and throughput goals.

---

2. Concurrency versus parallelism

Concurrency means multiple tasks make progress during overlapping periods.

Parallelism means tasks execute simultaneously on multiple processing resources.

An event loop can provide concurrency on one thread.

Processes can provide parallelism across CPU cores.

Threads can provide concurrency and, depending on interpreter build and executed code, may provide parallelism.

Do not use the terms interchangeably.

---

3. Workload classification

I/O-bound

Time is spent waiting for:

• network;
• database;
• filesystem;
• subprocess;
• external API;
• queue.

Suitable tools may include:

• asyncio;
• threads;
• async-capable libraries;
• thread pools for blocking APIs.

CPU-bound

Time is spent computing:

• compression;
• image processing;
• parsing;
• numerical algorithms;
• cryptography;
• model inference;
• large transformations.

Suitable tools may include:

• processes;
• native extensions that release the GIL;
• vectorized libraries;
• free-threaded builds when ecosystem support and synchronization permit;
• specialized distributed workers.

Measure real workload behavior.

---

4. Threads

from threading import Thread


def worker(
    name: str,
) -> None:
    print(
        f"Working: {name}"
    )


thread = Thread(
    target=worker,
    args=("A",),
)

thread.start()
thread.join()

Threads share:

• process memory;
• module globals;
• file descriptors;
• heap objects.

Benefits:

• easy access to shared objects;
• useful for blocking I/O;
• lower isolation overhead than processes;
• integration with synchronous libraries.

Risks:

• data races;
• deadlocks;
• shared-state complexity;
• nondeterministic ordering;
• library thread-safety assumptions;
• difficult shutdown.

---

5. The GIL: accurate mental model

In ordinary GIL-enabled CPython, the global interpreter lock generally allows only one thread at a time to execute Python bytecode within an interpreter.

This limits CPU parallelism for pure Python code.

However:

• blocking I/O releases or does not hold the GIL while waiting;
• many native extensions release the GIL during heavy work;
• threads remain useful for I/O-bound work;
• separate processes bypass the shared interpreter lock;
• optional free-threaded CPython builds can run with the GIL disabled.

Therefore, “threads never run in parallel” is not a complete rule.

---

6. Free-threaded CPython

Modern CPython offers optional free-threaded builds where the GIL can be disabled.

Potential benefit:

• Python threads can execute Python code in parallel across cores.

Important qualifications:

• the build may not be the default installation;
• third-party extension support varies;
• an incompatible extension may re-enable the GIL;
• built-in internal locking is not a replacement for application synchronization;
• iterators and shared mutable objects still require careful coordination;
• single-thread performance and memory behavior can differ;
• race conditions previously hidden by the GIL may become visible.

Write thread-safe code using locks and ownership—not assumptions about interpreter internals.

---

7. Thread synchronization

from threading import Lock


class Counter:
    def __init__(self) -> None:
        self._value = 0
        self._lock = Lock()

    def increment(
        self,
    ) -> None:
        with self._lock:
            self._value += 1

    @property
    def value(self) -> int:
        with self._lock:
            return self._value

Even if one operation appears atomic on one CPython build, do not treat undocumented implementation behavior as a stable concurrency contract.

Use:

• locks;
• queues;
• immutable messages;
• ownership transfer;
• higher-level concurrent containers;
• process isolation.

---

8. Deadlocks

with lock_a:
    with lock_b:
        ...

Another thread:

with lock_b:
    with lock_a:
        ...

Each can wait forever for the other.

Prevention:

• consistent lock ordering;
• fewer shared locks;
• short critical sections;
• timeouts where useful;
• message passing;
• avoiding callbacks while holding locks;
• careful reentrancy design.

A lock protects data only when every access follows the same protocol.

---

9. Thread-local versus context-local state

from threading import local


thread_state = local()

Thread-local storage is scoped to a thread.

In async code, many tasks share one thread. Thread-local values can bleed across tasks.

Use ContextVar for task-local asynchronous context.

Choose based on execution model.

---

10. Thread pools

from concurrent.futures import (
    ThreadPoolExecutor,
)


with ThreadPoolExecutor(
    max_workers=10
) as executor:
    futures = [
        executor.submit(
            fetch_url,
            url,
        )
        for url in urls
    ]

    results = [
        future.result()
        for future in futures
    ]

Thread pools are useful for:

• many blocking I/O calls;
• adapting synchronous libraries;
• bounding thread creation;
• simple parallel task submission.

Do not create an unbounded thread per request.

Consider:

• worker count;
• downstream connection limits;
• timeout handling;
• cancellation;
• queue growth;
• context propagation;
• shutdown behavior.

---

11. Processes

from multiprocessing import (
    Process,
)


def calculate(
    value: int,
) -> None:
    print(
        value * value
    )


process = Process(
    target=calculate,
    args=(10,),
)

process.start()
process.join()

Processes have separate memory spaces.

Benefits:

• CPU parallelism;
• isolation;
• bypasses a GIL-enabled interpreter's bytecode limitation;
• worker failure can be contained.

Costs:

• startup overhead;
• inter-process communication;
• serialization;
• duplicated memory;
• platform-specific start methods;
• more complex debugging;
• separate logging and metrics.

---

12. Process pools

from concurrent.futures import (
    ProcessPoolExecutor,
)


with ProcessPoolExecutor() as executor:
    results = list(
        executor.map(
            calculate,
            values,
        )
    )

Tasks and arguments generally need to be serializable according to the process model.

Avoid submitting:

• open connections;
• locks;
• closures with non-serializable state;
• huge objects repeatedly;
• framework request objects.

Send small, explicit data.

---

13. Start methods

Process behavior differs by platform and selected start method.

Common concepts include:

• spawn;
• fork;
• forkserver on supported systems.

Implications:

• imported state;
• open handles;
• thread state;
• startup speed;
• safety;
• need for if __name__ == "__main__":.

Portable entry point:

def main() -> None:
    ...


if __name__ == "__main__":
    main()

Do not assume Unix fork behavior in cross-platform applications.

---

14. Inter-process communication

Options include:

• queues;
• pipes;
• shared memory;
• managers;
• sockets;
• external brokers;
• files or databases.

Message passing is often easier to reason about than shared mutable memory.

Design:

• message schema;
• backpressure;
• failure acknowledgment;
• duplicate delivery;
• process death;
• shutdown;
• serialization cost.

---

15. Asyncio mental model

asyncio uses an event loop to schedule tasks and callbacks.

import asyncio


async def fetch(
    url: str,
) -> bytes:
    ...

Calling:

coroutine = fetch(
    "https://example.com"
)

creates a coroutine object. It does not automatically execute it.

Run a top-level entry point:

asyncio.run(
    main()
)

Await:

result = await fetch(url)

At suitable await points, control returns to the event loop so another task can make progress.

---

16. Cooperative scheduling

Async tasks switch cooperatively at awaits.

This code blocks the event loop:

async def bad() -> None:
    time.sleep(5)

Correct non-blocking sleep:

async def good() -> None:
    await asyncio.sleep(5)

CPU loop also blocks:

async def calculate() -> int:
    total = 0

    for value in range(
        100_000_000
    ):
        total += value

    return total

async def does not make blocking work asynchronous.

---

17. Tasks

task = asyncio.create_task(
    fetch(url)
)

A task schedules a coroutine on the running event loop.

Keep a strong reference when task lifetime matters.

Avoid “fire and forget” without:

• ownership;
• exception observation;
• cancellation;
• shutdown;
• metrics;
• bounded concurrency.

A detached task can outlive the request that created it.

---

18. Sequential versus concurrent awaiting

Sequential:

first = await fetch(
    first_url
)

second = await fetch(
    second_url
)

Concurrent:

first_task = (
    asyncio.create_task(
        fetch(first_url)
    )
)

second_task = (
    asyncio.create_task(
        fetch(second_url)
    )
)

first = await first_task
second = await second_task

Prefer structured constructs such as task groups for related child tasks.

---

19. Task groups

async with asyncio.TaskGroup() as group:
    first_task = (
        group.create_task(
            fetch(first_url)
        )
    )

    second_task = (
        group.create_task(
            fetch(second_url)
        )
    )

Leaving the block waits for child tasks.

If a child fails, task-group semantics coordinate cancellation and propagate failures, potentially through an exception group.

Benefits:

• child lifetime tied to lexical scope;
• failures are not silently lost;
• cancellation is coordinated;
• shutdown is easier to reason about.

This is structured concurrency.

---

20. gather versus TaskGroup

results = await asyncio.gather(
    fetch(first_url),
    fetch(second_url),
)

gather remains useful for collecting known awaitables.

A task group offers stronger lifetime and failure-structure guarantees for nested concurrent work.

Choose based on semantics, not habit.

Questions:

• should one failure cancel siblings?
• do you need task handles before exit?
• how should multiple failures propagate?
• is partial success meaningful?
• who owns cancellation?

---

21. Cancellation

task.cancel()

Cancellation is delivered through CancelledError at an await point.

Cleanup:

async def worker() -> None:
    resource = await acquire()

    try:
        await use(resource)
    finally:
        await resource.close()

Do not broadly catch and swallow cancellation:

try:
    await work()
except BaseException:
    pass

Even catching Exception behavior should be understood for the Python version and cancellation class hierarchy.

When cancellation is caught for cleanup, it should normally be propagated after cleanup.

---

22. Cancellation is not immediate termination

A task can ignore or delay cancellation if it:

• never awaits;
• catches cancellation and does not re-raise;
• blocks in synchronous code;
• performs long CPU work;
• waits in cancellation-insensitive native code.

Design cooperative cancellation points.

For CPU-bound work, use processes, interruptible algorithms, or explicit cancellation checks where appropriate.

---

23. Timeouts

async with asyncio.timeout(
    5
):
    await operation()

Timeout scopes use cancellation internally.

Do not treat timeout as an unrelated wrapper with no effect on inner cancellation behavior.

Questions:

• is the operation idempotent?
• does the downstream request continue?
• is the resource reusable?
• should partial results be discarded?
• how is timeout translated?

---

24. Semaphores and bounded concurrency

Launching 100,000 requests at once can overload:

• memory;
• file descriptors;
• downstream APIs;
• connection pools;
• DNS;
• rate limits.

Bound concurrency:

semaphore = asyncio.Semaphore(
    20
)


async def limited_fetch(
    url: str,
) -> bytes:
    async with semaphore:
        return await fetch(
            url
        )

A queue and worker pool may provide clearer backpressure for large streams.

---

25. Async queues

queue: asyncio.Queue[
    Job
] = asyncio.Queue(
    maxsize=100
)

Bounded queues create backpressure:

await queue.put(job)

when full.

Workers:

async def worker() -> None:
    while True:
        job = await queue.get()

        try:
            await process(job)
        finally:
            queue.task_done()

Shutdown needs design:

• sentinel messages;
• task cancellation;
• queue draining;
• failed job handling;
• retry ownership;
• task-group scope.

---

26. Blocking work from async code

Run a blocking function in a thread:

result = await asyncio.to_thread(
    blocking_operation,
    argument,
)

This keeps the event loop responsive.

It does not make the blocking library cancellable internally.

Cancelling the await does not necessarily stop the underlying thread operation.

For CPU-heavy work, a process pool may be more appropriate in a GIL-enabled build.

---

27. Event-loop thread safety

Most asyncio objects are not designed for arbitrary cross-thread use.

Use documented thread-safe scheduling functions when communicating with an event loop from another thread.

Do not call normal task or future methods from random threads without a documented guarantee.

Async queues are for async tasks, not thread-safe producer-consumer communication.

Use the correct queue type for the model.

---

28. Shared mutable state in asyncio

One thread does not eliminate race conditions.

value = counter
await something()
counter = value + 1

Another task can run during the await and update counter.

Use:

• locks;
• actors or owners;
• immutable messages;
• queues;
• avoiding awaits inside critical read-modify-write sections.

Async races occur at cooperative scheduling points.

---

29. Async locks

lock = asyncio.Lock()


async with lock:
    update_shared_state()

Do not hold an async lock while:

• performing slow external I/O unnecessarily;
• calling unknown callbacks;
• waiting for another lock in inconsistent order;
• doing CPU-heavy work.

Async deadlocks are still deadlocks.

---

30. Exception groups in structured concurrency

Several child tasks may fail.

Task-group exit can propagate grouped failures.

try:
    async with asyncio.TaskGroup() as group:
        group.create_task(
            first()
        )
        group.create_task(
            second()
        )
except* ValidationError as group:
    ...
except* TimeoutError as group:
    ...

Handle only failures for which grouped recovery makes sense.

Do not discard sibling failures while reporting only the first.

---

31. Free-threaded asyncio

Modern free-threaded builds allow broader multithreaded interpreter execution, and current asyncio implementations support use in that environment.

This does not mean one event loop should be manipulated freely from every thread.

Event-loop ownership, task affinity, and documented thread-safe APIs still matter.

Free threading changes parallel execution capability, not the fundamental structured-concurrency principles.

---

32. Selecting the model

Choose asyncio when:

• high numbers of I/O operations;
• async libraries exist;
• structured cancellation matters;
• one service coordinates many network waits;
• backpressure can be modeled asynchronously.

Choose threads when:

• blocking I/O libraries dominate;
• shared memory is useful;
• integration needs synchronous APIs;
• workload is moderate and synchronization is manageable.

Choose processes when:

• CPU-heavy pure Python work;
• fault isolation matters;
• tasks are coarse enough to justify serialization;
• memory separation is acceptable.

Choose a free-threaded build when:

• environment support is confirmed;
• dependencies support it;
• thread-safe design exists;
• benchmarks show value;
• operational tooling supports the build.

Hybrid designs are common.

---

33. Node.js and TypeScript comparison

Node.js commonly uses one JavaScript event loop with asynchronous I/O and worker mechanisms for parallel work.

Python offers a similar event-loop model through asyncio, but Python codebases also commonly use:

• threads for blocking libraries;
• process pools for CPU work;
• synchronous web workers;
• native libraries;
• free-threaded interpreter builds.

In both ecosystems:

• async code can still block;
• promises or tasks need ownership;
• cancellation is not automatic;
• unbounded concurrency is dangerous;
• CPU work needs a parallelism strategy;
• event-loop responsiveness must be measured.

---

34. Observability

Measure:

• task count;
• queue size;
• executor queue depth;
• worker utilization;
• lock wait time;
• timeout rate;
• cancellation rate;
• event-loop lag;
• process CPU;
• per-process memory;
• child-process failures;
• downstream saturation.

Concurrency bugs often appear only under load.

Diagnostics should be designed before incidents.

---

35. Common mistakes

Async wrapper around blocking code

The event loop still blocks.

Creating unlimited tasks

Use bounded concurrency.

Fire-and-forget task

Own lifetime and observe exceptions.

Catching cancellation broadly

Propagate control flow after cleanup.

Assuming GIL means thread safety

It does not define application invariants.

Assuming free threading removes races

It increases the need for explicit synchronization.

Sending huge objects to process workers

Serialization can dominate.

Sharing event-loop objects across threads

Use documented APIs.

Choosing by fashion

Classify and measure the workload.

---

36. English vocabulary

Term	Meaning
concurrency	overlapping progress of multiple tasks
parallelism	simultaneous execution
event loop	scheduler coordinating async tasks and I/O
cooperative scheduling	tasks yield control explicitly
structured concurrency	child-task lifetime tied to a scope
cancellation	request for a task to stop cooperatively
backpressure	slowing producers when consumers are saturated
critical section	code requiring exclusive access
deadlock	tasks wait indefinitely for one another
executor	abstraction scheduling callables on workers

Useful sentences:

• “The coroutine blocks the event loop because it calls a synchronous API.”
• “The task group ties child lifetime to the request scope.”
• “The bounded queue provides backpressure.”
• “The GIL is not an application-level synchronization strategy.”
• “A free-threaded build permits parallel Python execution but does not remove data races.”
• “Process serialization overhead exceeds the computation time.”

---

37. Speaking task

Explain for fifteen minutes:

How should a Python engineer choose among asyncio, threads, processes, and free-threaded execution?

Use one API service example and one CPU-processing example.

---

38. Writing task

Write a 600-word architecture proposal for a service that downloads 20,000 files, verifies hashes, extracts archives, and stores metadata.

---

39. Exercises

Exercise 1

Use a thread pool for blocking URL requests.

Exercise 2

Use a process pool for CPU-heavy hashing.

Exercise 3

Create an asyncio task group that fetches three resources.

Exercise 4

Implement bounded concurrency with a semaphore.

Exercise 5

Design cancellation-safe worker cleanup.

Exercise 6

Identify races in an async read-modify-write operation.

Exercise 7

Compare behavior under GIL-enabled and free-threaded CPython.

---

40. Complete solutions

Solution 1

from concurrent.futures import (
    ThreadPoolExecutor,
)


def fetch_all(
    urls: list[str],
) -> list[bytes]:
    with ThreadPoolExecutor(
        max_workers=20
    ) as executor:
        return list(
            executor.map(
                blocking_fetch,
                urls,
            )
        )

Set worker count according to downstream limits and workload measurements.

Solution 2

from concurrent.futures import (
    ProcessPoolExecutor,
)


def hash_all(
    paths: list[Path],
) -> list[str]:
    with ProcessPoolExecutor() as executor:
        return list(
            executor.map(
                calculate_hash,
                paths,
            )
        )

Make calculate_hash top-level and pass serializable inputs.

Solution 3

import asyncio


async def fetch_three(
    urls: tuple[
        str,
        str,
        str,
    ],
) -> tuple[
    bytes,
    bytes,
    bytes,
]:
    async with asyncio.TaskGroup() as group:
        first = group.create_task(
            fetch(urls[0])
        )
        second = group.create_task(
            fetch(urls[1])
        )
        third = group.create_task(
            fetch(urls[2])
        )

    return (
        first.result(),
        second.result(),
        third.result(),
    )

Solution 4

async def fetch_limited(
    urls: list[str],
    *,
    maximum_concurrency: int,
) -> list[bytes]:
    if maximum_concurrency < 1:
        raise ValueError(
            "maximum_concurrency "
            "must be positive"
        )

    semaphore = asyncio.Semaphore(
        maximum_concurrency
    )

    async def one(
        url: str,
    ) -> bytes:
        async with semaphore:
            return await fetch(
                url
            )

    async with asyncio.TaskGroup() as group:
        tasks = [
            group.create_task(
                one(url)
            )
            for url in urls
        ]

    return [
        task.result()
        for task in tasks
    ]

For very large input streams, use a bounded queue instead of creating one task per URL.

Solution 5

async def worker(
    queue: asyncio.Queue[
        Job
    ],
) -> None:
    resource = await acquire()

    try:
        while True:
            job = await queue.get()

            try:
                await process(
                    resource,
                    job,
                )
            finally:
                queue.task_done()
    finally:
        await resource.close()

Production shutdown must decide whether queued jobs are drained, retried, or abandoned.

Solution 6

Unsafe:

value = state["count"]
await asyncio.sleep(0)
state["count"] = value + 1

Two tasks can read the same value before either writes.

Fix:

async with lock:
    state["count"] += 1

or assign one task exclusive ownership of the state and communicate through a queue.

Solution 7

In a GIL-enabled build, pure Python CPU-bound threads generally do not execute Python bytecode in parallel, though I/O and native extensions can overlap.

In a free-threaded build, Python threads may execute Python code in parallel. The program must still use synchronization for shared state, and dependency support must be verified.

The correct comparison requires benchmarks on the actual interpreter, platform, and libraries.

---

41. Chapter checkpoint

You should now be able to explain:

1. concurrency versus parallelism;
2. workload classification;
3. threads;
4. GIL-enabled CPython;
5. free-threaded CPython;
6. synchronization and deadlocks;
7. thread pools;
8. processes and process pools;
9. start methods and serialization;
10. coroutine and event-loop mental models;
11. tasks;
12. task groups;
13. cancellation;
14. timeouts;
15. semaphores and queues;
16. async race conditions;
17. exception groups;
18. model-selection criteria;
19. Node.js comparison;
20. concurrency observability.