Designing Data-Intensive Applications, on the third attempt

I've started this book three times. The first two attempts stalled around chapter four, in the middle of the log-structured storage discussion, and I put it down thinking it was too dense. That was my mistake, not the book's.

What it actually is

Kleppmann's Designing Data-Intensive Applications gets shelved as a "distributed systems book." It isn't, quite. It's a book about the assumptions you inherit when you pick a database, a queue, or a replication strategy — and what each of those assumptions costs you later.

The structure works in three movements:

1. Foundations — reliability, scalability, maintainability; data models; storage engines.
2. Distributed data — replication, partitioning, transactions, consistency and consensus.
3. Derived data — batch and stream processing, and where systems are heading.

The chapter that changed my defaults

Chapter 7, on transactions. Specifically the walkthrough of write skew — the anomaly where two transactions each read a consistent snapshot, each make a decision that's individually valid, and together violate the invariant.

The on-call doctor example is the one that stuck:

-- Both transactions run at the same time, both see 2 doctors on call.
BEGIN;
SELECT COUNT(*) FROM doctors WHERE on_call = true AND shift_id = 1234;
-- returns 2, so it's "safe" to go off call
UPDATE doctors SET on_call = false WHERE name = 'Alice' AND shift_id = 1234;
COMMIT;

Run that concurrently for Alice and Bob under snapshot isolation and you end up with zero doctors on call. Neither transaction did anything wrong. The isolation level simply doesn't defend the invariant you thought it did.

Serializable isolation is the only level that lets you reason about transactions as though they ran one at a time. Everything below it is a set of specific anomalies you have agreed to tolerate — usually without being told which ones.

I now read isolation levels as a list of things I'm choosing to accept, rather than a performance dial.

Where it shows its age

The book is from 2017, and it shows in the tooling: heavy on Kafka and Hadoop-era batch processing, and quiet on everything that arrived after — cloud-native OLAP, vector stores, the whole serverless database category.

It doesn't matter much. The mechanics of replication lag and consensus haven't changed; only the logos have.

Who should read it

Read it if you've been operating a database for a year or two and have started to feel the gap between what the docs promise and what production does. That friction is exactly what the book explains.

Wait if you're earlier than that. Without the scar tissue, chapter 5 is just vocabulary.

How to actually finish it

What worked on attempt three: one chapter a week, and after each one, writing down which system at work matched the failure mode described. Chapter 5 on replication turned into a real bug ticket about read-your-own-writes on a stale replica. Chapter 9 explained an outage from the previous year that we'd closed as "network weirdness."

The book stops being abstract the moment you make it about your own systems.

Verdict: the rare technical book that changes your defaults instead of your vocabulary. Worth the three attempts.