Building Software
Sections

Contents Section 5, State

Concurrency and Race Conditions

What goes wrong when many things happen at once, within one process or across many workers: races, deadlocks and overload.

Section 5 of 9, subsection 3 of 4. Depth: explain. An 8-minute read.

Mistakes to catch in review

  1. Check-then-act races, such as checking stock and then decrementing it in two separate steps.

  2. An unbounded fan-out of thousands of parallel requests that overwhelms a downstream service or trips its rate limits.

  3. Mutable module-level state shared across requests, leaking one user's data into another user's response.

  4. A deadlock from two code paths taking the same two locks in opposite order.

When Things Happen at Once

An agent writes the "buy" handler for a small shop. It checks that the item is in stock, then takes one off the shelf:

TypeScript
async function buy(itemId: string, userId: string) {
  const item = await db.getItem(itemId);
  if (item.stock > 0) {
    await db.setStock(itemId, item.stock - 1);
    await db.createOrder(itemId, userId);
  }
}

There is one unit left. Two customers click "buy" within the same second. Both get an order confirmation, and the stock ends at zero instead of minus one, which hides the fact that you sold something you do not have.

Run alone, each call behaves correctly. The failure needs two calls to overlap.

What an interleaving is

A program that runs one thing at a time has one possible order of events. Two tasks running together have many. Each task's own steps stay in order, but the steps of the two tasks can be shuffled together in any way, and each shuffle is called an interleaving. A race condition is a bug that shows up in some interleavings and not others.

Here is the interleaving that breaks buy:

Step Customer A Customer B stock
1 reads stock = 1 1
2 reads stock = 1 1
3 1 > 0, so sets stock to 0 0
4 1 > 0, so sets stock to 0 0
5 creates order creates order 0

Both customers checked a condition, and both acted on it, but between B's check and B's action the condition stopped being true. This shape is called check-then-act, and you will see it often in agent-written code. Look for any if whose condition was computed from state that something else can change before the body runs.

The fix is to make the check and the act one indivisible step, so no other task can run between them. With a database, that means a single conditional statement: UPDATE items SET stock = stock - 1 WHERE id = $1 AND stock > 0, then create the order only if one row changed. Run the update and the order insert in one transaction, so a failed insert puts the unit back. With in-memory state, it means a lock. A version number that the write must match also works: a stale write changes nothing, and the code reads again and retries.

This block runs both versions against a fake database where every call takes 10 milliseconds. takeOne stands in for the conditional UPDATE: the database checks and writes in one step.

TypeScriptRuns in your browser
const roundTrip = () => new Promise((r) => setTimeout(r, 10));
const items = new Map([["mug", { stock: 1 }]]);
const orders: string[] = [];

async function getItem(id: string) {
  await roundTrip();
  return { ...items.get(id)! };
}
async function setStock(id: string, stock: number) {
  await roundTrip();
  items.get(id)!.stock = stock;
}
async function takeOne(id: string): Promise<boolean> {
  await roundTrip();
  const item = items.get(id)!;
  if (item.stock <= 0) return false;
  item.stock -= 1;
  return true;
}

async function buyCheckThenAct(id: string, user: string) {
  const item = await getItem(id);
  if (item.stock > 0) {
    await setStock(id, item.stock - 1);
    orders.push(user);
  }
}
async function buyAtomic(id: string, user: string) {
  if (await takeOne(id)) orders.push(user);
}

await Promise.all([buyCheckThenAct("mug", "A"), buyCheckThenAct("mug", "B")]);
console.log("check-then-act:", orders, "stock left:", items.get("mug")!.stock);

items.set("mug", { stock: 1 });
orders.length = 0;
await Promise.all([buyAtomic("mug", "A"), buyAtomic("mug", "B")]);
console.log("one atomic step:", orders, "stock left:", items.get("mug")!.stock);

The first line shows two orders for one mug, and the stock still reads zero, so nothing in the data says a unit was oversold.

Concurrency and parallelism are different

Concurrency means several tasks are in progress during the same period. They may take turns on one CPU. A Node.js server handling a thousand open requests on a single thread is concurrent: while one request waits for the database, the event loop runs another. No two lines of JavaScript ever execute at the same instant.

Parallelism means several tasks are executing at the same instant, on different CPU cores. Resizing 8,000 images with eight worker processes on an eight-core machine is parallel: eight resizes are happening in the same microsecond.

In a single-threaded event loop, a task can only be interrupted where it waits. In JavaScript that is every await. Between two awaits, your code runs without any other task touching memory. That is why the buy race needs the await on db.getItem: it is the point where customer B's request gets to run.

It also means a slow synchronous computation, like parsing a 200 MB JSON file, blocks every other request on that server until it finishes.

Two rows. The top row, concurrency on one Node.js thread, shows request A running, then an arrow labelled await to request B running, then another arrow labelled await to request A resuming, one after another. A note reads: switches only where a task waits. The bottom row, parallelism on eight cores, shows eight boxes, Core 1 through Core 8, side by side in two rows of four, each resizing images at the same time. A note reads: all at the same instant.
On one Node.js thread, requests take turns: a task gives way only where it waits, here at each await, and never in the middle of synchronous code. With eight cores, eight resizes run at the same instant.

With true parallelism, as in Go, Java or Rust threads, a task can be interrupted between any two machine instructions. Even count++ is a read, an add and a write, and two threads can lose an increment the same way the customers lost a unit of stock. When two threads touch the same memory at the same time, at least one writes, and nothing orders them, that is a data race. A data race and a race condition are different things, and neither implies the other. The buy bug is a race condition with no data race, because the database makes each individual read and write atomic. In the other direction, Java's String.hashCode caches its result in a plain field that several threads may write without synchronization, which is a data race, and every thread writes the same value, so the outcome never depends on the order.

Shared state between requests

Event-loop concurrency makes one kind of shared state dangerous. A module-level variable in Node lives once per process and is shared by every request that process handles.

TypeScript
let currentUser: User | null = null;

app.use(async (req, res, next) => {
  currentUser = await loadUser(req);
  next();
});

app.get("/account", async (req, res) => {
  const orders = await db.ordersFor(currentUser!.id);
  res.json({ name: currentUser!.name, orders });
});

Alice's request sets currentUser to Alice and then waits on the orders query. During that wait, Bob's request arrives and the middleware sets currentUser to Bob. When Alice's query returns, her handler reads currentUser.name and sends Bob's name in Alice's response. It works perfectly in development, where you send one request at a time, and leaks data in production, where requests overlap.

Alice's request runs along the top: it sets currentUser to Alice, awaits the orders query with db.ordersFor, reads currentUser!.name, and sends her response. Below sits one currentUser variable for the whole process, which holds Alice, then Bob, with an arrow from Alice's request into it. On the left, Bob's request sets the same variable to Bob, with an arrow labelled during Alice's wait. An arrow runs from currentUser up to Alice's read, so Alice's response, highlighted, holds Bob's name. A note reads: shared by every request.
Bob's request sets currentUser while Alice's handler waits on its query, so Alice's response carries Bob's name.

The question that exposes it: is this value about one request, and is it stored somewhere that outlives the request? Per-request data belongs in the request object, in function parameters, or in a mechanism built for request scope such as Node's AsyncLocalStorage. Module-level variables are for things that are the same for every request, like configuration and connection pools.

Locks and deadlocks

A lock (also called a mutex, for mutual exclusion) is a flag that only one task can hold at a time. A task that wants it while someone else has it waits. If every piece of code that touches some shared state takes the same lock first, the check and the act inside that code cannot be interleaved with any other task's.

Locks have two costs. The first is waiting. While one task holds the lock, every other task that needs it stands still, so the time a lock is held should be as short as possible. Holding a lock while calling a slow network service makes every other task wait for that service too.

The second cost is deadlock. In JavaScript a lock matters only when the code it protects awaits, because between two awaits no other task can run anyway. Suppose a transfer locks both accounts and then writes to a ledger:

TypeScript
async function transfer(from: Account, to: Account, amount: number) {
  await from.mutex.lock();
  try {
    await to.mutex.lock();
    try {
      from.balance -= amount;
      to.balance += amount;
      await ledger.record(from.id, to.id, amount);
    } finally {
      to.mutex.unlock();
    }
  } finally {
    from.mutex.unlock();
  }
}
A square cycle of four boxes. The request running transfer(a, b, 10) has an arrow to lock b, labelled waits for. Lock b is held by the request running transfer(b, a, 5), which has an arrow to lock a, labelled waits for. Lock a is held by the first request, closing the loop. A note below reads: both hang.
Each transfer holds one account's lock and waits for the other's, so the cycle never breaks and neither request finishes.

Now one request runs transfer(a, b, 10) while another runs transfer(b, a, 5). The first locks a and pauses at its next await, and the second locks b. The first now waits for b, which the second holds. The second waits for a, which the first holds.

An in-process lock usually waits forever: both requests hang, and nothing throws. A database detects the cycle and aborts one of the transactions with a deadlock error (PostgreSQL checks after the wait passes deadlock_timeout, one second by default), which surfaces as a failed request that is hard to reproduce on demand.

The standard fix is lock ordering: every code path that takes more than one lock takes them in the same global order. Here, always lock the account with the smaller ID first, whichever direction the money moves. A transfer from an account to itself needs its own case, because a mutex like this one is not reentrant: a task that already holds it and asks for it again waits for itself.

This block builds a small mutex, runs the two opposite transfers with each version, and gives each pair 200 milliseconds to finish.

TypeScriptRuns in your browser
class Mutex {
  private held = false;
  private waiting: Array<() => void> = [];
  async lock(): Promise<void> {
    if (!this.held) {
      this.held = true;
      return;
    }
    await new Promise<void>((resolve) => this.waiting.push(resolve));
  }
  unlock(): void {
    const next = this.waiting.shift();
    if (next) next(); // hand the lock straight to the next waiter
    else this.held = false;
  }
}

type Account = { id: number; balance: number; mutex: Mutex };
const account = (id: number): Account => ({ id, balance: 100, mutex: new Mutex() });
const ledgerWrite = () => new Promise((r) => setTimeout(r, 5));

async function inOrder(first: Account, second: Account, work: () => Promise<void>) {
  await first.mutex.lock();
  try {
    await second.mutex.lock();
    try {
      await work();
    } finally {
      second.mutex.unlock();
    }
  } finally {
    first.mutex.unlock();
  }
}

async function move(from: Account, to: Account, amount: number) {
  from.balance -= amount;
  to.balance += amount;
  await ledgerWrite();
}

async function transferNaive(from: Account, to: Account, amount: number) {
  await inOrder(from, to, () => move(from, to, amount));
}

async function transferOrdered(from: Account, to: Account, amount: number) {
  if (from === to) return;
  const [first, second] = from.id < to.id ? [from, to] : [to, from];
  await inOrder(first, second, () => move(from, to, amount));
}

function within(ms: number, work: Promise<unknown>) {
  const timer = new Promise((r) => setTimeout(() => r(`still waiting after ${ms} ms`), ms));
  return Promise.race([work.then(() => "both finished"), timer]);
}

for (const transfer of [transferNaive, transferOrdered]) {
  const a = account(1);
  const b = account(2);
  const result = await within(200, Promise.all([transfer(a, b, 10), transfer(b, a, 5)]));
  console.log(`${transfer.name}: ${result}; a=${a.balance} b=${b.balance}`);
}

The naive version gets stuck before either transfer moves any money. The ordered version finishes with a at 95 and b at 105.

When you see code that takes two locks, find every other place that takes the same two, and check the order matches.

Too much at once

Concurrency can also hurt the service on the other end. An agent writes a job to refresh profiles for 50,000 users:

TypeScript
await Promise.all(users.map((u) => fetchProfile(u.id)));

This tries to start 50,000 HTTP requests immediately. The profile service allows 100 requests per second per client. Thousands arrive in the first second, and most come back as 429 Too Many Requests or time out while queued behind each other.

On your side, 50,000 open sockets and pending promises may exhaust file descriptors or memory. If the downstream service has no limit of its own, you may take it down for its other clients too.

The fix is a concurrency limit: allow at most N requests in flight, and start a new one only when one finishes.

TypeScript
async function mapWithLimit<T, R>(
  items: T[],
  limit: number,
  fn: (item: T) => Promise<R>,
): Promise<R[]> {
  if (!Number.isInteger(limit) || limit < 1) throw new RangeError("limit must be a positive integer");
  const results: R[] = new Array(items.length);
  let next = 0;
  async function worker() {
    while (next < items.length) {
      const i = next++;
      results[i] = await fn(items[i]);
    }
  }
  const workers = Array.from({ length: Math.min(limit, items.length) }, worker);
  await Promise.all(workers);
  return results;
}

This code shares the variable next between several workers and does a check-then-act on it (next < items.length, then next++). It is safe here because it runs on a single-threaded event loop and there is no await between the check and the increment. If an await sat between the check and the increment, several workers could pass the check while one item remained. Each would then take its own index, so the late ones would run fn on indexes past the end of the array.

Choosing N starts from Little's law, which relates averages: the number of requests in flight equals throughput times the average time each one takes. If the service allows 100 requests per second and each takes 200 milliseconds on average, then 100 × 0.2 = 20 requests in flight produce the allowed rate. Set the limit to 20, or a little below it to leave room for other clients.

Two rows. In the top row, Promise.all sends all 50,000 requests at once to the profile service, which allows 100 per second, and the result is 429 or timeouts. In the bottom row, mapWithLimit keeps 20 requests in flight, each taking 200 ms at the profile service, which produces 100 per second, the allowed rate. A note reads: 100 × 0.2 = 20 in flight.
Promise.all sends all 50,000 requests at once into a service that allows 100 per second; a cap of 20 in flight at 200 ms each lands at the allowed rate.

That sizes the cap, but a concurrency cap is not a rate limit. Throughput is requests in flight divided by response time, so if the service slows to one second per request, 20 in flight gives only 20 per second, and if it speeds up to 50 milliseconds, 20 in flight gives 400 per second, four times the allowed rate. Enforce the rate separately, for example with a token bucket or by spacing request starts, and treat a 429 as the signal to slow down.

A concurrency limit is one form of backpressure: the consumer's speed limits how fast the producer can hand it work. The alternative, putting an unbounded queue in front of a slow consumer, only delays overload. The queue grows, each item waits longer, memory climbs, and the failure arrives later and larger. A bounded queue makes the producer wait, or rejects new work, once it is full, so overload shows up right away as a slower producer or a clear error.

Sharing less

One way to design shared state out is to give each piece of it a single owner. Only one worker ever touches the stock for a given item, and every other part of the system sends that worker a message ("reserve one unit of item 42") through a queue.

The worker handles messages one at a time, so check-then-act inside it is safe without locks. This is message passing, and Go channels and actor systems are built around it.

When you read concurrent code, list every piece of state that more than one task can reach, and find each place that reads it and later acts on what it read. For each one, name what must stay true until the action finishes, then find what protects it: a lock, a single atomic operation, a single owner, or a version check that makes a stale action fail and retry.

Check your understanding

7 questions on this lesson. Pick an answer to see whether it holds and why.

  1. 1An agent wrote this signup handler. Two people submit the same username within a few milliseconds. Which change makes it impossible for both signups to succeed?
    TypeScript
    async function signUp(username: string, password: string) {
      const existing = await db.findUserByName(username);
      if (existing) throw new Error("username taken");
      await db.insertUser(username, await hash(password));
    }
    Show the answer

    Answer B. The unique constraint makes the database do the check and the insert as one step, so the second insert fails no matter how the requests interleave. A second lookup is still check-then-act: the lookup and the insert are separate awaited calls, so both requests can pass the lookup before either insert lands.

  2. 2What does this print?
    TypeScript
    let count = 0;
    
    async function bump() {
      const current = count;
      await Promise.resolve();
      count = current + 1;
    }
    
    await Promise.all([bump(), bump(), bump()]);
    console.log(count);
    Show the answer

    Answer C. Each call reads count while it is still 0, then pauses at the await, so all three write 0 + 1. Single-threaded does not mean safe: the await lets the other calls run between the read and the write. The result is the same every run because the event loop schedules these calls in a fixed order.

  3. 3In which of these is more than one piece of your program's code executing at the same instant?
    Show the answer

    Answer D. Four processes on four cores run at the same moment, which is parallelism. Promise.all over three requests is concurrency: the requests are in progress together, but your JavaScript still runs one piece at a time on one thread while the waiting happens on the network.

  4. 4Two background jobs in the same service use these functions. They have run fine for months, and then one night both jobs hang and never finish. What fix removes the cause?
    TypeScript
    async function archiveUser(user: User, team: Team) {
      await user.mutex.lock();
      await team.mutex.lock();
      await moveFiles(user, team);
      team.mutex.unlock();
      user.mutex.unlock();
    }
    
    async function renameTeam(team: Team, user: User) {
      await team.mutex.lock();
      await user.mutex.lock();
      await updateNames(team, user);
      user.mutex.unlock();
      team.mutex.unlock();
    }
    Show the answer

    Answer A. The hang is a deadlock: one job holds the user lock and waits for the team lock while the other holds the team lock and waits for the user lock. Taking the locks in one fixed order everywhere makes that cycle impossible. Releasing the first lock early avoids the hang but drops the protection, since the other job can now change the user while the first is still working.

  5. 5This pool was changed to wait for a rate limiter inside the loop. The limiter lets one waiting worker through at a time. The input has 3 items and the pool runs 3 workers. What goes wrong?
    TypeScript
    let next = 0;
    
    async function worker() {
      while (next < items.length) {
        await rateLimiter.wait();
        const i = next++;
        results[i] = await fn(items[i]);
      }
    }
    Show the answer

    Answer B. All three workers pass next < items.length while it is still true, then wait. When they resume, next++ still hands out a distinct index each time, so no index repeats, but the check that justified taking one is stale. Some workers take 3 and 4, which are past the end of a 3-item array.

  6. 6A downstream API allows 50 requests per second and answers in 0.5 seconds on average. You cap your fan-out at 25 requests in flight. What does that cap guarantee?
    Show the answer

    Answer B. Little's law says throughput is requests in flight divided by response time, so 25 in flight at 0.5 seconds gives 50 per second. If the API speeds up to 0.1 seconds, the same cap allows 250 per second. A cap sizes the load; enforcing the rate takes a separate limiter that spaces request starts.

  7. 7Which of these is safe to keep in a module-level variable of a Node server that handles many requests at once?
    Show the answer

    Answer D. A module-level variable is shared by every request the process handles. The pool is the same for every request, so sharing it is the point. Clearing a list at the start of each request feels safe, but while one request waits on an await, another clears or fills the same list, and errors end up in the wrong response.

You understand it when you can

Drill

An agent wrote a script that starts sending a welcome email to all 80,000 new users at once with no concurrency limit. The loop sets a module-level greeting variable just before each call to the send function, and the send function reads that variable only after it has awaited the mail connection. Find both defects and the symptom each one produces in production.

Topics

Race Conditions
How interleaved operations produce results no single ordering would, and how to spot check-then-act patterns.
Locks and Deadlocks
Mutual exclusion, lock ordering, and why holding a lock across a slow operation hurts.
Async and Parallelism
Concurrency without parallelism in event loops, true parallelism across cores, and the bugs each allows.
Message Passing and Shared-Nothing Design
Avoiding shared mutable state by communicating through queues and messages.
Backpressure and Bounded Queues
Limiting work in flight so a fast producer cannot bury a slow consumer.

Further reading

Start here

Short reads

Watch

  • Concurrency is not Parallelism by Rob Pike. Rob Pike, 2012. 31-minute talk.

    Separates concurrency, which is structuring independent tasks, from parallelism, which is running them at the same time, and builds the idea around communicating through channels instead of shared state.

  • Zach Tellman - Everything Will Flow. Zach Tellman, 2015. 40-minute talk.

    Explains queueing behavior under load and why unbounded queues hide overload until latency collapses, making the case for bounded queues and backpressure.

Books

  • Operating Systems: Three Easy Pieces. Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau, 2023, Version 1.10. Free to read online.

    Its concurrency part covers race conditions, locks, condition variables, event-based concurrency and deadlock prevention through lock ordering, in short free chapters.

  • The Art of Multiprocessor Programming. Maurice Herlihy, Nir Shavit and 2 others, 2020, 2nd edition.

    The rigorous treatment of mutual exclusion, lock implementations and concurrent data structures, for readers who want to know why a lock is correct.

Primary sources