Building Software
Sections

Contents Section 3, The Machine

Networks and Protocols

How machines exchange data, from packets and routing up through DNS, TCP, TLS and HTTP.

Section 3 of 9, subsection 4 of 4. Depth: explain. A 10-minute read.

Mistakes to catch in review

  1. A state-changing action exposed as a GET request, which link prefetchers and crawlers will trigger.

  2. Outbound network calls with no timeout, so a slow dependency holds connections open until the service runs out of them.

  3. A chatty client that makes dozens of sequential round trips where one batched request would do.

  4. Certificate verification disabled to make a TLS error go away.

What Happens Between Two Machines

An agent builds a dashboard page. The server that renders it calls an internal API twelve times, once per widget, each call waiting for the one before.

On your laptop, with the API running on the same machine, the page loads in 90 milliseconds. In production the API runs in a data center across the country, and the same page takes more than a second before the first widget appears. The code is identical in both places, and the whole difference is the distance to the API.

Every network call your code makes pays for a trip that is invisible in development, because in development the other machine is your own. To review networked code you need to know what that trip consists of, what it costs, and what each protocol along the way does and does not promise.

Packets and the speed of light

A network moves data in packets, small chunks of bytes with an address on the front. On a typical Ethernet link a packet carries at most about 1,500 bytes, so a 1 MB response travels as several hundred of them. Each packet carries the IP address of its source and destination, and every router on the way looks at the destination and forwards the packet one hop closer. Packets can be dropped when a router's queue is full, and they can arrive out of order.

Light in optical fiber covers about 200,000 kilometers per second, two thirds of its speed in a vacuum. New York to London is about 5,570 km, so a packet needs at least 28 ms one way and 56 ms for a round trip, before any router touches it. Real round trips on that route run around 70 to 80 ms because cables do not follow straight lines. Within one data center a round trip is well under a millisecond.

Bandwidth, the number of bytes per second a link carries, keeps getting cheaper. Round-trip time is set by distance and physics, and no purchase lowers it below the fiber limit. For most requests an application sends, the payload is small and the round trips dominate the total time.

From a name to a connection

Before your code can send anything to api.example.com it needs an IP address, and DNS supplies it. Your program asks a resolver, usually run by your network or cloud provider. If the resolver has no cached answer, it asks a root server which servers handle .com, asks one of those which servers are authoritative for example.com, and asks the authoritative server for the address.

Output
$ dig api.example.com
;; ANSWER SECTION:
api.example.com.    300    IN    A    203.0.113.10

The 300 is the TTL, the number of seconds any resolver may cache this answer. When you change the record, resolvers that fetched it a minute ago keep serving the old address until their copy expires, and some clients cache it longer on their own. That is why a DNS change takes time to spread, and why you lower the TTL a day before a planned migration.

A program asks a resolver for api.example.com. On a cache miss the resolver asks three servers in turn. First the root, asked who handles .com, answers with the .com servers. Second the .com servers, asked who is authoritative for example.com, answer with the example.com servers. Third the authoritative server, asked for the address, answers 203.0.113.10. The resolver, which may cache the answer for 300 seconds, hands 203.0.113.10 back to the program.
On a cache miss the resolver asks the root, then the .com servers, then the authoritative server, one after another, and hands the address back to the program. It may keep the answer for 300 seconds.

With an address in hand, the client opens a TCP connection. TCP turns unreliable packets into a reliable, ordered stream of bytes. It numbers every byte, the receiver acknowledges what it got, and the sender retransmits anything unacknowledged.

Setting up a connection takes a handshake: the client sends SYN, the server replies SYN-ACK, and the client sends ACK along with its first data. That costs one round trip before any request is sent.

Reliability has a price. Because TCP delivers bytes strictly in order, one lost packet holds back every packet behind it until the retransmission arrives, even packets that belong to unrelated requests sharing the connection. This is called head-of-line blocking.

UDP is the other main transport: it sends individual datagrams with no handshake, no retransmission and no ordering, and the application handles loss itself. Video calls and games use it, and HTTP/3 runs over QUIC, a protocol built on UDP that keeps separate streams from blocking each other.

Two panels. TCP, one connection: request A, packet lost, and request B, packets arrived, both feed one stream, strictly in order, whose dashed arrow to the application reads both wait. QUIC, over UDP: stream A, packet lost, has a dashed arrow to the application reading A waits; stream B, packets arrived, has a moving arrow reading delivered. Notes read: one loss stalls both, and streams do not block each other.
On one TCP connection a lost packet from request A holds back request B's bytes too; QUIC keeps the two streams apart, so only A waits.

TLS: who you are talking to

For HTTPS, the client next runs a TLS handshake over the TCP connection. In TLS 1.3 this takes one more round trip (TLS 1.2 took two).

The handshake does two jobs. The two sides agree on encryption keys that no one watching the packets can learn. And the server proves its identity with a certificate: a statement that this public key belongs to api.example.com, signed by a certificate authority the client already trusts. The client checks the signature chain, the expiry date and that the name matches the host it meant to reach.

After that the connection has confidentiality (outsiders cannot read it), integrity (outsiders cannot change it without detection) and server authentication. TLS says nothing about what the server does with the data, and the IP addresses and usually the server name remain visible to anyone on the path.

Here is the fix an agent reaches for when it hits a certificate error.

TypeScript
const agent = new https.Agent({ rejectUnauthorized: false });
https.get("https://billing.internal/api/invoices", { agent }, onResponse);

Setting NODE_TLS_REJECT_UNAUTHORIZED=0 in the environment does the same for every connection the process makes. The traffic is still encrypted, but the client no longer checks who holds the other end of the key. Anyone who can intercept traffic on the path, on a hotel network, a misconfigured proxy or a compromised router, can present their own certificate, and the client will encrypt everything to them.

The error was reporting a real problem: an expired certificate, a missing intermediate certificate, a hostname mismatch, or a server signed by a private certificate authority the client does not trust. Each has a real fix. For a private authority, give that one client the authority's certificate as a trusted root, which keeps verification on.

TypeScript
const agent = new https.Agent({ ca: readFileSync("internal-ca.pem") });

Adding up a page load

Put the pieces together for a fresh HTTPS request with a 70 ms round trip. The DNS lookup takes somewhere between nothing (cached) and a few round trips. The TCP handshake takes one round trip and TLS takes one more. Sending the request and receiving the first byte of the response takes a third, plus the server's own processing time. So the first byte arrives about 210 ms after the connection starts, plus the server time.

Along the top, left to right: DNS, 0 to a few round trips; TCP handshake, 1 round trip; TLS handshake, 1 round trip; and the request, plus server time. A timeline runs beneath the last three, starting at 0 ms when the connection starts, with a 70 ms span under each of TCP, TLS and the request. It ends at a highlighted first byte, about 210 ms plus server time. A note reads: each new host pays again.
With a 70 ms round trip, TCP, TLS and the request take one round trip each, so the first byte arrives about 210 ms after the connection starts, plus server time.

For a web page, the browser then parses the HTML and discovers the stylesheets, scripts and images it needs, and each new host it contacts pays the handshakes again.

curl shows these phases for any URL.

Output
$ curl -o /dev/null -s -w 'dns %{time_namelookup} connect %{time_connect} tls %{time_appconnect} ttfb %{time_starttransfer}\n' https://api.example.com/health
dns 0.012 connect 0.083 tls 0.155 ttfb 0.241

The numbers are cumulative seconds from the start. Connect minus DNS is about 70 ms, one round trip for TCP. TLS minus connect is another 72 ms. The last gap, 86 ms, is one round trip plus about 16 ms of server work.

Browser developer tools show the same breakdown on the Timing tab of each request.

Now the dashboard. Twelve calls in sequence, each with 10 ms of server work, over a connection that is reused:

TypeScript
const widgets = [];
for (const id of widgetIds) {
  widgets.push(await fetch(`${API}/widgets/${id}`).then(r => r.json()));
}

That costs about 12 × (70 + 10) = 960 ms, plus the handshakes for the first call. If each call opens a new connection, add two handshake round trips per call and it becomes 12 × (3 × 70 + 10) = 2,640 ms. Sending the calls at once lets their round trips overlap:

TypeScript
const widgets = await Promise.all(
  widgetIds.map(id => fetch(`${API}/widgets/${id}`).then(r => r.json()))
);

Now the total is one set of handshakes (two round trips, 140 ms), one round trip for the requests (70 ms) and the slowest server response (10 ms): 70 + 70 + 70 + 10 = 220 ms, plus DNS. A batch endpoint that returns all twelve widgets in one response gets the same result with one request. Moving the server closer to the API cuts the round trip itself.

This program runs both shapes against a fake call that takes 80 ms, one round trip plus the server's work, over a connection that is already open.

TypeScriptRuns in your browser
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
// Stands in for fetch: one 70 ms round trip plus 10 ms of server work.
const getWidget = async (id: number) => { await sleep(70 + 10); return { id }; };
const widgetIds = Array.from({ length: 12 }, (_, i) => i + 1);

let start = Date.now();
const inTurn = [];
for (const id of widgetIds) inTurn.push(await getWidget(id));
console.log(`one at a time: ${inTurn.length} widgets in ${Date.now() - start} ms`);

start = Date.now();
const together = await Promise.all(widgetIds.map((id) => getWidget(id)));
console.log(`all at once: ${together.length} widgets in ${Date.now() - start} ms`);

The loop lands near 960 ms and the parallel version near 80, a few milliseconds over each because timers fire late.

When you see await inside a loop around a network call, multiply the loop count by the round-trip time between where the code runs and where it is calling. That number is the floor on how long the loop takes.

Methods carry promises

HTTP gives each request a method, and the method tells every machine that handles the request what it is allowed to assume. RFC 9110, the HTTP specification, defines GET, HEAD and OPTIONS as safe: the client is only asking to read. PUT and DELETE are idempotent, meaning that sending the same request twice has the same effect as sending it once. POST is neither.

Software all over the network relies on those promises. Chat apps fetch links to build previews, and email security scanners open links in messages to check them. Search crawlers follow every link they find, and browsers prefetch pages they expect you to visit. All of them send GET, because GET is supposed to be harmless.

HTTP
GET /projects/42/archive
Four senders on the left, chat preview, email scanner, crawler and browser prefetch, all point at a highlighted node GET /projects/42/archive, a link. It leads to archived, before anyone clicks. Below, the fix: GET returns a page with a form, and an arrow labelled a person submits leads to POST, archives. A note reads: GET must change nothing.
Chat previews, email scanners, crawlers and prefetch all send GET, so an archive link fires before anyone clicks. The fix returns a page with a form whose POST does the archiving.

An agent that implements "archive project" as a link to that URL has built a button that any of those programs will press. If the chat app's preview fetcher can reach that URL, for example because the link carries its own login token, pasting the link into a channel archives the project before anyone clicks. The design that works is a GET that returns a page describing the action, with a form that submits a POST to perform it. The question to ask of any route is whether calling it twice, or having a robot call it, changes anything.

Reading a response

Status codes group by the first digit. 2xx means success: 200 OK, 201 Created, 204 No Content. 3xx means look elsewhere: 301 and 308 are permanent redirects, 302 and 307 temporary ones, and 304 Not Modified means your cached copy is still good.

4xx means the client's request was wrong: 400 malformed, 401 not authenticated, 403 the server understood the request and refuses to carry it out, often because of a permission check, 404 not found, 409 conflicts with current state, 429 too many requests. 5xx means the server failed: 500 an error in the server, 502 and 504 a bad or missing answer from a server behind it, 503 temporarily unavailable.

Caching headers tell every cache on the path what it may keep. Here is a response captured in developer tools, followed by the browser's next request for the same file.

Output
HTTP/2 200
cache-control: public, max-age=3600
etag: "a1b2c3"

GET /app.css
if-none-match: "a1b2c3"

HTTP/2 304

max-age=3600 lets any cache reuse the response for an hour without asking. After that, the browser sends the etag back in if-none-match, and if the file has not changed the server answers 304 with no body. no-store tells every cache, the browser's included, not to keep a copy, which suits a response that must not be stored anywhere, such as a one-time token. private lets the user's own browser keep the response and keeps it out of shared caches such as a CDN, which suits personal data the browser may reuse.

Every call needs a deadline

A network call can fail fast, with a refused connection or an error status, or it can hang. The other side may have accepted the connection and then stopped responding. Python's requests library waits forever by default in that case, and Node's built-in fetch waits five minutes for the response headers before it gives up.

TypeScript
const res = await fetch(GEOCODER_URL, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify(address),
}); // no deadline of our own: relies on fetch's five-minute default

Suppose this runs in a server that works on at most 40 requests at a time, and 20 requests per second each call the geocoder. When the geocoder hangs, every request that reaches this line stops there and holds its slot. After two seconds all 40 are stuck, and the server stops answering everything, including pages that never touch the geocoder. Connection pools, file descriptors and memory for pending requests fill the same way.

TypeScript
try {
  const res = await fetch(GEOCODER_URL, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(address),
    signal: AbortSignal.timeout(3000),
  });
  return await res.json(); // reading the body is still under the same deadline
} catch {
  return fallbackLocation(address); // a TimeoutError lands here, before or after the headers
}

AbortSignal.timeout(3000) covers the whole exchange, connecting and reading the body included. If the 3 seconds run out before the response headers arrive, fetch rejects with a TimeoutError. If they run out while the body is still arriving, fetch has already returned, and it is res.json() that rejects, which is why the body read sits inside the same try. Some clients only limit the gap between bytes, so a server that trickles data never trips them; a total deadline closes that gap. Choose the limit from what the caller can afford to wait, and decide what the code does when it fires.

Here is the same story on a faster clock: a request every 5 ms, 40 slots, and a geocoder that never answers.

TypeScriptRuns in your browser
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));

// The geocoder accepted the connection and then stopped answering.
// Only an abort signal ever ends a call to it.
const hungGeocoder = (signal?: AbortSignal) =>
  new Promise<never>((_, reject) => signal?.addEventListener("abort", () => reject(signal.reason)));

async function simulate(label: string, timeoutMs: number | null) {
  const SLOTS = 40;
  let busy = 0, peak = 0, refused = 0, fallback = 0;
  const pending: Promise<unknown>[] = [];
  for (let i = 0; i < 100; i++) {  // a request every 5 ms for half a second
    if (busy === SLOTS) {
      refused++;
    } else {
      busy++;
      peak = Math.max(peak, busy);
      const signal = timeoutMs === null ? undefined : AbortSignal.timeout(timeoutMs);
      pending.push(hungGeocoder(signal).catch(() => fallback++).finally(() => busy--));
    }
    await sleep(5);
  }
  if (timeoutMs !== null) {
    await sleep(timeoutMs + 10);  // let the last deadlines fire
    await Promise.all(pending);   // every accepted request has now settled
  }
  console.log(`${label}: at most ${peak} slots in use, ${refused} requests refused, ${fallback} answered with a fallback`);
}

await simulate("no timeout", null);
await simulate("30 ms timeout", 30);

Without a timeout every slot fills within 200 ms and the server refuses the rest. With one, only the calls from the last 30 ms are waiting at any moment, so a handful of slots is ever in use, nothing is refused, and once the last deadline passes all 100 requests have an answer.

A timeout on an outbound call has to leave time before the caller's own deadline. If the request must answer within 10 seconds and the geocoder's timeout is also 10 seconds, nothing remains to handle the failure and still respond, so set the dependency's limit lower, for example 3 seconds, and spend the difference on a fallback or a clean error.

Check your understanding

7 questions on this lesson. Pick an answer to see whether it holds and why.

  1. 1A phone app sends a request, and the connection drops before the response arrives, so the app cannot tell whether the server acted on it. Which request can it resend unchanged without risking a double effect?
    Show the answer

    Answer D. PUT is idempotent: it sets the address to a value, so applying it twice leaves the same state as applying it once. Each POST here adds something, so a resend can create a second order or a second credit when the first one went through. TCP is the tempting answer, but its delivery promise covers one connection. Once that connection drops, a resend is a new request on a new connection, and TCP cannot tell it is a copy.

  2. 2The connection to the pricing API is already open, the round trip is 60 ms and each call spends 5 ms on the server. About how long does this loop take for 15 items, and what cuts it the most?
    TypeScript
    const prices = [];
    for (const id of cartItemIds) {
      prices.push(await getPrice(id));
    }
    Show the answer

    Answer B. Each await waits a full round trip before the next call starts, so the loop costs 15 times 65 ms. The payloads are tiny, so bandwidth is not what limits it, and DNS was settled when the connection opened. Sending the calls at once lets the round trips overlap, and one batch request needs only one. Reusing the connection saves the handshakes but does nothing about the 15 sequential round trips.

  3. 3A service calls a partner API and fails with the error below. The same URL opens without a warning in a browser. Which fix keeps the connection safe?
    Output
    Error: unable to verify the first certificate
      code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE'
    Show the answer

    Answer C. The client could not link the server's certificate to a trusted root, which usually means the server is not sending its intermediate certificate. Browsers often fetch or remember missing intermediates, so they hide the problem. Disabling verification on one client is tempting because it seems contained, but that client will then encrypt to anyone who can intercept its traffic.

  4. 4An agent added ETags to the catalog endpoint. A browser that already holds the catalog revalidates it, and the catalog has not changed. What does the page's fetch get back?
    TypeScript
    function handleCatalog(req: Request): Response {
      const body = JSON.stringify(catalog);
      const etag = `"${hash(body)}"`;
      if (req.headers.get("if-none-match") === etag) {
        return new Response("", { status: 200, headers: { etag } });
      }
      return new Response(body, { headers: { etag, "content-type": "application/json" } });
    }
    Show the answer

    Answer B. Only a 304 tells a cache that its stored copy is still good. A 200 is a complete new response, so the browser hands the page an empty body and res.json() fails. The matching ETag is the tempting reason to expect the cached copy, but the server has to say so with the status code, and the browser does not rewrite a 200 for it. Returning status 304 with no body fixes it.

  5. 5An agent set these headers on /api/me, which returns the signed-in user's account. The site runs behind a shared CDN. What can go wrong?
    HTTP
    HTTP/2 200
    cache-control: public, max-age=86400
    content-type: application/json
    
    {"user": "ana", "balance": 1520}
    Show the answer

    Answer A. public tells every cache on the path, including shared ones, that it may keep the response, and max-age=86400 lets it reuse the response for a day. The URL is the same for every user, so the next visitor can get Ana's balance. A CDN terminates TLS so it can cache at all, which is why the HTTPS option is wrong. private or no-store fits per-user data.

  6. 6Users see intermittent 502 responses from your load balancer. Where should you look first?
    Show the answer

    Answer C. A 502 means a gateway got a bad answer, or none, from the server behind it, so the fault is past the load balancer: crashed app processes, closed connections, or a wrong upstream address. Malformed requests are 4xx, permission refusals are 403 and an expired cache entry is revalidated quietly. The first option mixes up which side each class of codes blames.

  7. 7You moved api.example.com to a new server and updated its A record, which has a TTL of 86400. An hour later some clients still reach the old server. What explains it?
    Show the answer

    Answer A. The TTL lets every resolver that fetched the old answer keep using it for 86,400 seconds, so the change spreads only as cached copies expire. Lowering the TTL a day before the move would have made the switch take minutes. Browsers and operating systems do cache answers, but they also expire them, so the last option overstates a real effect.

You understand it when you can

Drill

An agent built an unsubscribe link as a GET request to /unsubscribe?user=ID and calls a third-party email API with no timeout. Find how users get unsubscribed without ever clicking, and what happens to your web servers when the email API stops responding.

Topics

Packets, IP and Routing
How data is split into packets, addressed and routed across networks.
TCP and UDP
Reliable ordered streams versus lightweight datagrams, the cost of setting up connections, and head-of-line blocking.
DNS
How names become addresses, how caching and TTLs work, and why DNS changes take time to spread.
TLS
What transport encryption guarantees about confidentiality and server identity, and what it leaves uncovered.
HTTP Semantics
Methods, status codes, headers and caching, and which methods are safe to repeat.
Latency and Bandwidth
Why round trips dominate response time, and how batching, connection reuse and proximity reduce them.

Further reading

Start here

  • 1.1 Introduction (reposted) - What is the Internet. Jim Kurose, 2022. 14-minute lecture.

    The first lecture of the textbook's author, describing the Internet as hosts, links, packet switches and protocols, and framing the layered stack from DNS and HTTP down to IP.

Short reads

  • It's the Latency, Stupid. Stuart Cheshire, 1996.

    Why you can buy more bandwidth but not less latency, argued from the speed of light.

  • Primer on Latency and Bandwidth. Ilya Grigorik.

    The components of network delay, with the numbers you need to reason about a page load.

  • Some ways DNS can break. Julia Evans, 2022.

    Real DNS failures collected from readers, each small enough to understand in a minute.

Watch

Books

  • Computer Networking: A Top-Down Approach. James F. Kurose and Keith W. Ross, 2025, 9th edition.

    The standard textbook, starting from HTTP and DNS and working down through TCP congestion control, IP routing and link layers.

  • High Performance Browser Networking. Ilya Grigorik, 2013. Free to read online.

    Quantifies how round trips from TCP setup, slow start and the TLS handshake dominate response time, and how connection reuse and batching cut them.

  • Bulletproof TLS and PKI: Understanding and Deploying SSL/TLS and PKI to Secure Servers and Web Applications. Ivan Ristić, 2022, 2nd edition.

    Explains what TLS guarantees, how certificate validation and PKI work, and the real attacks that succeed when verification is skipped or misconfigured.

Primary sources