Node internals · the ownership boundary

Where Node ends
and the kernel begins

Protocols, layers, and the one question that separates them. Not which layer is this, but whose code actually runs.

13 sections12 figuresevery figure interactivev1.0

Node.js does not implement TCP. Node asks the operating system for a TCP socket, and then implements application-level protocols on top of whatever comes out of it.

Everything else in this document is a consequence of that one sentence.

§00

The short answer

Two different questions get asked with the same words, and almost every confusion about Node networking is one answered in place of the other.

the layer
What the protocol is, and where it sits in the model. Drawn as a band in every figure here.
the owner
Which component implements it, and therefore whose bug it is. Drawn as a hue.

The two are independent. A band can hold two owners, and one owner can span three bands, which is why the figures spend a separate channel on each.

Your Node program sits mostly at the application layer. TCP and UDP live below it and are implemented by the kernel. Node bridges the two through C++ and libuv. That bridge is not a layer in any model. It is a path, and knowing where it starts and stops is most of what this document is for.

FIG 01 The stack, in three projections

Two things are worth noticing before going further.

  • The hues do not follow the bands. The application band is entirely Node's code and yours. The three bands below it are entirely the kernel's.
  • epoll and the scheduler sit off to the side. They belong to no layer at all. They are services the kernel offers, and Node's whole concurrency model is built on one of them.
the rule

Node's application APIs expose networking primitives. The kernel owns the transport machinery. Every time you cannot decide which one is responsible for something, ask which side of the system call boundary the code runs on.

§01

Transport: TCP and UDP

Two transports sit under everything in this document, and the choice between them decides which problems are yours.

TCP UDP
connection a handshake, and state on both ends none. Each datagram stands alone
delivery retransmitted until acknowledged best effort. A lost datagram is gone
order reassembled into the order you wrote whatever order they arrive in
boundaries none. A byte stream preserved. One send is one receive
flow control the receive window, and congestion control none. You can flood the receiver
in Node net, and everything built on it dgram, and dns.resolve4()

Both are the kernel's. The rest of this section follows TCP, because that is what a server is built on. Suppose you write this:

server.js
import net from "node:net";

const server = net.createServer(socket => {
  socket.on("data", data => {
    console.log(data);
  });
});

server.listen(3000);

It looks like Node is creating TCP. It is not. Underneath, that call descends through the net module to a C++ object, to a libuv handle, to a system call, and only then reaches an actual TCP implementation, written by whoever wrote your kernel.

The descent is worth stepping through line by line, because the interesting part is how little happens on the first line and how much happens on the second. It is worth stepping back up too: when a connection finally arrives, the descriptor the kernel hands back has to be wrapped by each of those layers again, in reverse, before anything reaches your callback — and that return journey makes no system call at all.

FIG 02 Two lines of JavaScript, six system calls, and the way back

Node implements none of the machinery that makes TCP work. The kernel does all of it, and the third column is the only trace of any of it that reaches your code.

implemented by what you see in Node
sequence numbers, acks the kernel nothing at all
retransmission, congestion control the kernel latency, and nothing else
receive window, flow control the kernel a slow drain when the peer stops reading
handshake, connection state the kernel, into the accept queue a 'connection' event, already established
segmentation, reassembly the kernel where the 'data' boundaries land, which is §03

A Python server on the same machine uses that same kernel code, and so does a Go one. The difference between a Node server and a Python server is entirely above the system call boundary.

a consequence worth holding on to

EADDRINUSE comes from bind(), which is fired by listen() — never by createServer(). Two processes can both call createServer on port 3000 quite happily. Only one of them can call listen.

§02

What Node actually gets back

Eventually the kernel gives your process a file descriptor. Something like 17.

That number is not the connection. It is not a pointer, not a handle you can pass to another machine, not even a value that means anything in another process. It is an index into a table your process owns, and row 17 of that table refers to a kernel object, here a listening socket.

FIG 03 A descriptor is a row number

The kernel matches an arriving packet to a socket on all four of source address, source port, destination address and destination port. That is the four-tuple, and it is how one listening port serves thousands of connections at once. A client's ephemeral port is the only part of it nobody chose deliberately.

This is also why close() is a weaker operation than it sounds. It removes one row. If something else still refers to the same socket, whether a forked worker or a duplicated descriptor, the socket carries on serving traffic and the port stays held. Nothing you can see in your own process explains why.

§03

TCP is a byte stream

This distinction becomes extremely important the moment you reach the application layer, so it is worth being exact about it.

TCP promises a byte stream. The promise is narrower than it sounds, and the second column is where every framing bug comes from.

TCP guarantees TCP says nothing about
the bytes all of them, in the order you wrote them how they are grouped on the way
duplicates none. Exactly once how many reads it takes to see them
corruption caught by a checksum, and resent what any of the bytes mean
your write boundaries nothing. There is nowhere in a TCP header to record one where your reads will split instead

TCP does not know that some of those bytes spell POST /users HTTP/1.1 and others spell {"name":"Ary"}. From its position they are 50 4F 53 54 20 2F 75 73 65 72 73.

FIG 04 The same request, delivered four ways

Switch between the four deliveries and watch which numbers move. Bytes delivered never changes. Handler runs never changes. What changes is the number of 'data' events. The thing most people accidentally build their framing on is the one thing they do not control. Whether the cut lands between two headers or in the middle of the word Content-Length is decided by the path MTU, MSS, Nagle's algorithm and the congestion window.

the classic bug

socket.on("data", d => JSON.parse(d)) works on your laptop, works in CI, works in staging, and fails in production under load. Production is the first place where the message was large enough, or the network busy enough, for it to arrive in two pieces.

§04

Application protocols define what the bytes mean

An application protocol is an agreement about interpretation. HTTP's agreement is three rules.

  1. The first line. A method, a target and a version.
  2. Then header lines. Until a blank one ends them.
  3. Then a body. Its length comes from the headers, or, if the headers do not say, from reading chunked framing.

TCP knows none of that, and does not need to.

So when the question is where is HTTP implemented in Node, the answer is Node itself. Specifically in llhttp, a state machine compiled to C, wrapped by node:http, which builds the IncomingMessage your handler receives.

the two halves
// TCP        → the OS kernel
// HTTP       → Node.js
//
// same socket. same bytes. different authors.
§05

net vs http, on identical bytes

These two programs receive exactly the same bytes from exactly the same kernel through exactly the same descriptor. The difference is what stands between the socket and your callback.

FIG 05 Two steps, or five
net http
you receive a Buffer an IncomingMessage
boundaries a segment boundary you did not choose message boundaries the parser found
framing yours to solve solved, by llhttp
your callback runs on every read that returned bytes once, at headers_complete
bytes on the wire identical identical

The last row is the point. Same kernel, same descriptor, same bytes. http is net plus a parser, and the parser is the only thing you are choosing between.

§06

The parser, callback by callback

Conceptually, TCP hands Node a run of bytes and Node's HTTP machinery walks it. "Walks it" hides the mechanism, and the mechanism is the interesting part. llhttp does not build an object and hand it over. It emits a callback each time it finishes recognising something, and node:http assembles the object from those callbacks.

FIG 06 Bytes in, callbacks out

Step to headers_complete and look at the last meter. Zero new bytes came off the wire during any of this. Every field appeared because a state machine read bytes that had already arrived. And your handler is called at headers_complete — which is why (req, res) => {} can run before a single body byte exists, and why the body is an event stream rather than an argument.

§07

WebSocket is another application protocol

WebSocket is a good second example because it starts as HTTP and then stops being HTTP, on the same connection, without the kernel noticing anything at all.

FIG 07 A handshake, then a bit layout

The framing half is where the layering claim gets concrete. A WebSocket frame opens with a bit-level header, and every field in it is read and written by JavaScript in your process.

bits what it says
FIN 1 whether this is the last fragment of a message
RSV1–3 3 reserved, and zero unless an extension was negotiated
opcode 4 what kind of frame it is. See opcode
MASK 1 whether the payload is masked. Always set client to server
payload length 7 the length, or an escape code past 125 bytes

TCP delivers those bits as bytes and has no opinion about any of them.

In Node, none of that framing is in core. Core does the HTTP half, emitting 'upgrade' and handing you the socket. A package does the rest, including the masking key and the opcode, in JavaScript, in your process.

§08

DNS: one module, two machines

DNS is slightly different, because it commonly uses UDP rather than TCP, and because Node's dns module contains two genuinely different implementations behind one name.

FIG 08 lookup goes to a thread. resolve goes to the wire.
dns.lookup() dns.resolve4()
speaks DNS no. It asks the OS yes. It builds the messages
calls getaddrinfo, the blocking OS resolver c-ares, over a UDP socket
costs a thread yes. the thread pool, four slots, shared with the file system none. It is just another descriptor
reads /etc/hosts yes, and every other OS resolution rule no. It talks to the nameserver
used by http.get and net.connect, on your behalf only code that calls it directly

Row three is why a slow resolver can look exactly like a slow disk. Row four is why the two can return different answers for the same hostname, and both be correct.

§09

TLS sits between them

HTTPS introduces a layer that is genuinely awkward to place, because it is not really a layer. TLS takes plaintext from HTTP, wraps it in records, and hands those records to TCP as ordinary bytes. TCP does not know. IP does not know. Nothing below the application layer changes at all.

FIG 09 The same request, from four vantage points

Switch to https and walk the four positions. Your handler cannot tell the difference. OpenSSL — a library in your process, in userspace — is where ciphertext becomes plaintext. The kernel moves records it cannot read. And the router still sees addresses, ports, packet sizes, timing and usually the SNI hostname, because those live in headers that have to be readable for delivery to work.

Four protocols in one request, four different authors, one socket.

implemented by runs where
HTTP Node, in llhttp your process
TLS OpenSSL, linked into Node your process, in userspace
TCP and IP the kernel kernel space
Ethernet the driver and the card the driver, then the hardware
§10

The complete journey, to the byte

Every layer that touches your response adds a header, and every one of those headers is added by software you did not write. The original diagram for this was four labelled boxes; the honest version has a ruler on it, because the sizes are the point.

FIG 10 Encapsulation, with the numbers

A fourteen-byte JSON body ships inside a fifty-four-byte escort. A two-megabyte file is cut into fifteen hundred frames and the overhead disappears into the noise. Both facts come from the same arithmetic, and Node participates in neither. The bytes added by Node is zero in every case.

§11

Where the event loop enters

Data arrives on fd 21, and none of this involves your process.

  1. The card. Writes the bytes into memory by DMA and raises an interrupt.
  2. The driver. Hands the frame up the stack.
  3. IP. Checks the header and routes it to TCP.
  4. TCP. Acknowledges it, puts it back in order, and appends it to the socket's receive buffer.
  5. The kernel. Flips a bit that says this descriptor is now readable.

Your process has been asleep for all of that, inside epoll_wait, using no CPU.

FIG 11 Nine steps, and you are only in the last three

Then switch to the control. With your JavaScript busy, the first six steps happen exactly as before — the kernel does not need you — and then everything stops, because the one thread that could come back to the poll phase and look is occupied. Nothing is lost and nothing errors. The readiness simply waits, and your latency graph gets a tail nobody can explain from the network numbers.

§12

The most important distinction

Think of the layers as different pieces of software cooperating, and of the diagram as a map of who to talk to when something is wrong.

FIG 12 Who implements what — and where the bug report goes

Once that boundary is clear, net, http, streams, backpressure, libuv, descriptors, the poll phase, TCP sockets and HTTP requests stop being separate Node concepts. They are positions in one continuous pipeline, and each of them belongs to exactly one of three worlds.

the shortest form of it

Above the line, Node decides what bytes mean. Below the line, the kernel decides that they arrive. Neither one can do the other's job, and almost every hard networking bug is a misattribution between them.

Back to JIT-Atlas