Mission 11 · Node.js Internals Campaign

cluster — four processes, one port, and a file descriptor in the post

The sixth lesson, and the last of the arc. Part 10 gave you a second runtime inside one process. This part gives you a second process — which immediately hits a problem threads never have, because two processes cannot both bind port 3000. The answer involves posting a file descriptor through a unix socket, and everything else follows from it.

12 sections12 figuresone portSCM_RIGHTS · SO_REUSEPORT
§ 0 · THE SHORT ANSWER FIRST

Four processes, one port, and a file descriptor in the post

Part 10 gave you a second runtime inside one process. This part gives you a second process — and immediately runs into a problem that has no equivalent at the thread level.

Two processes cannot both bind() port 3000. Part 8 showed exactly why: the second bind returns EADDRINUSE, and it is the kernel refusing, not Node. So how do four worker processes end up serving one port?

JAVASCRIPTTHE FOUR LINES THAT HIDE ALL OF IT
if (cluster.isPrimary) {
  for (let i = 0; i < os.availableParallelism(); i++) cluster.fork();
} else {
  http.createServer(handler).listen(3000);   // four processes, same port
}

Before the answer, the same orientation Part 10 opened with — because the thing that changes here is one level further out. Part 10 added a runtime inside your process; cluster adds whole processes beside it.

The same machine, divided differently
§0 · processes on a real machine
the machine1 CPU · 32 cores
the CPU 32 cores
the primarypid 4821
one processnever runs your handler
V8 isolate + event loopforks and supervises workersowns the LISTENING socket, fd 17
One core, mostly idle. If it dies the workers keep serving, but nothing replaces them.
worker process 1pid 4822
one processruns your handler
its own V8, heap and JIT stateits own address space — no shared memorythe socket arrives as fd 12
~40 MB resident, and it recompiles the same hot functions the others already have.
worker process 2pid 4823
one processruns your handler
its own V8, heap and JIT stateits own address space — no shared memorythe socket arrives as fd 12
Identical and independent. Its in-memory cache shares nothing with worker 1's.
… up to one per core× 32
processesall serving one port
32 copies of your entire application32 heaps, 32 module registries~1.3 GB before a single request
Past the core count they time-slice and throughput FALLS — §5 measures the curve.
one core — and one core runs one thread at a time. This is one physical CPU package, the shape of almost every cloud instance and what os.availableParallelism() counts. A large bare-metal server may carry a second package, a second socket — a word unrelated to the network sockets of Parts 0, 5, 8 and 9.
Compare this with Part 10. There, one process frame enclosed every thread. Here there is no enclosing frame at all, because each box IS a process — which is why the listening socket has to be POSTED from the primary to each worker as a file descriptor, and why nothing else crosses between them. The isolation is the product; the memory is the price.
no process frame around them — that absence is the whole difference from Part 10
What each process gets
§0 · the same thing as a list
the machine · N cores
hardware
hw
Unchanged, and still the only place true parallelism lives. Everything below is a different way of dividing the same cores.
the primary process
your cluster.fork() calls · no requests
main
Its own address space, its own V8, its own event loop. In the default mode it also owns the one listening socket and hands connections out — but it does not run your handler.
worker process 1 · its own everything
address space · heap · loop · fd table
kernel
A SEPARATE PROCESS. Nothing in its memory is reachable from any other worker. It has its own file descriptor table, which is exactly why the listening socket has to be sent to it rather than shared.
worker processes 2..N
identical, and equally isolated
kernel
Each is a full Node runtime with its own copy of your modules and its own JIT warmup. Four workers means four of everything except the machine.
one listening socket, held by all of them
passed as a file descriptor
node
The single thing they share besides the hardware. Part 0 §2: the fd number does not travel, the kernel installs a reference to the same socket in each receiver's table.
compare Part 10: threads share the process · processes share almost nothing
JAVASCRIPTTHE SMALLEST CLUSTER, AND WHAT IT PRINTS
const cluster = require("node:cluster");
const http = require("node:http");
const os = require("node:os");

if (cluster.isPrimary) {
  // the primary forks and supervises; it never serves a request
  for (let i = 0; i < os.availableParallelism(); i++) cluster.fork();
  cluster.on("exit", (w) => console.log(`worker ${w.process.pid} died`));
} else {
  // every worker runs THIS branch, in its own process, on the same port
  http.createServer((req, res) => res.end(`served by ${process.pid}\n`))
      .listen(3000);
}
SHELLTHREE REQUESTS TO ONE PORT
$ curl localhost:3000   ->  served by 48213
$ curl localhost:3000   ->  served by 48214
$ curl localhost:3000   ->  served by 48215

Three requests to one port, answered by three different processes. Note the shape: the file runs once per process and the branch decides which role each copy plays — the same pattern as Part 10's isMainThread, one level out. The primary is a supervisor, not a server.

The answer is that they do not each bind it. In the default mode exactly one socket is ever created — by the primary — and the file descriptor for it is sent to the workers over a unix domain socket, using a kernel facility built for precisely this. The listen(3000) in a worker is a polite fiction: it is intercepted and turned into a request for a descriptor the primary already holds.

!Lock this in

An fd is a small integer indexing a per-process table, as Part 5 said. Sending one to another process does not send the number — it makes the kernel install the same underlying open file description in the receiver's table, under whatever number happens to be free there.

§ 1 · WHAT CHANGED FROM PART 10

Everything that was shared is now copied

Part 10 moved the JS heap from the shared column to the private one. A process moves everything else with it. This is Part 5's per-thread/shared table, redrawn one level up.

Redrawing Part 5's table one level up
§1 · what a second PROCESS gets
its own address space
a fresh page table
kernel
No pointer means anything across the boundary. A SharedArrayBuffer cannot cross here — the one door Part 10 left open is closed.
its own fd table
starting from scratch
kernel
Which is exactly why the listening socket has to be sent rather than shared, and why it arrives under a different fd number at each end.
its own V8, JIT state and heap
nothing is reused
node
Every worker recompiles the same hot functions and warms up its own inline caches. Four workers do the JIT work four times.
its own crash domain
the point of the exercise
main
A segfault or an OOM kills one worker. The primary sees the exit and forks a replacement. This is the thing threads cannot give you at any price.
shared: the machine, the port, the fd once sent
and nothing else
hw
Cores, memory bandwidth, the NIC — and any descriptor explicitly passed across.
isolation you cannot get from a thread, at a price you pay per worker
!Lock this in

The trade is exact and it is the whole reason both features exist. A worker thread shares the process and cannot survive its death. A cluster worker survives anything the others do and shares almost nothing.

§ 2 · HOW A DESCRIPTOR TRAVELS

The kernel installs it; the number is meaningless

The mechanism is a POSIX facility older than Node: a unix domain socket can carry ancillary data of type SCM_RIGHTS, and that payload is a list of file descriptors. The kernel reads it, finds the open file description each number refers to in the sender, and installs a reference to the same description in the receiver.

What actually crosses between processes
§2 · the handoff
primary: socket() + bind() + listen()
fd 17 here
kernel
One listening socket, created once. The SYN queue and accept queue from Part 8 belong to this single socket and are shared by everyone who ends up holding it.
sendmsg() with SCM_RIGHTS
over the IPC channel
kernel
The number 17 is NOT what travels. The kernel treats the ancillary data as a reference to the open file description behind it.
worker: the fd arrives as, say, fd 12
a different number, the same socket
kernel
Each process has its own fd table, so the integer differs at each end. Both now refer to one socket with one accept queue.
worker: accept() on fd 12
pops a connection
node
Part 8 again, unchanged — except that four processes are now popping from the same queue.
one socket · one accept queue · four processes holding it

Node does not expose this directly, but it is not hidden either: child.send(msg, socketOrServer) takes a handle as its second argument, and that is the same machinery. cluster is a thin layer over it.

§ 3 · ROUND-ROBIN VS THE KERNEL

Who decides which worker gets the connection

Once several processes hold one listening socket, something has to decide which of them serves the next connection. Node offers two answers, and they behave differently under load.

Round-robin or SO_REUSEPORT
§3 · the two schedulers
the control is the one where the kernel is not distributing anything

SCHED_RR — the default everywhere except Windows — keeps the accept in the primary. The primary accepts each connection and hands the connected socket to a worker in turn, which gives an even distribution at the cost of a round trip through the primary.

SCHED_NONE gives every worker the listening fd and lets them all call accept(). The kernel picks the winner, which is faster and less even — and on Linux, historically, quite badly uneven.

§ 4 · THE KERNEL DOES THE BALANCING

Several real sockets, one port, hashed by connection

There is a third arrangement that cluster does not use but which underpins a lot of modern deployment: SO_REUSEPORT lets several processes each create their own listening socket on the same port, and asks the kernel to distribute incoming connections between them.

Who holds a socket and who chooses
§4 · three arrangements
cluster SCHED_RR
one socket · primary chooses
node
One accept queue in the primary. Even distribution, one extra hop, and the primary is a single point of failure for accepting.
cluster SCHED_NONE
one socket · kernel wakes someone
kernel
One accept queue, four processes blocked on it. Whoever the kernel wakes wins — and it tends to favour whoever is already warm.
SO_REUSEPORT
four sockets · kernel hashes the connection
kernel
FOUR separate accept queues, one per process, and the kernel picks by hashing the connection tuple. No primary in the path at all, and each worker has its own backlog to fill.
the difference is where the accept queue lives and who picks
!Lock this in

The count of accept queues is the thing to hold on to. One queue means the workers are competing for the same backlog from Part 8; four queues means one slow worker fills only its own, and the kernel keeps handing it connections anyway because the hash does not know it is busy.

§ 5 · THE BILL FOR ISOLATION

Memory and warmup multiply; the machine does not

Part 10's workers were expensive because each carried a V8 isolate. A cluster worker carries that and a process: its own address space, its own copy of every module, its own JIT state, and its own share of the page cache pressure.

What four workers actually cost
§5 · the bill
the control is the case where more workers make throughput worse

The shape to notice is that throughput does not keep climbing. Up to the core count each worker gets a core and the line is roughly linear; past it, the workers are time-slicing against each other, and every extra one adds memory and context switches while adding no capacity.

Copy-on-write helps less than people expect. The pages are shared after fork(), but V8 writes to almost all of them as it warms up — every inline cache and every JIT-compiled function is a write that unshares a page.

§ 6 · THE REWRITE CLUSTER FORCES

Anything in memory is now per worker

This is the part that breaks working code. Every in-process convenience becomes wrong the moment there are four processes, and none of them fails loudly.

In-memory anything, times four
§6 · what silently breaks
an in-memory cache
four caches, four miss rates
worker
Each worker warms its own. Your hit rate falls by roughly the worker count, and nothing reports it.
an in-memory session store
the classic outage
worker
A user logs in on worker 2 and their next request lands on worker 3, which has never heard of them. Sticky sessions hide it; a restart does not.
a rate limiter counting in a Map
four times the limit
worker
Each worker independently allows the full quota. The limit you configured is now the limit per worker.
setInterval doing a cron job
it runs four times
main
Part 4's timers, per process. Four workers means four copies of the nightly cleanup running at once.
a WebSocket connection registry
a broadcast reaches a quarter
worker
Sockets live in the worker that accepted them. Broadcasting from one worker reaches only its own connections.
none of these throws · all of them are wrong
!Lock this in

The rule is blunt: with cluster, anything you keep in memory is per worker. State that must be shared belongs outside the process — Redis, the database, or a message the primary relays — and "outside the process" is the price of the isolation you bought.

§ 7 · THE REASON TO USE CLUSTER AT ALL

One crash, one worker, one fork

A worker thread cannot survive a segfault, an OOM or a hard process.exit — it shares the process, so the process goes. A cluster worker can, and that is the strongest argument for paying its costs.

JAVASCRIPTTHE SUPERVISOR IN THREE LINES
cluster.on("exit", (worker, code, signal) => {
  console.error(`worker ${worker.process.pid} died (${signal || code})`);
  cluster.fork();                      // replace it
});

The naive version has a well-known failure: if the crash is deterministic — a bad deploy, a config error — this loop forks a fresh worker to die immediately, forever, at whatever rate the machine allows. Real supervisors add a backoff and a cap, which is a large part of what a process manager gives you over these three lines.

!Lock this in

Restarting is only a strategy if the failure is transient. A crash loop is the same shape as Part 4's starvation: a process at 100% CPU, achieving nothing, with nothing in the logs saying it is stuck.

§ 8 · WHAT THE FD PASSING BUYS YOU

Replace the workers while the socket stays open

Because the listening socket belongs to the primary and not to any worker, workers can be replaced without ever closing it. Connections that arrive mid-reload queue in the accept queue — Part 8's backlog — rather than being refused.

One worker at a time
§8 · the rolling restart
primary keeps the listening socket
fd 17 never closes
kernel
Nothing is refused during the reload, because nothing ever calls close() on the listener. Arrivals sit in the accept queue.
worker.disconnect()
stop accepting, finish in flight
node
The worker stops taking new connections and is left to finish the requests it already has — the graceful half of the operation.
wait for 'exit', then fork a replacement
one at a time
main
Capacity dips by one worker rather than all of them. Doing this to every worker in turn swaps the whole fleet with no dropped connection.
a timeout, because disconnect can hang
the part people forget
worker
A worker with a long-lived connection — a WebSocket, a streaming response — will never exit on its own. Give it a deadline, then kill().
the accept queue absorbs the gap
§ 9 · THREADS, PROCESSES, OR NEITHER

Three questions that decide it

Parts 4, 10 and 11 have now given you three different answers to "my server is slow", and picking between them is not a matter of taste.

What is actually holding your thread?
§9 · the decision
nothing — it is waiting on I/O
you already have concurrency
node
Parts 5, 8 and 9. The loop sleeps in epoll_wait and one thread serves thousands of connections. Adding threads or processes here buys nothing and costs memory.
one expensive computation
worker_threads
worker
Part 10. Hashing, parsing, compression, image work. Move the CPU off the loop and keep one process, one heap and cheap communication.
all your cores, all the time
cluster
main
This part. Your handler itself is CPU-heavy across every request, so you want every core running the whole stack — and you get crash isolation with it.
you want to survive crashes
cluster, regardless of CPU
kernel
The one thing threads cannot give you. Worth the memory on its own for anything that must stay up.
measure what is holding the thread before you pick
§ 10 · THE MODELS WORTH CORRECTING

Ten claims, three of them true

These are the beliefs that produce the outages. Several are true of some other stack, which is exactly why they survive.

Which of these are true?
§10 · mental models
sort before you check — a guess you have committed to is worth more than a correction you read
!Lock this in

Nearly every wrong take here assumes the workers share something. They share the machine and whatever descriptors were explicitly passed, and nothing else.

§ 11 · THE FOUR SENTENCES WORTH KEEPING

If you remember nothing else

Cluster does not make one server faster. It runs several servers behind one port and gives you a supervisor.

What to carry away
§11 · the four sentences
only one process binds the port
§0 · §2
kernel
The listening fd is passed over a unix socket with SCM_RIGHTS. A worker's listen() is intercepted, not honoured.
the number of accept queues is the design
§4
kernel
One queue means workers compete for one backlog; SO_REUSEPORT gives each its own and lets the kernel hash between them.
anything in memory is per worker
§6
worker
Caches, sessions, rate limits and timers all multiply, and none of them fails loudly when they do.
crash isolation is the thing you cannot get otherwise
§7
main
A worker thread dies with its process. A cluster worker does not — which is the argument for cluster even when CPU is not the problem.
four sentences, one port

That closes the arc the series set out to draw. Part 5 found the one thread; Parts 8 and 9 showed how far it goes on its own; Part 4 opened the loop that drives it; Parts 10 and 11 are the two honest ways to get a second one — and both of them are more expensive than not needing one.

Back to JIT-Atlas