Node internals · notification, at every altitude
How Node.js
finds out
Fifteen notification mechanisms, from a softirq up to a JavaScript listener, examined against five questions. The third question — which stack executes the handler — accounts for most of the defects.
A notification takes one of two forms. There is no third.
- A wakeup. A bit is set, a thread is marked runnable, a descriptor is marked readable. The notifier mutates a state and releases control.
- A call. The handler executes on the notifier's own stack, and the notifier blocks until that handler returns.
No component of this stack is a message bus. Where a queue exists, it was written deliberately, and that queue is where ordering and loss are decided.
A wakeup or a call
Every mechanism in this document resolves to one of two forms.
- a call
- The notifier pushes a frame onto its own stack, executes the handler inside that frame, and blocks until the frame returns.
- a wakeup
- The notifier mutates a state and releases control. The notified entity observes the mutation later, on its own stack.
- a message bus
- A channel that owns storage between the two parties. No mechanism in this document qualifies, and §06 quantifies the difference.
| a call | a wakeup | |
|---|---|---|
| stack ownership | the notifier's | the notified entity's, later |
| execution timing | before the notifier's next instruction | at a time neither party selected |
| a thrown exception | unwinds into the notifier's frames | reaches no frame at all |
| handler latency | is charged to the notifier | is charged to nobody |
| control returns | to the notifier, once the handler's frame returns | to the notifier immediately, before the handler runs |
| residual state | none | one observable state |
| late observation | unavailable, a listener registered one line later observes nothing | available, the state persists and can be polled |
| share of the fifteen | 6 mechanisms, including emit and
MakeCallback |
7 mechanisms, including
wake_up and
epoll readiness |
One mechanism is both and one is neither. The thread pool pushes a payload and then rings a doorbell; the timer heap has no notifier at all.
Both diagrams encode the form in the arrowhead.
- Solid, with a head. A call. It terminates on a stack frame.
- Dotted, with a burst. A wakeup. It terminates on a state.
Hue encodes which component implements the mechanism. The two channels are independent: calls occur inside the kernel, and wakeups cross into JavaScript.
A system call is a call in the ordinary sense. The thread continues executing on the same stack under a different privilege mode. A wakeup crossing the same boundary in the opposite direction is not a call: no userland code executes, and the only change is a field in a structure userspace cannot address.
Where a mechanism has a queue, that queue was written by somebody. It is a data structure with an owner, a capacity and a drain policy. Ordering, loss and fairness are properties of that structure, not of the notification layered on top of it.
Fifteen against five
Fifteen mechanisms carry notifications inside a Node process, from a softirq at the bottom to a promise reaction at the top. Five questions separate them, and the remainder of this document supplies the answers.
- Push or poll? And does a poller present as a push from above?
- Missed or coalesced? Edge or level. Two arrivals, one wakeup.
- Which stack executes the handler? The notifier's, the notified entity's, or a third party's.
- One subscriber or many? And what the zero case does.
- Where is the buffer, and which component owns it?
Question three carries the most weight. The majority of
blocked, reordered and never-fired defects are that question
answered incorrectly, and no API declares the answer.
socket.on('data', …) does not
state which thread executes the handler.
Atomics.wait does not state that it
blocks the one thread every other mechanism depends on.
The left column groups the mechanisms by who notifies whom. Selecting a row expands its five answers in full. The controls above the table reproject all fifteen by their answer to a single question, and those groupings cut across both the left column and the hue.
MakeCallback is a stack frame, not a signal. It is included because it determines when the microtask checkpoint runs, and because three of its five cells resolve to empty, which is itself informative. The timer heap is included for the opposite reason: it is the control case, the single row where nothing notifies anything and a callback executes regardless.
Push or poll
Almost no mechanism here is purely one or the other. The recurring shape is a push that summons a poller.
- a push
- The notifier acts once, unprompted, and stops.
- a poll
- The consumer queries the interface repeatedly until the source is drained.
| pushed by | polled by | why the split | |
|---|---|---|---|
| softirq / NAPI | the card, once per batch | the softirq, until the ring empties or the budget expires | one interrupt per frame costs more than one loop |
| epoll readiness | whichever path marked the descriptor readable | epoll_wait, on the loop thread |
the process sleeps instead of spinning |
| the timer heap | no notifier at all | the loop, comparing the clock against the heap | there is no notifier to push |
Node observes only the poll half, so the event loop presents as asking for work it has already been told about. The net module conceals the distinction without consequence, because from above a push that summoned a poller is indistinguishable from a poller that found work on its own.
The NAPI poll drains the ring with interrupts disabled. When its budget expires, the remaining work is handed to ksoftirqd, an ordinary kernel thread the scheduler can preempt. Without that bound, one saturated card starves every other thread on the CPU.
Row three is the control case: a callback executes there and nothing pushed anything. That is why this document does not define a notification as the event that causes callbacks to run.
Missed or coalesced
Two failure modes are indistinguishable from outside, and a third underlies both.
- coalescing
- Two events are delivered as one notification. Frequently benign, sometimes deliberate.
- missing
- A notification occurs with no listener registered and leaves no trace. Never benign, and almost always a race.
- a lost wakeup
- A waiter evaluates a condition, is preempted, misses the notification that would have unblocked it, and sleeps indefinitely.
Edge versus level triggering determines which of the three occurs.
| an edge | a level | |
|---|---|---|
| what it reports | a transition | a condition that persists |
| queried afterwards | returns nothing, the transition has passed | returns the condition, still true |
| listener registered later | never executes | executes, or reads true on the next poll |
| a partial read | leaves the remainder unreported | leaves the descriptor ready next turn |
| failure mode | a lost event | a late event, never a lost one |
| in libuv | one watcher, the
eventfd behind
uv_async_send |
every other watcher |
| in JavaScript | emit, and an abort listener |
a settled promise, and
signal.aborted |
libuv's single exception is deliberate. That handle does not track how
many wakeups arrived, only that one did, so it can be signalled once
and skip a read per iteration. The
property that makes edge triggering hazardous everywhere else is the
property this handle requires.
AbortSignal is the one object that is
both, which is why it appears in both columns above. It derives from
EventTarget rather than
EventEmitter, so it carries a different base class and different method
names, and it answers every other question here exactly as
emit does.
The thread pool coalesces its doorbell and loses nothing, because the completions are held in loop->wq and the doorbell reports only the queue is non-empty. A mechanism that coalesces a notification which is the payload loses events instead — a standard signal, or an edge-triggered descriptor left undrained. Identical behaviour, opposite consequence, and question five is the entire difference.
Whose stack
This is the question the document exists to answer. A handler is a function and a function executes on a stack. The mechanism determines which stack, and what sits beneath it. No API declares either.
| executes on | frames beneath it | the consequence | |
|---|---|---|---|
emit |
the calling thread's stack | the emitter's own frames, then the handler | a throw unwinds into them, and a slow listener
is charged to the emitter |
| a libuv callback | the loop thread | uv_run, then Node's main |
one slow handler stalls every other connection in the process |
| a C signal handler | whichever thread had that signal unblocked | the frames of whatever it interrupted | almost nothing is safe to execute there, so libuv writes one byte and returns |
| a microtask | no stack at all | nothing | a try/catch around the code that resolved the
promise catches nothing |
Inverting the last row gives the reason an exception thrown inside an
emit listener can be caught by
the emitter's caller: that mechanism shares frames.
The JavaScript object in hand declares nothing about which row applies. A socket is a JavaScript object holding a C++ object holding a libuv handle holding an integer. Every one of those layers is silent about which thread will be executing when a callback travels back up through it.
Two mechanisms answer twice. A thread-pool operation executes the work
on the thread pool and the
completion on the loop thread. A signal executes libuv's handler on an
arbitrary thread and the
process.on('SIGINT') listener on the loop
thread. The API presents one operation. The implementation uses two
stacks.
Nine of the fifteen mechanisms terminate with a handler on the loop
thread — the thread that executes uv_run and owns the V8
isolate. That concentration is what makes Node tractable to reason
about and what makes it fragile. One handler that does not yield
delays every other mechanism in the process, regardless of which of
the fifteen delivered it. The sharpest case is
Atomics.wait — a kernel wait queue,
entered from JavaScript, over a word in a
SharedArrayBuffer — which Node
permits on the main thread.
One, many, or none
Arity is decided in two different places, and the two ends of the stack disagree about which party decides.
| at the kernel end | in JavaScript | |
|---|---|---|
| which party decides | the notifier, per call | the subscribers, by registering |
| one | wake_up stops at the first
exclusive waiter |
no interface for it, every listener executes |
| all | wake_up_all walks the entire
list |
the default, in registration order |
| none | a silent, cost-free no-op on an empty wait queue, recorded nowhere | emit returns
false, except on 'error', which
throws and terminates the process |
| suppressing a herd | EPOLLEXCLUSIVE installs an
exclusive wait-queue entry |
not applicable, a single process |
The zero case is the informative row, because the two columns answer it in opposite directions. Nothing anywhere records a wakeup that reached no waiter, and one event name converts the same silence into a process-terminating exception.
The case the kernel accounts for is a
listening socket shared
across several processes. A connection completes, the kernel places it
on that socket's
accept queue, and every
process blocked in accept or watching
the descriptor is an entry on the same list. Exactly one process
receives the connection, and waking the remainder is wasted work.
The remedy in the last row is one flag on one wait-queue entry. Same list, same walk, terminated after the first waiter. Arity control and readiness notification are the same data structure, which is the strongest evidence in this document that these are not fifteen independent ideas.
Where the buffer is
The thesis holds that nothing here is a message bus. A bus owns storage between producer and consumer, and every property expected of one derives from that ownership.
| a message bus | every mechanism in this document | where Node supplies it | |
|---|---|---|---|
| storage | owned by the bus | none by default | a queue somebody wrote |
| replay | a consumer reads from an offset | none, a listener one line late receives nothing | none, the payload is stored by application code |
| ordering | guaranteed across producers | guaranteed only within a single drain | governed by the queue being drained |
| delivery | at least once, and configurable | none, a wakeup with no waiter is a cost-free no-op | only where a queue holds the payload |
| fan-out | N consumers, each independent | N listeners on one stack, or one waiter woken | emit and
EventTarget |
| backpressure | the producer blocks or is refused | advisory at best | the TCP window, and
highWaterMark as advice |
| ownership | the bus, as a service | none, each mechanism is a state or a frame | whoever wrote the queue |
Column three is the second half of the thesis. Every bus-like property obtainable in Node comes from a queue built deliberately, which is why ordering and loss are properties of that queue rather than of the notification above it.
Queried for the location of its data, most mechanisms have none.
- A wait queue. Sleeping tasks, and no bytes.
- A futex. One 32-bit word.
- An eventfd. One 64-bit counter.
-
emit. No storage of any kind. The arguments pass straight through to the listeners and are unreachable once it returns.
Five of the fifteen mechanisms have no buffer. Three more have a buffer that carries no payload. Where a queue does exist, somebody selected its owner, its bound, and its behaviour on overflow.
| owner | holds | bounded by | on overflow | |
|---|---|---|---|---|
| loop->wq | libuv | completed work | nothing | it allocates and grows |
| the MessagePort queue | the port | structured clones | nothing | it allocates and grows |
| the socket receive buffer | the kernel | bytes | tcp_rmem |
the window closes and the sender stops |
| the writable's buffer | the stream | queued chunks | nothing,
highWaterMark is advice |
it allocates and grows |
The last row is where a bus is most often assumed, and the backpressure
line above states why.
highWaterMark resembles a capacity and is not
one. Writes past it are accepted in full, and the buffer grows to hold
them. The value governs advice — the boolean
write() returns — and that advice may be
ignored without penalty. The failure mode is therefore unbounded memory
growth rather than an error.
The rows, one at a time
Every cell of the matrix, at length. The same fifteen mechanisms and the same five questions, one mechanism at a time, with the answers written out rather than compressed into a column.
Four origins
Four events occur to a running Node process. All four terminate in a JavaScript function being called, and the four paths that reach that point have almost nothing in common. A single running example would present them as variations on one route.
| notified by | how far it travels | terminates on | |
|---|---|---|---|
| 1 · bytes arrive | the card, then the kernel | every layer in this document | the loop thread |
| 2 · a timer expires | no notifier at all | nowhere, one thread and one data structure | the loop thread |
| 3 · a pool job finishes | a pool thread, through an eventfd | inside the process | the loop thread |
| 4 · a promise resolves | application code | inside V8, no descriptor, no thread, no system call | no stack at all |
Origin three is the informative case. A pool thread finishes work and requires the loop to observe it, so it writes to a descriptor the loop is already watching. The loop wakes itself through its own file descriptor. No channel exists between the two threads other than the one the loop already uses for sockets, which makes the general-purpose notification path and the network path the same path.
Origin two never leaves one thread. The loop computed a timeout from
the timer heap, passed it to
epoll_wait, returned, and compared
the clock against the heap. A callback executed. Nothing notified
anything, and no layer above can distinguish the two cases. A
callback firing is not evidence that a notification occurred.
What this explains
The five questions earn their place because production defects are not distributed evenly across them. Each entry below is one question answered incorrectly, and the correction is always to answer it correctly rather than to introduce a queue.
Two rows in the matrix were never notifications, and the taxonomy shows why that distinction mattered. MakeCallback produces defects about when rather than whether. The timer heap produces none of this class at all, because there is no notifier to misidentify, only a clock.
The figure runs in this page, on the same V8 that runs Node. What it displays is what the code does.
The values were produced by Node 22 during the build and are printed as they were returned. The page displays a transcript, not a simulation.
A drawing of behaviour the page cannot execute: kernel paths, thread scheduling, or a structure assembled from source. Verified where verification is possible, not measured.
The boundary between those three labels is deliberately visible. Half of this subject executes in a browser tab and half of it cannot, and concealing that would claim more than the page can demonstrate.