NODE STREAMS — the producer names an event, not a consumer

the coupling argument

Start with the wiring, not the bytes. A net.Socket produces bytes, and several things in the process want them: a parser, a TLS layer, a logger, the function you wrote. The producer has to reach all of them, and it has to do so without being rewritten every time one is added.

Suppose it called them directly. Then the producer holds a reference to each consumer, and every new consumer is an edit to the producer. Count the wires and the shape of the problem shows up immediately: producers × consumers. That product is what the design is trying to avoid.

Naming an event turns the product into a sum. The producer commits to one thing — a name, 'data' — and calls whatever sits in a list under that name. It never learns what is in the list, or how long the list is, or what any entry does with the bytes. Each consumer commits to the same name and registers interest. Wires: producers + consumers.

— this figure is an argument about wiring, not a measurement. The consumer list follows the server toggle.

The trade has a cost, and the rest of this document keeps running into it. emit walks a list and returns a boolean. Whatever a consumer returns is discarded, so there is no channel from a consumer back to the producer along the path the notification took. A consumer cannot answer slow down on that path, because the call that reached it has nowhere to put an answer. Everything Node calls backpressure is built out of machinery that sits beside the emitter contract rather than inside it.

the three nouns

One vocabulary runs through the whole document, one word per idea. A producer makes something available. An event is the name it is made available under. A consumer registers interest in that name. Registration is the only act; the interesting variable is how long it lasts.

noun	who chooses it	what it never knows
net.Socket — the producer	the runtime, when it constructs the object	who is listening, or whether anyone is
'data' — the event	the producer, and it is documented, not enforced	nothing — it is a string in a map
fn — the consumer	you, at registration time	where on the stack it will be called, or from which layer

socket.on('data', onChunk)

socket: The producer, and the receiver of this call. It owns the descriptor and the read path. After this line it holds one new reference: to onChunk.
.on: The registration verb. It stores the function in the emitter's own registration list under the key 'data' and returns the socket, which is why these calls chain. It does not call the function. It does more than store it, though: a socket is a Readable, the read side overrides this method, and registering for 'data' on a stream nobody has explicitly paused also puts it into flowing mode — which is demand, and demand ends at an armed descriptor. On a plain emitter none of that happens.
'data': A key in a map, not a type. Nothing validates it against anything the producer might emit, so a misspelling registers real interest in a name that never fires, and the failure looks like silence.
onChunk: Stored, not invoked. The function object is the entire interface between the two sides; the producer knows nothing else about your code.

Where that list lives is worth seeing once, because the same object explains the ordering rules later. It is the emitter's own _events — a null-prototype object keyed by name, so a listener for a name like constructor cannot collide with anything inherited. The first registration under a name is stored as the function itself; a second registration turns that value into an array. Both details are checked against a running Node in the accuracy harness rather than asserted here.

one request, five names

One captured request carries the whole document. A POST /orders with a 3 MB body, sent keep-alive so a second request follows on the same socket, with the header deliberately torn across three client writes, mid-field. Every number below was measured on that request.

Five names surface, in this order: 'connection', 'request', 'data', 'end', 'finish'. The ordering is not a claim about how it usually goes — the accuracy harness runs a real server and diffs that sequence against live Node on every build.

Two directions run through the same layers, and keeping them apart is most of the work. Demand descends as arming: a consumer asking for bytes ends in uv_read_start, which registers interest in a descriptor. Data ascends as notification: bytes that have already arrived travel up as a call. Pull controls whether the producer is armed. It never controls when the bytes show up.

— altitudes and ordering from the capture and from the ordering the accuracy harness checks. All three toggles change this figure.

The step marked read(2) is the only one that moves bytes across the process boundary. Everything above it is bookkeeping about memory that is already here, and everything below it happened whether or not your program was running.

write boundaries are not read boundaries

The client wrote 27 times. The server saw 51 chunks. No layer between them preserved the split, and nothing was configured to make that happen — TCP carries a byte stream, and a chunk is however much happened to be in the receive queue when a read ran.

— region boundaries and all 51 chunk sizes from the capture; the tick under the ribbon is the chunk the current step delivered.

The first three chunks are 30, 65 and 65536 bytes. The header ends at byte 147, which lands inside the third one: the parser is handed a chunk whose first 52 bytes finish the header and whose remaining 65,484 are body. That single fact drives two later sections. It is why the parser keeps state instead of bytes, and it is why 'request' can fire while 3 MB is still in flight.

The chunk sizes also say something about memory that is easy to get wrong. The C++ layer supplies a 64 KB destination per read, and no chunk in the specimen exceeds 65536 bytes. When a read underfills that destination, the bytes reach JavaScript in an allocation sized to the read, not as a window into a larger one: every chunk measured has its own ArrayBuffer, with byteOffset zero and a backing length equal to the chunk. The 8192-byte pool is a different mechanism, and it belongs to Buffer.allocUnsafe.

for await (const chunk of req) { … }

for await: Calls next() once per turn and suspends the enclosing async function until the promise it returns settles. The listeners underneath are registered once, when the iterator is created, and they stay; what is taken and given up on every turn is the ask itself.
const chunk: Bound once per turn, in a new declarative environment record for that iteration. It holds a Buffer — a view over one allocation belonging to that read alone.
req: The IncomingMessage. It is the body's producer, not the body: nothing here has read 3 MB into one value, and no req.body property exists to hold it.
of: Resolves Symbol.asyncIterator on req and calls it once, before the first turn. What comes back is built out of listener registrations on the same emitter this loop appears to have nothing to do with.

the same shape without a descriptor

fs.readFile and dns.lookup look like different machinery and are not. Neither has a descriptor the loop can watch for readiness, so the waiting is done by a thread from the libuv pool: arm the work, let the loop carry on, notify when it finishes. The layers in the rail above stay exactly where they are — a pool thread simply stands where the poll step stands. What changes is the arming, never the notification.

the shape of the objects

Four objects carry the specimen, and they do not have the same lifetime. Knowing which one is constructed where — and which outlives which — settles most of the questions the later sections raise about who is holding bytes and who is allowed to ask for more.

— construction order and lifetimes from the capture: both requests in the specimen ran over one socket.

The socket is constructed once, when the connection is accepted, and it outlives everything built on top of it. The IncomingMessage and the ServerResponse are constructed together, inside the parser callback that reports the header complete, and they are gone when that exchange finishes. In the specimen the second request built a second pair against the same connection, which is what keep-alive means at the object level: new request objects, same socket.

const req = new IncomingMessage(socket)

new: new allocates an object whose prototype link already points at IncomingMessage.prototype, then runs the constructor body against it. The link is in place before a single line of the constructor executes, which is why the object can call inherited methods on itself while it is still being built.
this: Inside that body, this is the object just allocated. The read state — the buffer, the mark, the flags — is attached here, per object. Two requests on one socket therefore have two independent buffers, and neither can see the other's.
socket: Passed in, not created. The request borrows a connection that already existed and will still exist afterwards. Nothing about this object owns a descriptor.

where req.on resolves

Every registration in this document is a method call, and every method call is a lookup. The lookup follows one link per step — the prototype link — and stops at the first object that has the property. Nothing about a stream changes that rule.

— chains read out of the running server in the capture. The object being walked follows the server toggle.

The chain answers a question the vocabulary makes easy to get wrong, and the answer is not the tidy one. A lookup for on from a request or a socket does not reach EventEmitter.prototype. It stops one link short, at Readable.prototype, because the read side defines its own. once and emit are not overridden and do resolve where you would expect.

The override is not housekeeping. Registering for 'data' on a stream that has not been explicitly paused puts it into flowing mode, so on a stream the registration is itself the demand signal — the same demand that descends into arming. That is why these two objects, which inherit the same emitter, do not resolve the same method:

lookup	on a request or socket	on a response	what that means
on	Readable.prototype	EventEmitter.prototype	the read side makes registering an act of demand; the write side does not
once	EventEmitter.prototype	EventEmitter.prototype	one-shot interest is the plain emitter's, unmodified, everywhere
emit	EventEmitter.prototype	EventEmitter.prototype	the walk over the list is never specialised by anything above it

So the stream layers did add something to registration, on one side only, and the thing they added is the connection to the machinery below. The list, the ordering and the walk are still the emitter's.

The response's chain is worth a second look. It reaches Stream through OutgoingMessage, and Writable does not appear in it at all. Writing to a response works, but it works because OutgoingMessage implements that surface itself — not because the response is a Writable. Being shaped like something and inheriting from it are different claims, and only the first one holds here.

the copy trick

A socket is both sides at once, and a prototype link is a single slot. Something has to give, and what gives is inheritance.

— the chain and all 18 copied names read off Duplex.prototype in the capture.

Duplex inherits from Readable and from nothing else. The writing half arrives by assignment: the names Writable.prototype defines and Duplex does not are copied across, one property at a time, when the module loads. Eighteen names sit on both in this build, and seventeen of them are the same function objects — reached without a link, which is why walking the chain from a socket never finds Writable anywhere on it.

The eighteenth is worth the detour. Duplex defines destroyed itself rather than taking the writable one, because it has two halves and one flag to answer for both: with only the read side marked destroyed it still reports false, and it turns true when the write side follows. Copying works for the methods that only ever concerned one side. Where the two sides have to be reconciled, copying is exactly the wrong tool, and the class does its own work.

what a Buffer is

Only as much as the specimen touches. A Buffer is a Uint8Array subclass: a view with a length and a byteOffset over an ArrayBuffer whose bytes sit outside the V8 object heap. The view is small and lives with your other objects; the bytes do not.

— view and backing sizes for the specimen's first three chunks, measured per chunk.

Every chunk the read path produced has its own allocation: backing length equal to the chunk length, byteOffset zero, all the way through. That is the measurement, over every chunk of a full transfer, and it is worth stating plainly because the opposite is the common guess.

The explanation is one step beyond what is measured here, so take it as the reading it is: the C++ layer supplies a 64 KB destination per read, and on this version a read that underfills it has its bytes moved into an allocation sized to the read before they reach JavaScript. The sizes are consistent with that — nothing exceeds 65536, and the chunks that fall short carry backings exactly their own length — but what the harness checks is the outcome, not the copy.

— pool size and offsets measured on this build; the read-path row is the same measurement as the figure above.

There is a mechanism that hands out views into one shared slab, and it is a different one. Buffer.allocUnsafe draws from a pool — consecutive small allocations come back as views into a single ArrayBuffer at rising offsets. That is where the pooled picture comes from, and it does not describe stream chunks.

const head = chunk.subarray(0, 4)

.subarray: subarray makes another view over the same allocation, with the byteOffset moved along. No bytes move. Writing through either view is visible through the other, and the allocation stays alive as long as either one does.
Buffer.from(chunk): Buffer.from copies instead. The bytes become independent — but for small sizes the copy may still land inside the shared pool, so a distinct backing ArrayBuffer is not what makes it a copy. Independence is.
why it matters here: Keeping a four-byte view of a 64 KB chunk keeps the whole 64 KB allocation reachable. With a per-read allocation that is one chunk's worth; the pooled case is where a small retained view can pin a slab that other unrelated values were also cut from.

headers and body

The header is a bounded prefix. The body is not, and Node's whole handling of the two follows from that difference rather than from anything about HTTP.

A header can be buffered safely because there is a number that caps it: http.maxHeaderSize, 16384 by default. The parser can refuse to go past it, so the memory a header can cost is known before any of it arrives. A body has no such number. The specimen's is 3 MB; the next one could be 3 GB, and the header that announces it is the same size either way.

— the first 400 bytes of the specimen, at the size the first three reads actually arrived in.

So the header is assembled across reads and handed over complete, while the body is handed over as a stream. That is the entire reason 'request' fires when it does: the parser has the one thing it can promise is finished. In the specimen that moment is byte 147 of 3,145,875, with the rest still on the wire and your handler already running.

the property that is not there

Ask a request object for a body and the answer is the ordinary one for a property nothing defines.

— the lookup walked over the chain measured on the running server; the absence is measured too.

No link in the chain defines it, so req.body evaluates to undefined — not an error, and not an empty value standing in for one. What you were handed instead is a Readable with a high-water mark of 65536, which is a way of saying: the bytes will be offered to you in pieces, and the machinery will stop asking for more once that many are waiting unclaimed.

Frameworks that put req.body back are making a choice rather than fixing an omission. To hand you one value they must read the stream to its end first, which turns an unbounded input into a resident one and moves your handler to after the last byte instead of after the header. For a 200-byte JSON payload that is obviously worth it. The specimen is 3 MB, and the same code path would hold all of it before your first line ran.

the rendezvous

Everything so far has been building the vocabulary for one claim: on, once and next() are the same act. Each one puts your interest in a list, keyed by a name. The only thing that differs between them is how long that interest lives, and how the answer is delivered when it comes.

form	interest lives for	the answer arrives as	who removes it
emitter.on(name, fn)	every occurrence, forever	a call to fn	you, explicitly, or nobody
emitter.once(name, fn)	exactly one occurrence	a call to fn, then removal	the emitter, before fn's body runs
await iterator.next()	exactly one occurrence	a promise resolution, then removal	the iterator, when it settles the ask

Read down the last two rows and the loop falls out of it. An async iterator is once in a loop: next() registers one-shot interest and hands back a promise instead of taking a function, and the loop registers again on the next turn. That is the whole difference. It is not a second notification mechanism sitting under the first.

One level down, the one-shot thing is the ask rather than the listener. Creating the iterator puts a fixed set of listeners on the stream — five, measured — and settling an ask does not remove any of them. What is taken and given up each turn is the promise and the resolve function the stream holds while it is pending. The lifetime in the table is the lifetime of the interest; the listener under it is the standing wire that carries every one of them.

— the walk follows the ordering, copy-on-emit and removal rules the accuracy harness checks against a running Node.

The list is the same list in all three cases, and so is the walk over it. What Symbol.asyncIterator adds on a Readable is registrations, not a new channel — the harness counts listeners appearing on the stream the moment a next() is called. When your promise resolves, the thing that resolved it was an emitter callback, running on a stack that started in libuv.

const { value, done } = await it.next()

it.next(): Registers one-shot interest and returns a promise immediately. If a chunk is already buffered the promise is already settled with it; if not, the resolve function is stashed where the producer's callback can reach it.
await: await suspends this function here and returns control to its caller. The frame leaves the stack with its environment records intact; the call stack under it unwinds completely, back into the event loop.
the resolution: Settling does not resume you on the spot. The continuation is queued as a microtask, so the producer's callback runs to its end first, and your loop body runs after it — on a stack that no longer contains your loop.
done: done is true only once the stream has ended. Until then, every turn takes a value and registers again, which is why the loop is the registration.

two queues, and never both

Put a producer that pushes against a consumer that pulls and there are exactly two things that can go wrong, in opposite directions. Chunks can arrive that nobody has asked for. Asks can arrive when nothing has come. So the bridge between them needs two queues: one for chunks that arrived unclaimed, one for next() calls that asked and got nothing, each holding a suspended frame.

At most one of them is ever non-empty. If a chunk is waiting, an ask is answered on the spot and nothing parks. If a frame is parked, an arriving chunk goes straight to it and nothing buffers. Which queue is filling tells you which side is winning, and that is the entire content of the word backpressure.

— the machine is driven by you, so its state is a model; the three shapes it switches between are the real ones, and the numbers in the sections below are measured.

The third mode is where the cost from the opening section comes due. events.on() gives a plain emitter the same two queues and the same loop — but there is no arming to withdraw, so a full buffer changes nothing about the producer's behaviour. The queue simply grows. Pull only controls a producer that had to be armed in the first place.

when the producer is winning

The specimen was captured twice. In the first run the consumer took 12 ms per chunk while the client sent as fast as it could, and the measurements come out one-sided: of 40 iterations, 39 asks were answered from bytes already buffered and exactly one parked. The buffer was at or over the mark in 37 of them, and in those same 37 the socket's handle was not reading.

— buffered bytes, mark and handle state sampled from the producer-faster run in the capture.

Two things in that figure are worth slowing down for. The first is that read-stop is real and reaches the descriptor: the handle stops being armed, so the bytes pile up in the kernel's receive queue instead of the process. The harness checks that separately against a live socket, not only in the capture — pausing flips the handle's reading flag, it does not merely stop callbacks.

The second is that the buffer goes well past its own mark: 129,509 bytes against a mark of 65,536. The mark is advisory. It governs whether the stream asks for more; it does not cap what a single push may add, and it cannot recall bytes already in flight below. push returning false is a report, not a refusal.

when the consumer is winning

The second run inverted it: no consumer delay, and the client pausing 8 ms between bursts. Now 45 of 48 iterations parked, the buffer never held anything at all, and the handle stayed armed throughout. The same code, the same objects, the other queue.

— the parked frame and its wait, taken from an iteration of the consumer-faster run.

This is the state that makes the async iterator worth understanding. Your frame is not on the stack while it waits. It is off it, with its environment records intact, and one reference — the resolve function — held by the stream on your behalf. The event loop is free the whole time. When bytes arrive, a libuv callback reaches that reference, and your loop body runs afterwards as a microtask.

the shape over time

Laid side by side, the two runs are not two behaviours of a machine but the same machine with the load moved. Nothing in the code changed between them.

— per-iteration waits from both runs; the consumer's own work time is the delay each run was configured with.

One run spends 13.4 ms waiting across 40 iterations and keeps a full buffer with the handle switched off. The other spends 366.2 ms waiting across 48 iterations with an empty buffer and the handle armed the whole way. Same two queues; the difference is only which one has anything in it. Backpressure is not a feature you switch on — it is the name for the first of those pictures, and the machinery that produces it is the arming from the opening section, run in reverse.

llhttp

The rendezvous section answered a producer that owns the boundaries by keeping the bytes: buffer what arrived, hand it over when someone asks. There is a second answer, and the parser is built out of it. Keep the position instead of the bytes.

The specimen tears a header across reads on purpose, because that is the ordinary case. A read ends wherever the receive queue happened to end, so a field name can be split down the middle. A consumer that kept bytes would have to hold the incomplete field somewhere and stitch it to the next arrival. A consumer that keeps state does not have to hold anything: it advances as far as the bytes allow, stops on a node in its own grammar, and resumes there when more arrive.

— the torn-header case the accuracy harness runs against a live server: four writes, cut inside the request line and again inside a field name.

What crosses each boundary is a node, not a buffer. The harness cuts at byte 12 — mid-token in the request line — and again at byte 40, inside a field name, and the headers still arrive intact and identical. That is the whole property, and it is why the headers-complete callback can fire from the middle of a chunk rather than at the end of one.

strategy	what it holds between reads	what it costs	where it appears here
keep the bytes	every byte not yet claimed	memory proportional to the backlog, and a mark to bound it	the stream's buffer
keep the position	one node in a grammar, and a partial token	constant, and the grammar has to be resumable	llhttp

Both strategies sit in the same request, one on top of the other, and neither is a fallback for the other. The stream holds the body because a body has no bounded grammar to advance through. The parser holds the header because it does. Under the net setting of the server toggle no parser exists at all, and the whole job — where a message starts, where it ends, what is a field — is yours.

outbound

The write side is the same machine with the roles exchanged. Now your code is the producer, the socket is the consumer, and the return value of write() is the only channel telling you which side is winning.

In the measured run the response is 2 MB going to a client that stalls before reading. The fourth write is where the queue first passes the mark: 65,704 bytes queued against 65,536, and write() returns false. Nothing was refused — the bytes are queued and will go out. The false is a report about depth.

— queue depth after each write, from the outbound run in the capture.

Across the run the shape repeats exactly: three writes accepted under the mark, a fourth that crosses it, a 'drain' once the queue empties, and the same again. Thirty-two drains over the whole response.

— the same writes as a shape, with the measured drain events marked underneath.

The mirror is exact enough to be worth naming. Inbound, the buffer holds chunks nobody asked for and a parked next() holds a suspended frame. Outbound, the queue holds bytes the socket has not taken and a once registration on 'drain' holds your intention to continue. Same two queues, same rule that only one of them is doing anything, and the same one-shot registration in the middle of it — which is why this section is a page rather than a chapter.

the loop, written once

Write until false, stop, register one-shot interest in 'drain', resume. That loop is mechanical, so it exists already.

— the names each call registers, read off both streams in the accuracy harness.

pipe is registrations and nothing else. It puts 'data' and 'end' on the source and the lifecycle names on the destination, and it registers 'drain' only after a write has actually returned false — one-shot interest taken out exactly when it becomes needed, then dropped. Watching that name appear in the destination's list mid-run is the clearest evidence in this document that the loop and the machinery are the same thing.

pipeline adds what pipe leaves to you: an error anywhere propagates, and every stream is destroyed rather than left registered against a partner that has already failed. The difference is not throughput. It is what happens on the unhappy path, which is the only reason to prefer it.

This section stops at the kernel's send queue. How much it accepts is governed by the connection window and the peer's reading, and Node neither sees those numbers nor models them — write() returning false is a fact about a queue in your process, not about the network.

the vocabulary ladder

Every layer in this document does the same two things: it consumes the names offered from below, and it offers names of its own upward. Nothing invents a new mechanism on the way up. The words change; the act does not.

— every name on the ladder is one this document has shown firing; the arrangement into rungs is the document's, not Node's.

Read it downward and the trail of a chunk is one long chain of registrations. Read it upward and every rung is doing what the opening section described: refusing to name a consumer, naming an event instead, and letting whoever cares register. The kernel does not know libuv exists. libuv does not know your function exists. llhttp does not know a request object will be built from what it reports. Each one publishes a name into the layer above and stops there.

Which brings the question from the beginning to its answer. for await and on are not two systems to be reconciled. The loop is a registration with a lifetime of one occurrence, re-taken every turn, answered with a promise resolution instead of a call. Underneath it, the thing that resolves the promise is a listener the stream registered on itself, fired from a libuv callback, holding bytes the kernel copied in before your program knew they existed. There was only ever one mechanism. It just has a different name on every rung.