An emitter, a queue of Buffers, and one rule
Two earlier parts were the pieces. Part 2 was the EventEmitter; Part 3 was the Buffer. A stream is what you get when you put a queue of the second inside the first and add a single rule about when to stop filling it.
That rule is backpressure, and it is advisory. write() returns false when the internal queue is above a threshold — and it still accepts your data. Nothing enforces the rule; if you ignore the return value, the queue grows until the process runs out of memory.
if (!dest.write(chunk)) { // false = "the queue is above the threshold"
src.pause(); // stop producing
dest.once("drain", () => src.resume());
}
// write() returned false and STILL took the chunk.
// The only thing false does is tell you.Everything else in this part — flowing mode, highWaterMark, pipe versus pipeline — is machinery around that one branch, and most stream bugs are that branch missing.
A stream is an EventEmitter with a buffer and an ADVISORY rule about when to stop writing. The rule is not enforced, and the cost of ignoring it is memory.
A queue, a threshold, and a few flags
Both readable and writable streams keep an internal queue and a small amount of state. Knowing the names makes the behaviour predictable rather than magical.
write() RETURNS. It does not cap the queue, refuse data, or throw.null — neither — and §2 is about what moves it, because the transition can lose data.drain event will be emitted at all. If you never saw a false, there is no drain coming.What starts the data moving, and what misses it
A readable stream is either pushing chunks at you or waiting to be asked. Which one it is depends on what you did, and the transition is not always where people think.
The rule underneath is simple: attaching a data listener or calling resume() or pipe() starts the flow, and once it is flowing the stream will not wait for you. Anything emitted before your listener existed was delivered to nobody.
A readable stream starts in neither mode. The first thing you do to it decides the mode — and if that happens after the source has already begun producing, the chunks in between are simply lost.
A threshold that changes a return value
highWaterMark gets treated as a memory limit. It is not one, and the difference matters most exactly when you are hoping it is.
write() a million times without checking and you will have a million chunks queued. The mark was crossed on the second one.A fast producer and a slow consumer
Part 9 measures backpressure for one HTTP response. This is the general case, and the shape is the same whenever the thing producing data is faster than the thing consuming it — which is nearly always, because producing is usually memory and consuming is usually a socket or a disk.
Note what does not differ: every mode moves the same bytes and finishes in a comparable time. The only thing that changes is how much of the payload is resident in your process while it happens.
Backpressure does not make the transfer faster. It keeps the transfer from being paid for in memory — which is the cost that takes a process down rather than merely slowing it.
A correct program that dies under load
The reason this bug survives review is that the code is right in every other way. It produces correct output, it handles errors, and it works perfectly on the test file.
// looks fine, works on a 2MB fixture
for (const row of millionRows) {
res.write(JSON.stringify(row) + "\n"); // return value ignored
}
res.end();
// on a 2GB result set: every row that the socket has not accepted yet
// is queued in YOUR process. The queue is the payload.The symptom is memory that tracks payload size rather than concurrency, and it appears only when the consumer is slow — a client on a bad connection, a disk under load, a downstream service applying its own backpressure. Which means it appears in production and not in staging.
If your memory graph has the shape of your data rather than the shape of your traffic, this is the first thing to look for.
One of them leaks
pipe() handles backpressure for you, which is most of why it exists. What it does not handle is failure — and the consequence is a resource leak that no error message mentions.
This is why stream.pipeline exists and why it should be the default. It destroys every stream in the chain when any of them fails, and it gives you one callback — or one promise — for the whole thing rather than an error handler per stream.
pipe forwards data and nothing else. On failure it leaves the other end open, and a leaked file descriptor from Part 0 is exactly what that looks like a thousand requests later.
When a chunk is not bytes
A stream in object mode carries arbitrary JavaScript values instead of Buffers. Everything in this part still applies; only the units change.
write() tells you anything.null means end-of-stream in both modes, which makes it the one value an object-mode stream cannot carry.Ten claims about streams
Most of these treat a stream as something that manages memory on your behalf. It offers to tell you; it does not manage anything.
Ask two questions of any stream code: who is told to stop, and what happens to the other end when this one fails. Almost every stream bug is one of those two going unanswered.
If you remember nothing else
A stream is an emitter, a queue, and one advisory rule.
highWaterMark changes a return value and caps nothing at all.data, or calling resume or pipe, starts the flow — and anything emitted before that went to nobody.pipe leaves the other end open. Use pipeline and get one error path for the whole chain.That closes the three parts these last documents form: Part 2 was the emitter, Part 3 was the buffer, and a stream is what you get when you put one inside the other and agree on when to stop.