Node internals · one byte, end to end

Whose bytes
are these?

A single byte, from the socket to the response. Eleven stops, and at every one of them the same question the API never answers: do you own this, did you borrow it, or is it a copy?

11 sections10 figures11 stopsv1.0

A Buffer exists because the collector moves things and a syscall cannot chase them. So the bytes live where the collector will not touch them — and once they do, a Buffer is three separable things: an allocation, a window onto it, and a reading of it.

Every operation owns, borrows, or copies the first two. The third is a guess that nothing verifies.

§00

The claim

Follow one byte. It arrives on a wire, sits in kernel memory, gets copied into an allocation Node made, acquires a JavaScript object, is narrowed, decoded, gathered up with others, handed to something that does not know where it came from, and finally written back out to the kernel it started in.

At every one of those stops something either owns the bytes — holds an allocation nothing else looks into — borrows them, holding a window onto memory somebody else allocated, or copies them into new memory that is independent from then on. Three verbs. Every operation in the Buffer API is exactly one of them, and the method names are no guide to which.

The clearest evidence for that last sentence: buf.slice() returns a window onto the original and writing through it changes the original. Uint8Array.prototype.slice() — the same method name, on the class that Buffer extends, called on the same object — returns a copy. One name, two meanings, one inheritance chain.

the ledger

From §04 onward every section carries a strip recording the stops so far: what happened, which verb, who owns the bytes now, and what is still alive because of it. It only ever grows. By §10 it is the whole taxonomy, assembled by travelling rather than read off a table.

§01

What the collector cannot chase

Start before JavaScript exists. A network card writes frames into a ring in memory it was given at boot; the kernel takes them apart and the payload ends up in a socket receive buffer, sized by tcp_rmem. That memory belongs to the kernel. Your process cannot address it, and no object in your program refers to it. What your process holds instead is a file descriptor — an integer, and the only thing about that connection that crosses into userspace at all.

For those bytes to reach JavaScript, something has to read them into memory your process owns — and the address it hands the kernel has to still be valid when the kernel writes there. That is the constraint the whole subject grows out of. The loop discovers there is something to read during the poll phase, when epoll_wait reports the descriptor ready; the system call that follows needs an address to write to.

V8's collector relocates live objects when it compacts the heap. An object's address is not stable across a collection, so a pointer handed to the kernel could be pointing at nothing by the time the kernel uses it. There are only two ways out: pin the object so the collector cannot move it, or put the bytes somewhere the collector does not manage at all. Node takes the second.

FIG 01 Why the bytes are not in the heap modelled

Everything else follows from that one decision. If the bytes are outside the heap, somebody has to allocate and free them explicitly, which is why there is a pool. If they are outside the heap, the heap size limit does not bound them, which is why a process can be three megabytes of heap and a gigabyte of resident memory. And if they are raw bytes rather than a JavaScript value, they carry no type, which is why reading them is a guess.

a name worth being careful with

External memory does not mean outside the process. It means outside the region V8's collector manages. It is ordinary memory in the same address space, allocated by C++, and it is reported separately for exactly that reason.

§02

Somewhere to put them

Before the read can happen, libuv asks Node where to put the bytes. Node's answer, for anything small, is not a fresh allocation — it is a slice of a shared 8 KB slab it keeps around and hands out in pieces.

This is the first borrow, and it happens before you have written a line of code. A hundred-byte Buffer is a window at some offset into an allocation you did not make, alongside other people's bytes, and the rest of that slab stays alive for as long as your hundred bytes do.

FIG 02 A hundred bytes in an eight-kilobyte slab executed in node

Three allocators, three answers. Buffer.allocUnsafe takes from the pool and does not clear what it finds. Buffer.alloc makes its own allocation and zeroes it. Buffer.allocUnsafeSlow makes its own and does not. Only the first shares, and it is the one the I/O path uses, because clearing memory you are about to overwrite is waste.

why an unsafe allocator exists at all

It used to be reachable by accident. The old constructor branched on its argument's type — a number allocated that many uninitialised bytes, a string encoded it — so a value taken straight from JSON could flip between the two and hand back whatever the pool last held. The constructor is gone and the fast allocator is still here, under a name that says what it is.

§03

A window appears

The read has happened. The bytes are in the slab. Now a JavaScript object appears over them, and it is worth being precise about how many things that involves, because the answer is three and most confusion is treating them as one.

An ArrayBuffer is a fixed-length region of raw bytes. It has no elements and no type; it cannot be read or written directly. It exists to be pointed at.

A typed array points at one. It carries three numbers — which ArrayBuffer, at what byteOffset, for how many elements — and it reads the bytes at that window as elements of one fixed type. A Uint8Array reads each byte as a number from 0 to 255. A DataView is the same idea without a fixed element type: every read states its own width and byte order, which makes endianness something you state rather than something you inherit from the machine.

A Buffer is a Uint8Array — actually, genuinely, by inheritance — with extra methods bolted on for encodings and sized reads. That has two consequences that run through the rest of this document. It passes to any API that takes a typed array. And it inherits methods whose meaning it has changed.

FIG 03 The window, moved executed here

Two windows onto one allocation are two objects with no knowledge of each other. Neither holds a list of the other's readers, and writing through one is not an event the other can observe — it simply reads different bytes next time it is asked. Aliasing in a Buffer is not a feature with a name. It is the absence of any mechanism that would prevent it.

§04

Own, borrow, copy

The byte reaches your 'data' handler as a Buffer, having crossed a C++ object and a net module that both had the chance to say something about ownership and did not. Everything you can do with it now falls into three verbs, and picking the wrong one is the source of most of what follows.

Owning means holding an allocation nothing else has a window onto: it is freed when you drop it, and nobody can write into it behind your back. Borrowing means holding a window onto somebody else's allocation: free, instant, and it keeps their allocation alive. Copying means new bytes, written once, independent from then on: it costs a pass over the memory and buys you isolation.

The obvious test for which one you got is to compare .buffer on both objects, and in Node that test is wrong. Two independent copies taken from the pool share one ArrayBuffer, so the structural check reports borrowing about a copy. The test that holds is behavioural: write through the result and see whether the source moved.

FIG 04 Ten operations, sorted by what they actually do executed here

The pair worth staring at is buf.slice() against Uint8Array.prototype.slice(). Buffer overrides the inherited method and reverses its meaning: the superclass copies, the subclass returns a view. Both are reachable on the same object. subarray is the honest spelling of what Buffer's version does, which is why the other one is discouraged rather than fixed — fixing it would silently change what existing code copies.

§05

What borrowing costs

Borrowing looks free. No allocation, no copy, constant time whatever the size. The bill arrives later and in a different currency: a view keeps its whole ArrayBuffer reachable, however small the window is.

Take fifty megabytes off a socket, find the ten bytes you wanted, and keep a subarray of them. The fifty megabytes cannot be collected. Not the ten bytes' worth — all of it, because the only thing keeping any of it alive is a reference to the allocation, and your view holds one.

FIG 05 Ten bytes holding fifty megabytes executed in node

The fix is to stop borrowing: buf.toString() if you wanted a string, or a copy of the window if you wanted bytes. Either way the point is to end up holding something whose size matches what you care about. Retention like this does not show up as a leak in the usual sense — nothing is growing without bound, and every allocation is legitimately reachable. It shows up as a process that is inexplicably large.

the pool version of the same bug

It is smaller and far more common. Every pooled Buffer holds its 8 KB slab open, so a cache of ten thousand small pooled Buffers can hold tens of megabytes rather than the few it accounts for. Buffer.allocUnsafeSlow exists for precisely this: anything long-lived should own its allocation rather than borrow a corner of a shared one.

§06

A reading, not a conversion

The third of the three things a Buffer is. An allocation holds bytes and a window names which of them, but neither says what the bytes mean — and nothing anywhere in the system does.

toString takes an encoding name and produces a string. It is tempting to read that as a conversion, as though the bytes were in one format and are now in another. They are not. The bytes do not change; a rule was applied to decide how many of them make one character and which character that is. Apply a different rule and you get a different string from the same bytes.

FIG 06 One row of bytes, six readings executed here

Nothing validates the guess. Read arbitrary bytes as utf8 and any sequence the rule cannot interpret becomes the replacement character — one string, no error, and the original bytes unrecoverable. latin1 is the only reading here that round-trips arbitrary bytes through a string and back unchanged, because it maps each byte to the code point of the same value and never consumes two.

The variable-width readings have a second hazard that only appears once bytes arrive in pieces. A character encoded in four bytes can be split across a chunk boundary, and decoding each chunk on its own turns one character into damage on both sides of the seam. The StringDecoder holds back the trailing bytes of an incomplete character until the next chunk arrives; gathering the chunks first and decoding once also works. The reason the bug survives so long in so many codebases is that on ASCII test data all three strategies agree exactly.

§07

What copying costs

Copying buys isolation, and it is priced per byte. Once, over a whole list, that price is the size of the result and nothing more. Buffer.concat allocates the total, walks the inputs, and is done.

The version that appears in real code is the other one: an accumulator, concatenated with each chunk as it arrives. It reads more naturally than gathering into an array, it is correct, and it copies the entire accumulated result every time round the loop. For n chunks that is n squared over two chunk-lengths of copying to produce n chunk-lengths of output.

FIG 07 The same ten megabytes, gathered two ways executed in node

toString is a copy too, and one that crosses back into the heap the bytes were kept out of in the first place. It has a ceiling the byte side does not: a string cannot exceed V8's maximum string length, so a Buffer that fits in memory can still fail to become a string. Decoding a large payload is therefore two commitments at once — the copy, and the ceiling.

§08

Handing it over

Everything so far stayed inside code that understands what a window is. The boundary is where that stops being true, and three different things go wrong there.

The first is the .buffer property. It hands over the whole allocation, not your window — so passing it to anything that takes an ArrayBuffer passes the entire slab, including whatever else is in it. Anything reading it is reading other people's bytes. The byteOffset and length have to travel with it, and most code that reaches for .buffer does not carry them.

The second is alignment. A multi-byte view has to begin at an offset divisible by its element size. Node keeps pool offsets on eight-byte boundaries deliberately, so the pool itself is safe — but a subarray at an odd offset is not, and a view built over it fails with an error about multiples that mentions nothing about Buffers.

The third is what happens when the bytes cross a thread. A structured clone copies them, and the copy arrives as a plain Uint8Array — the prototype does not survive the crossing, so the encoding methods are gone. Transferring instead moves the allocation, and leaves the original detached: its byteLength becomes zero and every view onto it reads as empty, including views held by code that never heard about the transfer.

FIG 08 Three ways the window stops being honoured executed here

Detaching is the one place in JavaScript where ownership becomes directly observable. Every other question in this document has to be answered by writing through something and watching what changed. Here the object simply reports that its bytes are gone, and every window onto them agrees.

§09

Back to the kernel

The byte goes out the way it came in. Writing to a socket hands the kernel an address and a length, and the kernel copies from there into its own send buffer — which means that between the call and the completion, your bytes are being read by something that is not your program.

That is a borrow, in the direction nobody thinks about. For the duration of it the Buffer is not yours to modify: reuse it for the next chunk before the write completes and what goes on the wire is whichever version the kernel happened to read. Nothing enforces this and nothing detects it. The write does not fail; the bytes are just wrong, occasionally, under load.

FIG 09 The window where the bytes are not yours modelled

The callback exists to close that window. It is not a delivered signal and it is not an arrived signal — it says the kernel has finished reading your memory and you may have it back. Which is the same shape as everything else here: an ownership boundary, unmarked in the type system, communicated by convention.

§10

What is still alive

The byte has been through eleven stops. The reckoning is what is still held, by whom, and whether anything is counting.

Something is counting, but not the thing most people watch. process.memoryUsage() reports heapUsed for the region the collector manages and external and arrayBuffers for the bytes outside it. The heap size limit — the number --max-old-space-size sets — bounds only the first. Buffers are bounded by the machine.

FIG 10 Four numbers, and which one the limit governs executed in node

Read down the verb column. The bytes were copied exactly twice on the way through — once by the kernel on the way in, once by whatever decoded or gathered them — and borrowed everywhere else. That ratio is what makes Node fast at moving bytes, and it is also the reason that a mistake about which verb applied is so hard to see: most of the time the borrow is right, free, and invisible.

executed here The figure runs in this page, on the same engine Node runs. What it shows is what the code does.

executed in node The numbers came out of Node 22 during the build and are printed as they arrived. A transcript, not a simulation.

modelled A drawing of something the page cannot run — the collector compacting, the kernel reading out of your memory. Accurate as far as it is checked, and not a measurement.

Two of the ten figures are drawings, and both are drawings of the same thing: a moment when memory is being used by something that is not JavaScript. That is not a coincidence. It is the boundary the whole subject is about, and it is exactly the part a JavaScript engine cannot show you.

Back to JIT-Atlas