buffer — Buffer

Working with binary data in Node.js — Buffer.from, Buffer.alloc, encodings, converting to/from strings, and the relationship with Uint8Array.

What it is, in plain English: A Buffer in Node.js is a fixed-length sequence of raw bytes — Node's equivalent of the browser's Uint8Array (and in fact a subclass of it). Buffers are used everywhere binary data appears: reading files, HTTP bodies, cryptographic operations, network protocols. Buffer.from() creates a buffer from a string, array, or ArrayBuffer. Buffer.alloc() creates a zeroed buffer of a given size.

Where this bites you

Small Buffers don't get their own memory. Node carves them out of a shared, pre-allocated 8 KB pool. buf.buffer is that whole shared ArrayBuffer, so new Uint8Array(buf.buffer) isn't "the same bytes": it's 8,192 bytes of other Buffers' data, starting at the wrong place. Hand buf.buffer to a browser API, a worker or a typed-array parser and it silently reads the wrong data.

const buf  = Buffer.from("Hi");                 // small buffers share one pre-allocated pool
const view = new Uint8Array(buf.buffer);        // meant: "the same two bytes"

console.log(buf.length, view.length);
console.log("starts at byte", buf.byteOffset, "of a shared", buf.buffer.byteLength, "byte block");
// 2 8192
// starts at byte 320 of a shared 8192 byte block      (the offset varies)

Binary data in Node.js

In the browser, binary data is Uint8Array. In Node.js, that's Buffer — a subclass of Uint8Array with extra convenience methods. You encounter Buffers when reading files, receiving HTTP bodies, working with crypto, or using any binary protocol.

Creating Buffers

// From a string:
const buf = Buffer.from("Hello", "utf-8");
console.log(buf);          // <Buffer 48 65 6c 6c 6f>
console.log(buf.length);   // 5 (bytes, not characters)

// From a hex string:
const hex = Buffer.from("cafebabe", "hex");
console.log(hex);   // <Buffer ca fe ba be>

// From base64:
const b64 = Buffer.from("SGVsbG8=", "base64");
console.log(b64.toString("utf-8"));  // "Hello"

// Zeroed (safe — all bytes set to 0):
const empty = Buffer.alloc(16);

// Uninitialised (faster but UNSAFE — may contain old memory):
const unsafe = Buffer.allocUnsafe(16);  // always fill before use!

Converting Buffers

const buf = Buffer.from("The Codex", "utf-8");

// To string in any encoding:
buf.toString("utf-8");     // "The Codex"
buf.toString("base64");    // "VGhlIENvZGV4"
buf.toString("hex");       // "54686520436f646578"
buf.toString("latin1");    // same as utf-8 for ASCII

// To ArrayBuffer (for browser APIs):
const arrayBuf = buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength);

// Concatenate buffers:
const a = Buffer.from("Hello ");
const b = Buffer.from("World!");
const c = Buffer.concat([a, b]);
console.log(c.toString());  // "Hello World!"

That's why the "To ArrayBuffer" line above slices with buf.byteOffset and buf.byteLength. Whenever you reach for buf.buffer, take the offset and length with it:

const buf  = Buffer.from("Hi");
const view = new Uint8Array(buf.buffer, buf.byteOffset, buf.byteLength);   // only this Buffer's bytes

console.log(buf.length, view.length);
console.log(Buffer.from(view).toString());
// 2 2
// Hi

Try It Yourself

// Buffer — Node.js's raw binary data type
// Buffer is a subclass of Uint8Array

// Create from string:
const buf1 = Buffer.from("Hello, World!", "utf-8");
console.log(buf1);             // <Buffer 48 65 6c 6c 6f ...>
console.log(buf1.length);      // 13
console.log(buf1[0]);          // 72 (ASCII for 'H')
console.log(buf1.toString("utf-8"));  // "Hello, World!"

// Convert between encodings:
const b64 = buf1.toString("base64");
console.log("base64:", b64);                        // "SGVsbG8sIFdvcmxkIQ=="
console.log("back:", Buffer.from(b64, "base64").toString("utf-8")); // "Hello, World!"

const hex = buf1.toString("hex");
console.log("hex:", hex);                           // "48656c6c6f2c2..."

// Create zeroed buffer:
const zeroed = Buffer.alloc(8);
console.log(zeroed);   // <Buffer 00 00 00 00 00 00 00 00>

// Create from number array:
const buf2 = Buffer.from([0x48, 0x65, 0x6c, 0x6c, 0x6f]);
console.log(buf2.toString());  // "Hello"

// Write numbers at specific byte offsets:
const buf3 = Buffer.alloc(8);
buf3.writeUInt32BE(0xCAFEBABE, 0);  // write 32-bit big-endian
buf3.writeUInt32LE(0xDEADBEEF, 4);  // write 32-bit little-endian
console.log(buf3.toString("hex"));   // "cafebabeefdbeade"? depends on platform

// Concat multiple buffers:
const a = Buffer.from("Hello ");
const b = Buffer.from("World");
const combined = Buffer.concat([a, b]);
console.log(combined.toString());  // "Hello World"

// Buffer vs Uint8Array:
const arr   = new Uint8Array([1, 2, 3]);
const bufFromArr = Buffer.from(arr);  // Buffer is a Uint8Array subclass
console.log(Buffer.isBuffer(bufFromArr));  // true
console.log(bufFromArr instanceof Uint8Array);  // true

Reading and writing multi-byte values

const buf = Buffer.alloc(12);

// Write integers at specific offsets:
buf.writeUInt8(255, 0);          // 1 byte at offset 0
buf.writeUInt16BE(1000, 1);      // 2 bytes, big-endian, at offset 1
buf.writeUInt32LE(0xDEADBEEF, 3); // 4 bytes, little-endian, at offset 3
buf.writeFloatBE(3.14, 7);       // 4-byte float at offset 7

// Read back:
console.log(buf.readUInt8(0));          // 255
console.log(buf.readUInt16BE(1));       // 1000
console.log(buf.readUInt32LE(3));       // 3735928559 (0xDEADBEEF)
console.log(buf.readFloatBE(7).toFixed(2)); // "3.14"

// Use DataView for cleaner multi-type reading:
const dv = new DataView(buf.buffer);
console.log(dv.getUint8(0));     // 255
console.log(dv.getUint16(1, false));  // 1000 (big-endian)

Buffer in HTTP and file operations

// HTTP body is a Buffer:
req.on("data", (chunk) => {
  // chunk is a Buffer
  console.log(chunk.length, "bytes received");
});

// File content is a Buffer when no encoding specified:
const imageBuffer = await readFile("photo.jpg");  // Buffer
const imageBase64 = imageBuffer.toString("base64");
// Useful for embedding images in HTML or JSON API responses

// Streaming buffer manipulation — process chunks without loading all:
import { Transform } from "stream";
class CountBytes extends Transform {
  constructor() { super(); this.total = 0; }
  _transform(chunk, enc, cb) {
    this.total += chunk.length;
    this.push(chunk);  // pass through unchanged
    cb();
  }
}

Buffer vs Uint8Array, memory, and spec

// Buffer IS a Uint8Array (subclass):
const buf = Buffer.from([1, 2, 3]);
buf instanceof Uint8Array;  // true
Buffer.isBuffer(buf);       // true

// Shared memory: Buffer wraps an ArrayBuffer
const arr    = new Uint8Array([1, 2, 3]);
const bufView = Buffer.from(arr.buffer);  // shares memory
bufView[0] = 99;
console.log(arr[0]);  // 99 — same bytes!

// Buffer.from(string) COPIES the string's bytes (small ones into the shared pool below).
// It does NOT share the string's memory.

// allocUnsafe performance:
// Buffer.alloc(n)         — zero-fills, safe, slightly slower
// Buffer.allocUnsafe(n)   — no zero-fill, fast, must fill before use
// Buffer.allocUnsafeSlow(n) — not pooled, useful for long-lived buffers

// Node.js pre-allocates a "pool" buffer (default 8KB) shared across
// allocUnsafe calls for small buffers (< 4KB) for performance.
// This is why two allocUnsafe calls may share underlying ArrayBuffer memory.

// Safe copy:
const safe = Buffer.allocUnsafe(16);
safe.fill(0);  // now safe to use

Official Sources

The Codex links only to official documentation and the ECMAScript specification. No third-party tutorials.