Buffer API

Efficient mutable byte arrays with per-call endianness, whole-buffer bulk and mask operations, and in-process compression.

Overview

Buffer is a mutable byte array. The global identifier Buffer is a constructor namespace; calling one of its constructors returns a buffer instance whose methods are invoked as b.method(...).

Sizes, indices, byte values, offsets, and encoded integers are passed and returned as Zym numbers. Integer methods truncate toward zero; byte values are masked to 8 bits. Valid indices run 0 .. size() - 1. Out-of-range get / set / decode / encode calls raise a runtime error, and bad argument types produce a Zym runtime error of the form Buffer.method(args) .... Values passed to set, append, fill, insert, has, find, and friends are coerced to a uint8 (0–255).

var b1 = Buffer.new(4)
var b2 = b1               // alias: same buffer
b2.set(0, 255)
print("%n", b1.get(0))    // 255
var b3 = b1.duplicate()   // independent copy

Typed views (packed i32 / f32 arrays sharing storage) are not provided. For byte-level work use the bulk and mask methods; for individual multi-byte values use encode* / decode* at an offset.

Construction

Creates a buffer of size zero bytes.

• size (number) — length of the new buffer in bytes

Creates an independent copy of another buffer.

Creates a buffer from an even-length hex string (0-9a-fA-F). Rejects odd-length input and any non-hex character.

Creates a buffer containing the raw bytes of a Zym string.

Creates a buffer from a list of numbers, each coerced to a byte. Rejects elements that are not numbers.

var z = Buffer.new(16)                        // 16 zero bytes
var h = Buffer.fromHex("deadbeef")           // 4 bytes: de ad be ef
var s = Buffer.fromString("hi")              // raw bytes of the string
var l = Buffer.fromList([1, 2, 3])           // each element coerced to a byte

Size & State

Returns the length of the buffer in bytes.

Returns true when size() == 0.

Drops all bytes.

Grows or truncates the buffer to n bytes. New bytes on growth are not zero-initialized, so their contents are unspecified; follow with fill(0) (or set each new byte) if a clean region is needed.

Returns: a status code (0 on success).

Sets every byte to v & 0xFF.

Returns an independent copy of the buffer.

Element Access

Returns the byte at index i. Out-of-range indices raise a runtime error.

Writes v & 0xFF at index i. Out-of-range indices raise a runtime error.

Appends a byte. Returns true on success.

Alias for append.

Inserts v before index i. Returns a status code.

Removes the byte at index i.

Removes the first byte equal to v. Returns true if a byte was found.

var b = Buffer.new(2)
b.set(0, 0xFF)
b.append(7)                // size is now 3
print("%n", b.get(0))     // 255

Ordering & Search

Reverses the buffer in place.

Sorts the bytes ascending, in place.

Returns true if any byte equals v.

Returns the index of the first byte equal to v at or after from, or -1 if none is found.

• v (number) — byte value to search for (coerced to 0–255)
• from (number, optional) — start index (default 0)

Returns the index of the last byte equal to v at or before from, or -1 if none is found. Pass -1 (or size() - 1) to search from the end. Passing 0 only searches the first byte.

Returns the number of bytes equal to v.

Returns a binary-search insertion index. Assumes sorted input. Behavior on unsorted buffers is unspecified.

• before (bool) — true returns the lower bound, false the upper bound

var b = Buffer.fromList([1, 2, 3, 2, 1])
print("%n", b.find(2, 0))      // 1
print("%n", b.rfind(2, -1))    // 3  (search from the end)
print("%n", b.count(2))        // 2

Slicing & Composition

Returns a new, independent buffer containing a copy of the range [begin, end). Negative values count from the end.

Returns true when both buffers are byte-wise equal.

Returns a new buffer containing b followed by other. The result is independent of both inputs.

var b = Buffer.fromHex("00112233")
var mid = b.slice(1, 3)         // bytes 11 22
print("%s", mid.hex())         // "1122"

Bulk & Mask Operations

These methods operate over the whole buffer in a single native call. They are the fast path for anything that would otherwise be a per-byte Zym loop: image channels, mask-based painting, byte translation tables, XOR keystreams (WebSocket frames, stream ciphers), audio gain, delta encoding, predicate-driven filtering.

A mask is any buffer where a zero byte means OFF and any non-zero byte means ON. Predicates produce strict 0/1 so masks compose cleanly under the bitwise operations, but masks built from other sources work too, without normalisation. Arithmetic in this section saturates at 0–255 (wrap is recoverable via the bitwise scalar ops, e.g. bitAndScalar(0xFF)), and comparisons are unsigned, since bytes are u8.

Every elementwise method requires its operand buffer(s) to be the same size as the receiver; mismatches raise a runtime error of the form Buffer.method(...): <name> size N does not match receiver size M. The receiver may be passed as an operand (each per-byte write is local to its index) except for mapU8's lut, which is explicitly rejected. All methods in this section mutate the receiver in place and return null; the reductions (countNonZero, any, all) return their value.

Predicates — Building Masks

Each predicate fills the receiver m as a strict 0/1 mask computed from its source buffer(s).

Sets m[i] to 1 where src[i] == v, 0 elsewhere.

Sets m[i] to 1 where src[i] != v, 0 elsewhere.

Sets m[i] to 1 where src[i] < v (unsigned), 0 elsewhere.

Sets m[i] to 1 where src[i] <= v, 0 elsewhere.

Sets m[i] to 1 where src[i] > v, 0 elsewhere.

Sets m[i] to 1 where src[i] >= v, 0 elsewhere.

Sets m[i] to 1 where lo <= src[i] <= hi, both ends inclusive, 0 elsewhere.

Per-index comparison of two buffers: sets m[i] to 1 where srcA[i] == srcB[i], 0 elsewhere.

As eqBuffer, but tests srcA[i] != srcB[i].

As eqBuffer, but tests srcA[i] < srcB[i] (unsigned).

As eqBuffer, but tests srcA[i] <= srcB[i].

As eqBuffer, but tests srcA[i] > srcB[i].

As eqBuffer, but tests srcA[i] >= srcB[i].

Bitwise Operations

Per-byte bitwise operations. When both inputs are strict 0/1 masks, bitAnd / bitOr / bitXor double as mask intersection, union, and symmetric difference.

Per-byte AND: b[i] = b[i] & other[i].

Per-byte OR: b[i] = b[i] | other[i].

Per-byte XOR: b[i] = b[i] ^ other[i], e.g. a WebSocket frame XOR mask or an XOR keystream.

Per-byte NOT: b[i] = ~b[i]. This is a full bitwise NOT, not a mask logical-not.

Sets b[i] = b[i] & (v & 0xFF) for every byte.

Sets b[i] = b[i] | (v & 0xFF) for every byte.

Sets b[i] = b[i] ^ (v & 0xFF) for every byte.

Reductions

count(v), has(v), find(v, from), and rfind(v, from) (above) cover the equality-based queries.

Returns the number of bytes not equal to 0.

Returns true if any byte is non-zero.

Returns true if every byte is non-zero. Vacuously true for an empty buffer.

Masked Writes

Sets b[i] = v wherever mask[i] != 0.

Sets b[i] = src[i] wherever mask[i] != 0.

Sets b[i] = srcA[i] where mask[i] != 0, srcB[i] elsewhere. This is a numpy-style where over buffers.

var winnerHP = Buffer.new(n)
var aliveMask = Buffer.new(n)
aliveMask.gtScalar(hp, 0)                      // strict 0/1
winnerHP.select(aliveMask, hp, defaultHP)      // alive -> hp, else default

Masked Arithmetic

Adds v to every masked byte; v may be negative, and results clamp to 0–255.

Subtracts v from every masked byte, saturating at 0–255.

Per-index saturating add of src where the mask is set.

Per-index saturating subtract of src where the mask is set.

Adds a fresh random integer in [lo, hi] (inclusive) to each masked byte, saturating.

var sandMask = Buffer.new(grid.size())
sandMask.eqScalar(grid, MAT_SAND)              // strict 0/1 mask
rChan.maskedAddNoise(sandMask, -8, 8)          // jitter only sand pixels
gChan.maskedAddNoise(sandMask, -8, 8)
bChan.maskedAddNoise(sandMask, -4, 4)

Bulk Arithmetic

Adds v to every byte, saturating at 0–255; v may be negative.

Subtracts v from every byte, saturating at 0–255.

Per-index saturating add: b[i] = b[i] + other[i], clamped to 255.

Per-index saturating subtract, clamped to 0.

Clamps every byte to [lo, hi]; lo and hi are themselves clamped to 0–255.

samples.addScalar(20)         // brighten / +20 gain, clipped at 255
samples.clampRange(40, 220)   // soft-limit

Bulk Fill & Copy

Fills every byte with a fresh random integer in [lo, hi] (inclusive). Requires 0 <= lo <= hi <= 255.

A memcpy of a same-sized buffer into the receiver.

Intra-buffer block move with memmove semantics. Overlapping ranges are well-defined.

Inter-buffer block copy: copies len bytes from src[srcOffset..] into the receiver at dstOffset.

Bulk-copies a Zym list of numbers into the receiver in one native pass; each element is masked to a byte. List length must equal receiver size. Use this, not Buffer.fromList(list), when a pre-allocated scratch buffer is being reused, such as a per-frame mirror of script-managed grid state.

Partial form of copyFromList: copies len elements from list[listOffset..] into the receiver at dstOffset. Out-of-bounds is a runtime error.

// At setup — allocate once
var grid   = []                       // script-managed state, list of bytes
var matBuf = Buffer.new(N)            // pre-allocated scratch

// Every frame — single native call, no per-element method dispatch
matBuf.copyFromList(grid)
rChan.mapU8(matBuf, lutR)             // then run the bulk pipeline...

LUT Mapping

The 256-byte LUT is the workhorse primitive for palette indexing, byte translation tables (Latin-1 → ASCII fold, case folding, EBCDIC), and producing a colour channel from a material or tile-id buffer.

Sets b[i] = lut[src[i]] for every byte. lut must be a buffer of size >= 256. src may alias b; lut may not. That case is explicitly rejected.

var palR = Buffer.new(256)
palR.set(1, 200); palR.set(2, 80); palR.set(3, 30)   // R for tile ids 1..3
var rChan = Buffer.new(tileIds.size())
rChan.mapU8(tileIds, palR)

Text Conversion

Returns a lower-case hex encoding of the buffer.

Decodes the bytes as UTF-8. Invalid sequences are replaced with the U+FFFD replacement character.

Decodes the bytes as ASCII. Bytes >= 0x80 are treated as Latin-1 rather than erroring.

var t = Buffer.fromString("Zym")
print("%s", t.hex())      // "5a796d"
print("%s", t.toUtf8())   // "Zym"

Integer Decode & Encode

All integer decoders and encoders use little-endian layout by default; every multi-byte method accepts an optional trailing endian string, "le" or "be" (see Endianness). Decoders return numbers; encoders return null. Signed variants sign-extend; unsigned variants zero-extend. An offset o is valid when 0 <= o <= size() - width, where width is 1, 2, 4, or 8 bytes for the corresponding size. The 1-byte forms accept the endian argument for API symmetry but ignore it.

Decoders

Decodes an unsigned 8-bit integer (0–255) at offset.

Decodes a signed 8-bit integer (−128 to 127) at offset.

Decodes an unsigned 16-bit integer (0–65,535) at offset.

Decodes a signed 16-bit integer (−32,768 to 32,767) at offset.

Decodes an unsigned 32-bit integer (0–4,294,967,295) at offset.

Decodes a signed 32-bit integer (−2,147,483,648 to 2,147,483,647) at offset.

Decodes an unsigned 64-bit integer at offset. Values above 2^53 silently round. See the precision note above.

Decodes a signed 64-bit integer at offset. Values outside [-2^53, 2^53] silently round. See the precision note above.

Encoders

Encodes v as an unsigned 8-bit integer at offset.

Encodes v as a signed 8-bit integer at offset.

Encodes v as an unsigned 16-bit integer at offset.

Encodes v as a signed 16-bit integer at offset.

Encodes v as an unsigned 32-bit integer at offset.

Encodes v as a signed 32-bit integer at offset.

Encodes v as an unsigned 64-bit integer at offset. Values above 2^53 lose low-order bits.

Encodes v as a signed 64-bit integer at offset. Values outside [-2^53, 2^53] lose low-order bits.

var b = Buffer.new(8)
b.encodeU16(0, 65535)
b.encodeI16(2, -1234)
b.encodeU32(4, 0xCAFEBABE)
print("%n", b.decodeU16(0))    // 65535
print("%n", b.decodeI16(2))    // -1234
print("%n", b.decodeU32(4))    // 3405691582

Float Decode & Encode

Like the integer forms, float methods address the buffer by offset and take the optional trailing endian string "le" (default) or "be". encodeHalf and encodeFloat narrow a Zym number (a double) to the target precision. Round-tripping through them is lossy for most values.

Decodes a 2-byte IEEE 754 half-precision float at offset.

Decodes a 4-byte IEEE 754 single-precision float at offset.

Decodes an 8-byte IEEE 754 double-precision float at offset.

Encodes v as a 2-byte IEEE 754 half-precision float at offset, narrowing from double precision.

Encodes v as a 4-byte IEEE 754 single-precision float at offset, narrowing from double precision.

Encodes v as an 8-byte IEEE 754 double-precision float at offset.

var f = Buffer.new(8)
f.encodeFloat(0, 3.14159)
print("%n", f.decodeFloat(0))   // ~3.14159 (narrowed to single precision)
f.encodeDouble(0, 3.14159)
print("%n", f.decodeDouble(0))  // 3.14159

Compression

Buffers can be compressed and decompressed in-process. Output is always a new, independent buffer; the source is not modified. The algo string names the algorithm and is matched case-insensitively.

Compresses the buffer using algo, at the algorithm's default level when level is omitted. Levels are applied per call. Other compression paths (such as File's File.openCompressed) are unaffected, and defaults are restored before compress returns.

• algo (string) — algorithm name, case-insensitive (see table below)
• level (number, optional) — compression level; the valid range depends on the algorithm

Returns: a new buffer, or null on failure.

A level out of range raises a runtime error of the form Buffer.compress(algo, level?): level N out of range for "<algo>" (lo..hi). Passing a level to "fastlz" or "brotli" also raises a runtime error ("<algo>" does not accept a level).

Decompresses the buffer using algo, capping the output at maxOutputSize bytes. This is the faster path because the output is allocated once, so use it whenever the decompressed size, or a tight upper bound on it, is known.

• maxOutputSize (number) — hard cap on the decompressed size in bytes

Returns: a new buffer, or null if the data is malformed, compressed with a different algorithm, or exceeds the cap.

Decompresses without a known output size. The destination buffer grows automatically as data is produced. Slower than decompress (the output may be resized several times) but useful when the decompressed length is unknown, such as arbitrary HTTP response bodies. Only "gzip", "deflate", and "brotli" are supported. These are the algorithms whose decoder is genuinely streaming. Passing "fastlz" or "zstd" (or any other algo) raises a runtime error, and those must go through decompress.

• maxOutputSize (number, optional) — hard cap on the output size; must be >= 0 (passing 0 yields an empty buffer)

Returns: a new buffer, or null if the data is malformed or the cap is exceeded mid-stream.

Algorithms

Empty input: compress on an empty buffer produces a valid empty/header stream. decompress of empty input returns an empty buffer for gzip/deflate/brotli (zero bytes cannot be inflated), and null for fastlz/zstd (empty input is treated as malformed).

Cross-tool compatibility: gzip output is readable by the standard gzip / gunzip tool, and zstd output by the zstd CLI. deflate output is raw DEFLATE without a header and is not the same as a .gz file. fastlz has no widely-used external tool. brotli decompression accepts standard .br streams from curl --compressed, the brotli CLI, and similar sources.

// Build a compressible payload by repeating a short phrase.
var phrase = "the quick brown fox jumps over the lazy dog. "
var text = ""
for (var i = 0; i < 64; i = i + 1) { text = text + phrase }

var orig     = Buffer.fromString(text)
var packed   = orig.compress("zstd", 19)
var unpacked = packed.decompress("zstd", orig.size())
print("orig=%n  packed=%n  match=%v", orig.size(), packed.size(), orig.equals(unpacked))

Endianness

Every multi-byte decode* / encode* method accepts an optional trailing endian string: "le" for little-endian, the default when the argument is omitted, or "be" for big-endian. Any other string, or a non-string value, raises a runtime error. The 1-byte methods (decodeU8 / decodeI8 / encodeU8 / encodeI8) accept the argument for API symmetry but ignore it.

var b = Buffer.new(4)
b.encodeU32(0, 0x11223344, "be")       // bytes: 11 22 33 44
print("%n", b.decodeU32(0, "be"))      // 287454020 (0x11223344)
print("%n", b.decodeU32(0))            // 1144201745 (0x44332211) - read as LE

Choose the endianness at every call site; there is no buffer-wide mode.

Examples

Encode & Decode Round-Trip

var b = Buffer.new(8)
b.encodeU32(0, 0xCAFEBABE)
b.encodeI32(4, 42)

print("size: %n", b.size())
print("hex:  %s", b.hex())

print("magic: %n", b.decodeU32(0))
print("value: %n", b.decodeI32(4))

var c = Buffer.fromString("hi")
print("utf8: %s", c.toUtf8())
print("hex:  %s", c.hex())

var d = b.concat(c)
print("combined size: %n", d.size())

Unmasking a WebSocket Payload

XOR-mask a payload with a 4-byte key by tiling the key across the payload length, then applying a single in-place bitXor.

var payload = Buffer.fromBytes(frameBody)
var key4    = Buffer.fromList([0xAA, 0xBB, 0xCC, 0xDD])
// Tile the 4-byte key across the payload length:
var keyFull = Buffer.new(payload.size())
var i = 0
while (i < payload.size()) {
    keyFull.set(i, key4.get(i % 4))
    i++
}
payload.bitXor(keyFull)   // in-place unmask

Palette Rendering with Masks

Mirror a script-managed grid into a scratch buffer, translate tile ids into a colour channel through a 256-byte LUT, then jitter one material region under a mask.

// Setup — allocate once
var matBuf = Buffer.new(N)
var rChan  = Buffer.new(N)
var palR   = Buffer.new(256)
palR.set(1, 200); palR.set(2, 80); palR.set(3, 30)   // R for tile ids 1..3
var sandMask = Buffer.new(N)

// Every frame — bulk pipeline, no per-element method dispatch
matBuf.copyFromList(grid)              // grid is a script-managed list of bytes
rChan.mapU8(matBuf, palR)              // tile id -> red channel
sandMask.eqScalar(matBuf, MAT_SAND)    // strict 0/1 mask
rChan.maskedAddNoise(sandMask, -8, 8)  // jitter only sand pixels