Files
jsrt/docs/FORMAT.md
T
2026-09-02 13:16:05 +00:00

21 KiB
Raw Blame History

quickjs-ng serialized bytecode — format specification

Produced by qjsc -b, and by JS_WriteObject() with JS_WRITE_OBJ_BYTECODE.

Applies to quickjs-ng v0.16.1, BC_VERSION 27. Hand-maintained against quickjs.c; quickjs-ng raises BC_VERSION on any layout change, so re-check after bumping third_party/quickjs-ng.

  • Part 1 — the wire format, as one declaration.
  • Part 2 — the algorithms it depends on.
  • Part 3 — the C structures behind it, and the enumerations.
  • Appendices — differences from the original QuickJS; how to verify.

Notation

The wire format is written as TypeScript type declarations. Two caveats, since TypeScript is being used for its syntax rather than its semantics:

  • Field order is the wire order. TypeScript object types are unordered; here the declaration order is normative.
  • Widths come from the primitive aliases below, not from number.
Alias Encoding
u8 u16 u32 u64 Unsigned, fixed width, native byte order
leb Unsigned LEB128, 7 bits per byte, at most 5 bytes
sleb Signed LEB128, zigzag (§2.4)
bit One bit inside a packed integer
type u8 = number; type u16 = number; type u32 = number; type u64 = bigint;
type leb = number; type sleb = number; type bit = boolean;

An optional field written x?: T is present only under the condition stated in its comment. A union of object types is a discriminated union: the shared literal field is a real field on the wire, and its width is given in the type's doc comment, since a TypeScript literal type carries no width.

Fixed-width integers are written with memcpy, so a file carries the producer's byte order. Every mainstream target is little-endian; files are not portable across endianness.


Part 1 — Wire format

// ═══ container ═══════════════════════════════════════════════════════════

/** Repeats until end of file. The count is not stored. */
type File = SerializedObject[];

type SerializedObject = {
  version:    u8;                 // === 27, else reject
  checksum:   u32;                // over bytes 5..end of object, §2.1
  atom_count: leb;                // limit 1_000_000
  atoms:      AtomEntry[];        // length = atom_count
  root:       Value;              // MODULE, or FUNCTION for a classic script
};

// ═══ primitives ══════════════════════════════════════════════════════════

type JSString = {
  header: leb;                    // (len << 1) | is_wide_char
  data:   u8[] | u16[];           // len Latin-1 bytes, or len UTF-16 units
};

/** `type` is a u8. */
type AtomEntry =
  | { type: 0;         raw:  u32 }        // a predefined atom number
  | { type: 1 | 2 | 3; text: JSString };  // string | global_symbol | symbol

/**
 * One leb. Decoding, where i = v >> 1:
 *   v & 1              -> tagged integer atom, value i
 *   i <  first_atom    -> predefined atom number i
 *   i >= first_atom    -> atoms[i - first_atom]
 */
type AtomRef = leb;

// ═══ values ══════════════════════════════════════════════════════════════

/** `tag` is a u8. */
type Value =
  | { tag: 1  }                                              // NULL
  | { tag: 2  }                                              // UNDEFINED
  | { tag: 3  }                                              // BOOL_FALSE
  | { tag: 4  }                                              // BOOL_TRUE
  | { tag: 5;  value: sleb }                                 // INT32
  | { tag: 6;  bits: u64 }                                   // FLOAT64, IEEE-754
  | { tag: 7;  s: JSString }                                 // STRING
  | { tag: 8;  count: leb; props: [AtomRef, Value][] }       // OBJECT
  | { tag: 9;  count: leb; items: Value[] }                  // ARRAY
  | { tag: 10; len: leb; mag: u8[] }                         // BIG_INT, two's complement LE
  | { tag: 11; count: leb; cooked: Value[]; raw: Value }     // TEMPLATE_OBJECT
  | { tag: 12; fn: FunctionBytecode }                        // FUNCTION
  | { tag: 13; module: Module }                              // MODULE
  | { tag: 14; kind: u8; len: leb; offset: leb; buffer: Value }   // TYPED_ARRAY
  | { tag: 15; len: leb; max: leb; data: u8[] }              // ARRAY_BUFFER
  | { tag: 16; len: leb; max: leb; ptr: u64 }                // SHARED_ARRAY_BUFFER, host pointer
  | { tag: 17; pattern: JSString; lre_bytecode: JSString }   // REGEXP
  | { tag: 18; timestamp: Value }                            // DATE
  | { tag: 19; inner: Value }                                // OBJECT_VALUE
  | { tag: 20; index: leb }                                  // OBJECT_REFERENCE
  | { tag: 21; count: leb; entries: [Value, Value][] }       // MAP
  | { tag: 22; count: leb; items: Value[] }                  // SET
  | { tag: 23; description: AtomRef };                       // SYMBOL

// ═══ functions ═══════════════════════════════════════════════════════════

type FunctionBytecode = {
  flags:             FunctionFlags;
  is_strict_mode:    u8;              // not part of flags
  func_name:         AtomRef;
  arg_count:         leb;
  var_count:         leb;
  defined_arg_count: leb;             // Function.prototype.length
  stack_size:        leb;             // max operand stack depth
  var_ref_count:     leb;
  closure_var_count: leb;
  cpool_count:       leb;
  byte_code_len:     leb;             // in bytes
  vardef_count:      leb;             // arg_count + var_count, or 0
  vardefs:           VarDef[];        // arguments first, then locals
  closure_vars:      ClosureVar[];    // length = closure_var_count
  cpool:             Value[];         // length = cpool_count; nested functions live
                                      // here. NOTE: before the code
  byte_code:         Instruction[];   // byte_code_len bytes in total
  debug?:            DebugBlock;      // present iff flags.has_debug
};

/** Packed into one u16, least significant bit first. */
type FunctionFlags = {
  has_prototype:                bit;  // bit 0
  has_simple_parameter_list:    bit;
  is_derived_class_constructor: bit;
  need_home_object:             bit;
  func_kind:                    FuncKind;  // bits 4..5
  new_target_allowed:           bit;
  super_call_allowed:           bit;
  super_allowed:                bit;
  arguments_allowed:            bit;
  backtrace_barrier:            bit;
  has_debug:                    bit;  // bit 11
};

type VarDef = {
  name:              AtomRef;
  scope_level:       leb;             // 0 = not lexically scoped; §3.3
  scope_next_plus_1: leb;             // int32(v) - 1; -1 none, -2 arg-scope end
  flags:             VarDefFlags;
  var_ref_idx?:      leb;             // present iff flags.is_captured
};

/** Packed into one u8, least significant bit first. */
type VarDefFlags = {
  var_kind:    VarKind;               // bits 0..3
  is_const:    bit;                   // bit 4
  is_lexical:  bit;                   // bit 5
  is_captured: bit;                   // bit 6
};

type ClosureVar = {
  name:    AtomRef;
  var_idx: leb;                       // index in the PARENT frame
  flags:   ClosureVarFlags;
};

/** Packed into one leb, least significant bit first. */
type ClosureVarFlags = {
  closure_type: ClosureType;          // bits 0..2
  is_const:     bit;                  // bit 3
  is_lexical:   bit;                  // bit 4
  var_kind:     VarKind;              // bits 5..8
};

type Instruction = {
  opcode:   u8;                       // §2.3 gives its size and operand format
  operands: u8[];                     // size - 1 bytes, fixed width per format
};

type DebugBlock = {
  filename:    AtomRef;
  line_num:    leb;                   // of the function's first character
  col_num:     leb;
  pc2line_len: leb;
  pc2line:     u8[];                  // §2.2
  source_len:  leb;                   // 0 when stripped
  source:      u8[];                  // the function's own text
};

// ═══ modules ═════════════════════════════════════════════════════════════

type Module = {
  module_name:  AtomRef;
  req_count:    leb;
  requests:     AtomRef[];            // import specifiers
  export_count: leb;
  exports:      ExportEntry[];
  star_count:   leb;
  star_exports: leb[];                // indexes into requests
  import_count: leb;
  imports:      ImportEntry[];
  has_tla:      u8;                   // top-level await
  func_obj:     Value;                // the body; always a FUNCTION
};

/** `export_type` is a u8. */
type ExportEntry =
  | { export_type: 0; var_idx: leb;                              export_name: AtomRef }
  | { export_type: 1; req_module_idx: leb; local_name: AtomRef;  export_name: AtomRef };

type ImportEntry = {
  var_idx:        leb;
  import_name:    AtomRef;
  req_module_idx: leb;
};

// ═══ enumerations, §3.5 ══════════════════════════════════════════════════

const enum FuncKind { normal = 0, generator = 1, async = 2, async_generator = 3 }

const enum VarKind {
  normal = 0, function_decl = 1, new_function_decl = 2, catch_ = 3,
  function_name = 4, private_field = 5, private_method = 6,
  private_getter = 7, private_setter = 8, private_getter_setter = 9,
  using = 10, using_method = 11,
}

const enum ClosureType {
  local = 0, arg = 1, ref = 2, global_ref = 3,
  global_decl = 4, global = 5, module_decl = 6, module_import = 7,
}

Notes on the above

Things the types cannot state.

  1. first_atom is not in the file. It is JS_ATOM_END (242) when the file contains bytecode and 1 otherwise. The engine takes it from the caller's JS_READ_OBJ_BYTECODE flag, so a reader must be told which applies.
  2. Atom operands inside byte_code are bare u32 with no shift. They use the same index space as AtomRef, but JS_WriteFunctionBytecode() writes them with bc_atom_to_idx() rather than bc_put_atom(), so the v >> 1 step does not apply there.
  3. Functions nest through cpool, and only through cpool. A FunctionBytecode appears in exactly two places: Module.func_obj, the module body, and FunctionBytecode.cpool[], a nested function. There is no list of functions anywhere. Since cpool also precedes byte_code — so a reader can resolve a push_const operand the moment it decodes it — a nested function's entire encoding sits inside its parent, ahead of the parent's first instruction.
  4. VarDef is variable-width because of the optional var_ref_idx. Reading or skipping it unconditionally desynchronises the rest of the table.
  5. Jump operands are signed offsets from the operand field's own position: target = operand_offset + value. For the atom_label_* formats the base is the label field, 4 bytes into the operands.
  6. 0xffffffff in checksum disables the check.
  7. Tags 14–16, 18, 21 and 22 never come from compiled JavaScript; they arise from serializing live objects. A regexp literal compiles to two STRING constants plus a regexp opcode, not to tag 17.
  8. source is required by Function.prototype.toString(), not merely diagnostic. qjsc -s empties it; a second -s clears has_debug and removes DebugBlock entirely.

Part 2 — Algorithms

2.1 Checksum

Over the object's bytes from offset 5 to its end, compared with the u32 at offset 1.

uint32_t bc_csum(const uint8_t *p, size_t n) {
    uint32_t a, b, c, h = 0;
    size_t i;
    for (i = 0; i + 4 < n; i += 4) {      /* note: <, not <= */
        h += get_u32(p + i);
        h *= 0x9e370001;
    }
    a = b = c = 0;
    switch (n - i) {                       /* note: no case 4 */
    case 3: c = p[i+2];  /* fallthrough */
    case 2: b = p[i+1];  /* fallthrough */
    case 1: a = p[i];    /* fallthrough */
    case 0: break;
    }
    h += a | b << 8 | c << 16;
    return h * 0x9e370001;
}

The loop stops while up to four bytes remain and the tail switch has no case 4, so a trailing group of exactly four bytes contributes nothing. Reproduce it exactly or valid files fail verification.

2.2 pc2line decoding

Produces checkpoints mapping a bytecode offset to a source position. State starts at pc = 0, with line and col from DebugBlock. Constants: PC2LINE_BASE = -1, PC2LINE_RANGE = 5, PC2LINE_OP_FIRST = 1.

while bytes remain:
    op = read u8
    if op == 0:                                  // escape for large deltas
        pc   += read leb
        line += read sleb
    else:
        op   -= PC2LINE_OP_FIRST
        pc   += op / PC2LINE_RANGE
        line += op % PC2LINE_RANGE + PC2LINE_BASE
    col += read sleb                             // in both branches
    emit checkpoint (pc, line, col)

One byte therefore encodes a small pc advance together with a line delta in the range −1..+3.

2.3 Opcode resolution

Short opcodes fold an operand into the opcode byte, and their numbers overlap the compiler's temporary opcodes, which never appear in a file. Resolve as the engine does:

#define short_opcode_info(op) \
    opcode_info[(op) >= OP_TEMP_START ? (op) + (OP_TEMP_END - OP_TEMP_START) : (op)]

For v0.16.1: OP_TEMP_START = 185, temporary block = 19, real opcodes = 252. Omitting the shift yields both the wrong instruction and the wrong width. Every opcode, its size and its operand format are in tests/report/OPCODES.md.

2.4 Signed LEB128

Zigzag over unsigned LEB128.

encode: leb( (2 * v) ^ -(v >> 31) )
decode: (u >> 1) ^ -(u & 1)

Part 3 — Why the format looks like this

Facts about the engine that explain choices in Part 1. The C structures themselves are in quickjs.c; only what bears on the wire is here.

3.1 Atoms exist only at run time

A JSAtom is a uint32_t index into JSRuntime.atom_array, shared by everything in the process and dependent on what was interned and when. It is meaningless in another process.

AtomEntry[] therefore corresponds to no structure. JS_WriteObjectAtoms() creates it: it walks the object, collects every atom referenced, writes them as a file-local table, and rewrites each atom field as an index into it. Indexes below first_atom are engine built-ins needing no entry, which is why only atoms the engine did not already know are written out.

3.2 What is not serialized

JSFunctionBytecode holds a GC header, a JSContext *realm, and a pointer to each of its arrays. None of that is written: pointers become inline data, and the header and realm are rebuilt on load. JSModuleDef is mostly link and evaluation state — module_ns, status, dfs_index, cycle_root, promise, async_parent_modules — and Module stores only the static shape the compiler determined.

Two fields are wire-only, with no structure behind them:

  • has_debug records the writer's allow_debug setting, which is how qjsc -s -s produces a function with no DebugBlock.
  • vardef_count is redundant with arg_count + var_count; the writer emits it anyway, and 0 when vardefs is absent.

One field is structure-only: JSVarDef.is_static_private is used while parsing private class fields and never written, which is why VarDefFlags carries seven meaningful bits and not eight.

3.3 The scope chain

scope_level and scope_next are a fragment of the compiler's scope tree, serialized for one consumer: direct eval.

scope_level indexes fd->scopes[], an array that is not serialized, so the number carries no meaning of its own — only equality does. Two variables with the same level were declared in the same block.

Level 0 is the exception and is universal: it is the function scope, so scope_level == 0 means the variable is not lexically scoped — a var, a hoisted function declaration, or a pseudo-variable such as this. Level 1 is ARG_SCOPE_INDEX, the argument scope, which is also the body scope unless the function has parameter expressions. Only then does the body get a scope of its own, at level 2, and only then can an argument appear twice: once in the argument half of vardefs and once as a lexical copy at level 1.

scope_next is a singly linked list, -1 at the end. It indexes the locals, that is vardefs[arg_count + scope_next], not vardefs directly. js_create_function() rebuilds the chain immediately before the function is written: the variables of one block link to each other, and the last of them links to the block outside it. Following scope_next from any variable therefore walks its own block and then the blocks around it, outward.

function h() {
  var v = 1;                        // level 0            not in the chain
  let w = 2;                        // level 1  next -1   end
  { let u = 3; }                    // level 2  next 1    -> w
}

OP_eval and OP_apply_eval hold the entry point: a u16 operand carrying a local index biased by +1. At run time add_closure_variables() starts there, follows scope_next, and makes a closure variable out of every variable it reaches, which is how evaluated code sees the caller's locals. Nothing else in the engine reads either field. A function that contains no direct eval still carries the chain.

Two cases do not follow the rules above:

Where Value Meaning
scope_next -2, stored as 0xffffffff ARG_SCOPE_END: stop here, and do not add the arguments. Written only when the function has parameter expressions — a default value or a destructuring pattern.
any entry with scope_level 0 scope_next arbitrary not maintained. The whole argument half of vardefs is level 0, and so is every var. JS_DUMP_BYTECODE_FINAL prints the pair only when scope_level != 0, for this reason.

Because of ARG_SCOPE_END the field must be decoded as a signed 32-bit value before the −1, matching bc_get_leb128_int(). Read as unsigned, −2 becomes 4294967294.

3.4 The capture pairing

VarDef and ClosureVar are the two halves of one capture. A parent marks a local is_captured and assigns it a var_ref_idx; the child lists the same variable with closure_type = local and a var_idx pointing into the parent's frame. Either side read alone is incomplete.

ClosureType.local and .arg index the parent's variables and arguments respectively; the remaining six index the parent's own closure variables, or describe a global or module binding.

3.5 Enumeration names

TypeScript C
FuncKind JSFunctionKindEnum — one bit for generator, one for async, so the four values compose
VarKind JSVarKindEnum — catch_ is JS_VAR_CATCH, renamed only to avoid the reserved word
ClosureType JSClosureTypeEnum
ExportEntry.export_type JSExportTypeEnum

VarKind.using_method is a hidden local holding the cached dispose method for the preceding using variable, always allocated immediately after it.

Appendix A — Differences from the original QuickJS

Read from Bellard's quickjs-2026-06-04, the release current when this was written. Both projects are active, so date the comparison before trusting it.

Aspect Bellard 2026-06-04 quickjs-ng v0.16.1
BC_VERSION 5 27
Header Version byte, then the atom count Version byte, then a u32 checksum
Atom entries String only Type byte, then a string or a raw number
Function body order Code, debug, then constant pool Constant pool, then code, then debug
Byte after flags js_mode is_strict_mode
Tags 19; no MAP, SET, SYMBOL, REGEXP 23; later tag numbers shifted
Debug block Filename and pc2line only line_num and col_num as well
VarDef No scope_level. var_ref_idx always written. Extra has_scope flag bit scope_level present. var_ref_idx only when captured
ClosureVar flags u16 leb128
Opcodes 244. get_var and its three siblings take a 3-byte var_ref index 252. The same four take a 5-byte atom

The pool order is the one that matters to a reader. ng moved it ahead of the code on purpose — its own comment says "write constant pool before code so code can be disassembled on the fly at read time" — so a push_const operand resolves as soon as it is decoded.

Appendix B — Verifying against a file

The listing tool prints one row per field, in the order given above:

cargo run -p qjsdump -- tests/bin/basic.bin

The parser is written in nom, one function per type in Part 1, so crates/qjs-bc/src/{parse,atoms,value,function,module}.rs reads as an executable form of this document. Indentation in the listing is depth in cpool: a nested function and its sections are indented one level under the parent whose constant pool holds them, which is also why a parent's .code appears after, but less indented than, the whole of a child.

tests/report/ holds a listing per fixture and OPCODES.md; make regenerates both from tests/src/.