axialmi/jsrt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial SSA impl + docs

Browse Source
This commit is contained in:
uditdey
2026-09-02 13:16:05 +00:00
parent a8acb7706c
commit c7d44584d9
6 changed files with 1255 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/FORMAT.md
+502
View File
@@ -0,0 +1,502 @@
# quickjs-ng serialized bytecode — format specification
Produced by `qjsc -b`, and by `JS_WriteObject()` with `JS_WRITE_OBJ_BYTECODE`.
Applies to **quickjs-ng v0.16.1, `BC_VERSION 27`**. Hand-maintained against
`quickjs.c`; quickjs-ng raises `BC_VERSION` on any layout change, so re-check
after bumping `third_party/quickjs-ng`.
- **Part 1** — the wire format, as one declaration.
- **Part 2** — the algorithms it depends on.
- **Part 3** — the C structures behind it, and the enumerations.
- **Appendices** — differences from the original QuickJS; how to verify.
---
## Notation
The wire format is written as TypeScript type declarations. Two caveats,
since TypeScript is being used for its syntax rather than its semantics:
- **Field order is the wire order.** TypeScript object types are unordered;
here the declaration order is normative.
- **Widths come from the primitive aliases below**, not from `number`.
| Alias | Encoding |
|---|---|
| `u8` `u16` `u32` `u64` | Unsigned, fixed width, **native byte order** |
| `leb` | Unsigned LEB128, 7 bits per byte, at most 5 bytes |
| `sleb` | Signed LEB128, zigzag (§2.4) |
| `bit` | One bit inside a packed integer |
```ts
type u8 = number; type u16 = number; type u32 = number; type u64 = bigint;
type leb = number; type sleb = number; type bit = boolean;
```
An optional field written `x?: T` is present only under the condition stated
in its comment. A union of object types is a discriminated union: the shared
literal field is a real field on the wire, and its width is given in the
type's doc comment, since a TypeScript literal type carries no width.
Fixed-width integers are written with `memcpy`, so a file carries the
**producer's** byte order. Every mainstream target is little-endian; files are
not portable across endianness.
---
# Part 1 — Wire format
```ts
// ═══ container ═══════════════════════════════════════════════════════════
/** Repeats until end of file. The count is not stored. */
type File = SerializedObject[];
type SerializedObject = {
version: u8; // === 27, else reject
checksum: u32; // over bytes 5..end of object, §2.1
atom_count: leb; // limit 1_000_000
atoms: AtomEntry[]; // length = atom_count
root: Value; // MODULE, or FUNCTION for a classic script
};
// ═══ primitives ══════════════════════════════════════════════════════════
type JSString = {
header: leb; // (len << 1) | is_wide_char
data: u8[] | u16[]; // len Latin-1 bytes, or len UTF-16 units
};
/** `type` is a u8. */
type AtomEntry =
| { type: 0; raw: u32 } // a predefined atom number
| { type: 1 | 2 | 3; text: JSString }; // string | global_symbol | symbol
/**
* One leb. Decoding, where i = v >> 1:
* v & 1 -> tagged integer atom, value i
* i < first_atom -> predefined atom number i
* i >= first_atom -> atoms[i - first_atom]
*/
type AtomRef = leb;
// ═══ values ══════════════════════════════════════════════════════════════
/** `tag` is a u8. */
type Value =
| { tag: 1 } // NULL
| { tag: 2 } // UNDEFINED
| { tag: 3 } // BOOL_FALSE
| { tag: 4 } // BOOL_TRUE
| { tag: 5; value: sleb } // INT32
| { tag: 6; bits: u64 } // FLOAT64, IEEE-754
| { tag: 7; s: JSString } // STRING
| { tag: 8; count: leb; props: [AtomRef, Value][] } // OBJECT
| { tag: 9; count: leb; items: Value[] } // ARRAY
| { tag: 10; len: leb; mag: u8[] } // BIG_INT, two's complement LE
| { tag: 11; count: leb; cooked: Value[]; raw: Value } // TEMPLATE_OBJECT
| { tag: 12; fn: FunctionBytecode } // FUNCTION
| { tag: 13; module: Module } // MODULE
| { tag: 14; kind: u8; len: leb; offset: leb; buffer: Value } // TYPED_ARRAY
| { tag: 15; len: leb; max: leb; data: u8[] } // ARRAY_BUFFER
| { tag: 16; len: leb; max: leb; ptr: u64 } // SHARED_ARRAY_BUFFER, host pointer
| { tag: 17; pattern: JSString; lre_bytecode: JSString } // REGEXP
| { tag: 18; timestamp: Value } // DATE
| { tag: 19; inner: Value } // OBJECT_VALUE
| { tag: 20; index: leb } // OBJECT_REFERENCE
| { tag: 21; count: leb; entries: [Value, Value][] } // MAP
| { tag: 22; count: leb; items: Value[] } // SET
| { tag: 23; description: AtomRef }; // SYMBOL
// ═══ functions ═══════════════════════════════════════════════════════════
type FunctionBytecode = {
flags: FunctionFlags;
is_strict_mode: u8; // not part of flags
func_name: AtomRef;
arg_count: leb;
var_count: leb;
defined_arg_count: leb; // Function.prototype.length
stack_size: leb; // max operand stack depth
var_ref_count: leb;
closure_var_count: leb;
cpool_count: leb;
byte_code_len: leb; // in bytes
vardef_count: leb; // arg_count + var_count, or 0
vardefs: VarDef[]; // arguments first, then locals
closure_vars: ClosureVar[]; // length = closure_var_count
cpool: Value[]; // length = cpool_count; nested functions live
// here. NOTE: before the code
byte_code: Instruction[]; // byte_code_len bytes in total
debug?: DebugBlock; // present iff flags.has_debug
};
/** Packed into one u16, least significant bit first. */
type FunctionFlags = {
has_prototype: bit; // bit 0
has_simple_parameter_list: bit;
is_derived_class_constructor: bit;
need_home_object: bit;
func_kind: FuncKind; // bits 4..5
new_target_allowed: bit;
super_call_allowed: bit;
super_allowed: bit;
arguments_allowed: bit;
backtrace_barrier: bit;
has_debug: bit; // bit 11
};
type VarDef = {
name: AtomRef;
scope_level: leb; // 0 = not lexically scoped; §3.3
scope_next_plus_1: leb; // int32(v) - 1; -1 none, -2 arg-scope end
flags: VarDefFlags;
var_ref_idx?: leb; // present iff flags.is_captured
};
/** Packed into one u8, least significant bit first. */
type VarDefFlags = {
var_kind: VarKind; // bits 0..3
is_const: bit; // bit 4
is_lexical: bit; // bit 5
is_captured: bit; // bit 6
};
type ClosureVar = {
name: AtomRef;
var_idx: leb; // index in the PARENT frame
flags: ClosureVarFlags;
};
/** Packed into one leb, least significant bit first. */
type ClosureVarFlags = {
closure_type: ClosureType; // bits 0..2
is_const: bit; // bit 3
is_lexical: bit; // bit 4
var_kind: VarKind; // bits 5..8
};
type Instruction = {
opcode: u8; // §2.3 gives its size and operand format
operands: u8[]; // size - 1 bytes, fixed width per format
};
type DebugBlock = {
filename: AtomRef;
line_num: leb; // of the function's first character
col_num: leb;
pc2line_len: leb;
pc2line: u8[]; // §2.2
source_len: leb; // 0 when stripped
source: u8[]; // the function's own text
};
// ═══ modules ═════════════════════════════════════════════════════════════
type Module = {
module_name: AtomRef;
req_count: leb;
requests: AtomRef[]; // import specifiers
export_count: leb;
exports: ExportEntry[];
star_count: leb;
star_exports: leb[]; // indexes into requests
import_count: leb;
imports: ImportEntry[];
has_tla: u8; // top-level await
func_obj: Value; // the body; always a FUNCTION
};
/** `export_type` is a u8. */
type ExportEntry =
| { export_type: 0; var_idx: leb; export_name: AtomRef }
| { export_type: 1; req_module_idx: leb; local_name: AtomRef; export_name: AtomRef };
type ImportEntry = {
var_idx: leb;
import_name: AtomRef;
req_module_idx: leb;
};
// ═══ enumerations, §3.5 ══════════════════════════════════════════════════
const enum FuncKind { normal = 0, generator = 1, async = 2, async_generator = 3 }
const enum VarKind {
normal = 0, function_decl = 1, new_function_decl = 2, catch_ = 3,
function_name = 4, private_field = 5, private_method = 6,
private_getter = 7, private_setter = 8, private_getter_setter = 9,
using = 10, using_method = 11,
}
const enum ClosureType {
local = 0, arg = 1, ref = 2, global_ref = 3,
global_decl = 4, global = 5, module_decl = 6, module_import = 7,
}
```
## Notes on the above
Things the types cannot state.
1. **`first_atom` is not in the file.** It is `JS_ATOM_END` (242) when the file
contains bytecode and 1 otherwise. The engine takes it from the caller's
`JS_READ_OBJ_BYTECODE` flag, so a reader must be told which applies.
2. **Atom operands inside `byte_code` are bare `u32` with no shift.** They use
the same index space as `AtomRef`, but `JS_WriteFunctionBytecode()` writes
them with `bc_atom_to_idx()` rather than `bc_put_atom()`, so the `v >> 1`
step does not apply there.
3. **Functions nest through `cpool`, and only through `cpool`.** A
`FunctionBytecode` appears in exactly two places: `Module.func_obj`, the
module body, and `FunctionBytecode.cpool[]`, a nested function. There is no
list of functions anywhere. Since `cpool` also precedes `byte_code` — so a
reader can resolve a `push_const` operand the moment it decodes it — a
nested function's entire encoding sits inside its parent, ahead of the
parent's first instruction.
4. **`VarDef` is variable-width** because of the optional `var_ref_idx`.
Reading or skipping it unconditionally desynchronises the rest of the table.
5. **Jump operands are signed offsets from the operand field's own position**:
target = `operand_offset + value`. For the `atom_label_*` formats the base
is the label field, 4 bytes into the operands.
6. **`0xffffffff` in `checksum` disables the check.**
7. **Tags 14–16, 18, 21 and 22 never come from compiled JavaScript**; they
arise from serializing live objects. A regexp *literal* compiles to two
`STRING` constants plus a `regexp` opcode, not to tag 17.
8. **`source` is required by `Function.prototype.toString()`**, not merely
diagnostic. `qjsc -s` empties it; a second `-s` clears `has_debug` and
removes `DebugBlock` entirely.
---
# Part 2 — Algorithms
## 2.1 Checksum
Over the object's bytes from offset 5 to its end, compared with the `u32` at
offset 1.
```c
uint32_t bc_csum(const uint8_t *p, size_t n) {
uint32_t a, b, c, h = 0;
size_t i;
for (i = 0; i + 4 < n; i += 4) { /* note: <, not <= */
h += get_u32(p + i);
h *= 0x9e370001;
}
a = b = c = 0;
switch (n - i) { /* note: no case 4 */
case 3: c = p[i+2]; /* fallthrough */
case 2: b = p[i+1]; /* fallthrough */
case 1: a = p[i]; /* fallthrough */
case 0: break;
}
h += a | b << 8 | c << 16;
return h * 0x9e370001;
}
```
The loop stops while up to four bytes remain and the tail `switch` has no
`case 4`, so a trailing group of exactly four bytes contributes nothing.
Reproduce it exactly or valid files fail verification.
## 2.2 pc2line decoding
Produces checkpoints mapping a bytecode offset to a source position. State
starts at `pc = 0`, with `line` and `col` from `DebugBlock`. Constants:
`PC2LINE_BASE = -1`, `PC2LINE_RANGE = 5`, `PC2LINE_OP_FIRST = 1`.
```
while bytes remain:
op = read u8
if op == 0: // escape for large deltas
pc += read leb
line += read sleb
else:
op -= PC2LINE_OP_FIRST
pc += op / PC2LINE_RANGE
line += op % PC2LINE_RANGE + PC2LINE_BASE
col += read sleb // in both branches
emit checkpoint (pc, line, col)
```
One byte therefore encodes a small `pc` advance together with a line delta in
the range −1..+3.
## 2.3 Opcode resolution
Short opcodes fold an operand into the opcode byte, and their numbers overlap
the compiler's temporary opcodes, which never appear in a file. Resolve as the
engine does:
```c
#define short_opcode_info(op) \
opcode_info[(op) >= OP_TEMP_START ? (op) + (OP_TEMP_END - OP_TEMP_START) : (op)]
```
For v0.16.1: `OP_TEMP_START` = 185, temporary block = 19, real opcodes = 252.
Omitting the shift yields both the wrong instruction and the wrong width.
Every opcode, its size and its operand format are in
`tests/report/OPCODES.md`.
## 2.4 Signed LEB128
Zigzag over unsigned LEB128.
```
encode: leb( (2 * v) ^ -(v >> 31) )
decode: (u >> 1) ^ -(u & 1)
```
---
# Part 3 — Why the format looks like this
Facts about the engine that explain choices in Part 1. The C structures
themselves are in `quickjs.c`; only what bears on the wire is here.
## 3.1 Atoms exist only at run time
A `JSAtom` is a `uint32_t` index into `JSRuntime.atom_array`, shared by
everything in the process and dependent on what was interned and when. It is
meaningless in another process.
`AtomEntry[]` therefore corresponds to no structure. `JS_WriteObjectAtoms()`
creates it: it walks the object, collects every atom referenced, writes them
as a file-local table, and rewrites each atom field as an index into it.
Indexes below `first_atom` are engine built-ins needing no entry, which is why
only atoms the engine did not already know are written out.
## 3.2 What is not serialized
`JSFunctionBytecode` holds a GC header, a `JSContext *realm`, and a pointer to
each of its arrays. None of that is written: pointers become inline data, and
the header and realm are rebuilt on load. `JSModuleDef` is mostly link and
evaluation state — `module_ns`, `status`, `dfs_index`, `cycle_root`,
`promise`, `async_parent_modules` — and `Module` stores only the static shape
the compiler determined.
Two fields are wire-only, with no structure behind them:
- **`has_debug`** records the writer's `allow_debug` setting, which is how
`qjsc -s -s` produces a function with no `DebugBlock`.
- **`vardef_count`** is redundant with `arg_count + var_count`; the writer
emits it anyway, and 0 when `vardefs` is absent.
One field is structure-only: `JSVarDef.is_static_private` is used while
parsing private class fields and never written, which is why `VarDefFlags`
carries seven meaningful bits and not eight.
## 3.3 The scope chain
`scope_level` and `scope_next` are a fragment of the compiler's scope tree,
serialized for one consumer: direct `eval`.
`scope_level` indexes `fd->scopes[]`, an array that is **not** serialized, so
the number carries no meaning of its own — only equality does. Two variables
with the same level were declared in the same block.
Level 0 is the exception and is universal: it is the function scope, so
`scope_level == 0` means the variable is not lexically scoped — a `var`, a
hoisted function declaration, or a pseudo-variable such as `this`. Level 1 is
`ARG_SCOPE_INDEX`, the argument scope, which is also the body scope unless the
function has parameter expressions. Only then does the body get a scope of its
own, at level 2, and only then can an argument appear twice: once in the
argument half of `vardefs` and once as a lexical copy at level 1.
`scope_next` is a singly linked list, `-1` at the end. It indexes the
**locals**, that is `vardefs[arg_count + scope_next]`, not `vardefs` directly.
`js_create_function()` rebuilds the chain immediately before the function is
written: the variables of one block link to each other, and the last of them
links to the block outside it. Following `scope_next` from any variable
therefore walks its own block and then the blocks around it, outward.
```js
function h() {
var v = 1; // level 0 not in the chain
let w = 2; // level 1 next -1 end
{ let u = 3; } // level 2 next 1 -> w
}
```
`OP_eval` and `OP_apply_eval` hold the entry point: a `u16` operand carrying a
local index biased by +1. At run time `add_closure_variables()` starts there,
follows `scope_next`, and makes a closure variable out of every variable it
reaches, which is how evaluated code sees the caller's locals. Nothing else in
the engine reads either field. A function that contains no direct `eval` still
carries the chain.
Two cases do not follow the rules above:
| Where | Value | Meaning |
|---|---|---|
| `scope_next` | `-2`, stored as `0xffffffff` | `ARG_SCOPE_END`: stop here, and do not add the arguments. Written only when the function has parameter expressions — a default value or a destructuring pattern. |
| any entry with `scope_level` 0 | `scope_next` arbitrary | not maintained. The whole argument half of `vardefs` is level 0, and so is every `var`. `JS_DUMP_BYTECODE_FINAL` prints the pair only when `scope_level != 0`, for this reason. |
Because of `ARG_SCOPE_END` the field must be decoded as a **signed** 32-bit
value before the −1, matching `bc_get_leb128_int()`. Read as unsigned, −2
becomes 4294967294.
## 3.4 The capture pairing
`VarDef` and `ClosureVar` are the two halves of one capture. A parent marks a
local `is_captured` and assigns it a `var_ref_idx`; the child lists the same
variable with `closure_type = local` and a `var_idx` pointing into the
parent's frame. Either side read alone is incomplete.
`ClosureType.local` and `.arg` index the parent's variables and arguments
respectively; the remaining six index the parent's own closure variables, or
describe a global or module binding.
## 3.5 Enumeration names
| TypeScript | C |
|---|---|
| `FuncKind` | `JSFunctionKindEnum` — one bit for generator, one for async, so the four values compose |
| `VarKind` | `JSVarKindEnum` — `catch_` is `JS_VAR_CATCH`, renamed only to avoid the reserved word |
| `ClosureType` | `JSClosureTypeEnum` |
| `ExportEntry.export_type` | `JSExportTypeEnum` |
`VarKind.using_method` is a hidden local holding the cached dispose method for
the preceding `using` variable, always allocated immediately after it.
## Appendix A — Differences from the original QuickJS
Read from Bellard's `quickjs-2026-06-04`, the release current when this was
written. Both projects are active, so date the comparison before trusting it.
| Aspect | Bellard 2026-06-04 | quickjs-ng v0.16.1 |
|---|---|---|
| `BC_VERSION` | 5 | 27 |
| Header | Version byte, then the atom count | Version byte, then a `u32` checksum |
| Atom entries | String only | Type byte, then a string or a raw number |
| Function body order | Code, debug, then constant pool | Constant pool, then code, then debug |
| Byte after flags | `js_mode` | `is_strict_mode` |
| Tags | 19; no `MAP`, `SET`, `SYMBOL`, `REGEXP` | 23; later tag numbers shifted |
| Debug block | Filename and `pc2line` only | `line_num` and `col_num` as well |
| `VarDef` | No `scope_level`. `var_ref_idx` always written. Extra `has_scope` flag bit | `scope_level` present. `var_ref_idx` only when captured |
| `ClosureVar` flags | `u16` | leb128 |
| Opcodes | 244. `get_var` and its three siblings take a 3-byte `var_ref` index | 252. The same four take a 5-byte atom |
The pool order is the one that matters to a reader. ng moved it ahead of the
code on purpose — its own comment says *"write constant pool before code so
code can be disassembled on the fly at read time"* — so a `push_const` operand
resolves as soon as it is decoded.
## Appendix B — Verifying against a file
The listing tool prints one row per field, in the order given above:
```sh
cargo run -p qjsdump -- tests/bin/basic.bin
```
The parser is written in nom, one function per type in Part 1, so
`crates/qjs-bc/src/{parse,atoms,value,function,module}.rs` reads as an
executable form of this document. Indentation in the listing is depth in
`cpool`: a nested function and its sections are indented one level under the
parent whose constant pool holds them, which is also why a parent's `.code`
appears after, but less indented than, the whole of a child.
`tests/report/` holds a listing per fixture and `OPCODES.md`; `make`
regenerates both from `tests/src/`.