GC roots: a precise root model for the interpreter and both JITs

Written 2026-10-29 against master c816f3f9ae, before any implementation. It was prompted by two suspected JIT root bugs (jit_stale_spill_slot, jit_inline_local_root) and a run of collector fixes that each patched one consequence of the same assumption. This was a design for review when it was written, and nothing below had landed. Phases 0 to 6 have since merged (#1511 was the last); the status blocks that follow are the record of what was built and how it differs, and the rest of the page is the design they implemented.

Status, 2026-10-30. The user chose the full redesign over a targeted fix.

1. Summary

The collector does not know which words in a frame are references. It finds roots in three ways:

The untyped words are:

Neither JIT backend tells the collector which of those words hold a live reference at a safepoint.

That guess has three consequences. Each is demonstrated below on a fresh Windows x64 build of master:

  1. Dead objects stay reachable. A spill slot that is never cleared, or never zeroed at entry, keeps whatever it last held. jit_stale_spill_slot keeps a dead 32 MB array alive (§3.1).
  2. Integers are rewritten. The nursery moves objects, and fixup "relocates" any untyped word equal to a forwarded object's address. That includes an Int that holds an address, such as GetInstanceID(). In the reproducer, 242 of 2,000 such integers changed value across a minor collection, interpreted and compiled alike (§3.3). This is a correctness bug, not a leak.
  3. Every new slot kind became a separate fix. The and/or slot, pending registers at a loop poll, self's frame copy, frame layout and closure captures were each fixed one at a time (§4).

The proposal has three parts:

Conservative scanning of frames and stacks is then removed, along with the window and the symptom patches that exist only because of it (§7, phase 5).

Verdicts on the two suspected bugs:

2. Evidence: method and environment

2.1 --jit=off does not mean "interpreter only"

--jit=off (and OBJECK_JIT_DISABLE=1) only raises the auto-JIT threshold. A native method is still compiled. Under --jit=off, OBJECK_JIT_REPORT=1 prints [jit] Probe2:Leave:: compiled for a native method, and System.String:Get is compiled too, because library methods are native as well.

The usage text says "'off' runs the interpreter only". The earlier finding that jit_stale_spill_slot "fails with the JIT off" came from this: the test's methods are all native. With native removed:

jit_stale_spill_slot, no native--jit=off--jit=1default (10)
-opt s0PASS (1,049,378 B)FAIL (34,603,835 B)PASS
-opt s3PASSFAILPASS

Two consequences follow:

3. What the two tests actually show

3.1 Stale spill slots (real)

Path 1: entry zeroing misses TMP_REG_0 (AMD64, both ABIs).

JitAmd64::RegisterRoot zeroes ((offset - 8) >> 3) + 10 words upward from the lowest local. On Win64 the locals plus the RED_ZONE word take offset / 8 words, so the count stops one word short of TMP_REG_0 (RBP-32). POSIX misses TMP_REG_0 (RBP-88) the same way.

The - 8 dates from the six-slot layout (+ 6). 5555f2960b (2026-09-09) widened the slots to ten without changing it.

In the test, Leave() spills the 32 MB array to TMP_REG_0 across the StaleSmall constructor and returns. Churn() is then entered at the same frame pointer and never writes that slot, so every collection under Churn() finds the array there.

Changing the count to (offset >> 3) + 10 makes s0 pass. That one-line change is kept only as evidence, on the local branch scratch/amd64-tmp-reg0-entry-zeroing-evidence; it is not the fix. The main checkout's deploy build from 2026-10-22, which has the same count, passes the same test, so frame placement decides whether this path triggers. The 10-24 call-stack change (13daaacfb2) is the likely trigger; this was not bisected.

Path 2: spills are never cleared (AMD64 and ARM64).

At -opt s3, the compiler inlines Leave into Run as bytecode. MakeBig's result is pending in Run's own frame across StaleSmall:New, so ProcessStackCallback spills it to Run's TMP_REG_0, reloads it for Keep, and leaves the slot as it is. Run stays live while Garbage() runs.

A trace build that prints the window words the collector finds allocated shows the array in every collection:

[gc] jit frame 'JitStaleSpillSlot:Run:' mem=...EB60 dclrs=2
[gc]   window 10 @...EBC0 -> 0000027E5B6D0060 size=33554457

Window word 10 on Win64 is RBP-32, which is TMP_REG_0. ARM64 zeroes all six TMP_X slots at entry (RegisterRoot), so it has path 2 only.

Severity: the effect is retention, at most one object per stale slot per live frame. Nothing is freed while still in use. It becomes serious when the object is large, as with the 32 MB array here.

3.2 Inlined callee locals (refuted)

The claim is that the AMD64 JIT pastes small callees into the caller and puts their locals below the walked region. The layout part is true of the code: inline_local_offset starts below the caller's locals, and the collector walks only org_local_space. But that code never runs:

The latent hazard is real: re-enabling ProcessInlineMethod would reintroduce exactly this defect. Phase 5 deletes it. Any future JIT inliner must give inlined locals declared, walked slots.

3.3 Integers rewritten by fixup (new; interpreter and JIT)

FixupRoots runs FixupSlot over every operand-stack word and every window word. The comment there says it "validates the forwarding target so a plain integer that merely aliases the young address range is not misread". But the check cannot tell an integer from a reference: an integer equal to a young object's start address passes it.

GetInstanceID() returns exactly such an integer (LOAD_INST_UID compiles to LOAD_INST_MEM). The reproducer:

o := Box->New(i);
before := o->GetInstanceID();
# o->GetInstanceID() is pending (operand stack / spill slot) while Churn allocates
if(Same(o->GetInstanceID(), Churn(i), before) = false) { bad += 1; };

Run with --nursery=64k, 2,000 iterations, no native:

--jit=off--jit=1
-opt s0FAIL, 242 changedFAIL, 242
-opt s3FAIL, 242FAIL, 242

With the default 128 MB nursery it passes, because no minor collection lands in the window. The 2026-10-22 binary fails the same way, so this is not a regression of this cycle.

A related but separate problem: an instance ID is an address, so it changes when the object is promoted. That is an identity-stability bug, out of scope here.

4. The nine patches, traced to the assumption each worked around

commitwhat it fixedthe root-discovery assumption that failed
9cdc656566 (#1429-#1431)the and/or slot was neither marked nor relocated in interpreted and ARM64 JIT frames"declarations describe every slot". The and/or slot has no declaration; its type depends on the pc. The fix marks it conservatively.
613e68a63f / 8617fda009pending values in registers at a loop's safepoint park were lost"nothing is live in registers at a safepoint". The fix borrows the conservative window by spilling there.
0f0c6b8bfe (#746)the frame's INSTANCE_MEM copy of self went stale after a park"self has one home". The JIT keeps a second copy that the collector cannot see.
55504a6fcdARM64 read every declared slot one word off when slot 0 was missing"the collector can infer the frame layout from declarations". The JIT laid the frame out from references.
4b67aa39baan unreferenced declared local shifted every slot below itthe same assumption, in the other direction
4542ec7e41 (#820)self "relocated" through a word that was not a forwarding address"any word can be relocated if it looks forwarded". This is hardening of the guess.
181d478f70 (G5)a conservative nursery word was read as a class pointer and dereferenced"a word that looks allocated is an object". The fix validates the guess.
5d92132687 (#880, #881)a stale window word marked a closure block first, so the typed walk skipped its captures"conservative and typed marking are interchangeable". A stale window word won the race.
193f076fdd(a) zero-field objects at the nursery end; (b) a capture block typed BYTE_ARY_TYPE holding references(a) is an allocator bound, unrelated to roots. (b) is the heap-side twin of this design: a reference container without a reference type.

The framing is confirmed for eight of the nine. Each fix either told the collector about one more slot kind, or made the conservative guess safer. None gave the collector a precise statement of what is live. 193f076fdd(a) is a separate allocator fault.

5. Root model

5.1 Slot kinds today

slot kindinterpreterAMD64 JITARM64 JIThow found today
declared localsframe->mem[1..]jit_mem upward, typedjit_mem upward, typedprecise, by declaration
and/or slot (id 0)frame->mem[1]lowest local; also window word 0jit_mem[-1]conservative (9cdc656566)
selfframe->mem[0]frame->mem[0], plus INSTANCE_MEM copysameprecise for mem[0]; the copy is reloaded after every park or callback
operand stackop_stack[0..pos], per threadvalues handed to callbackssameconservative, and relocated
pending values (working stack)not applicableregisters; spilled to TMP_REG_0..9 / TMP_XMM_0..2 at callbacks, loop parks and libm callsTMP_X0..5 / TMP_D0..3conservative window
scratch saves (div, shift, RCX save)not applicableTMP_REG_0/1not applicableconservative window, never cleared
saved frame wordsnot applicableWin64 window reaches saved RBP and the return address; POSIX reaches OP_STACK/STACK_POSwindow covers the caller's D8-D15 save areaconservative window (harmless today, never meaningful)
func-ref localstwo wordstwo wordstwo wordsprecise (FUNC_PARM) plus closure declarations
closure capture blockold BYTE_ARY_TYPEsamesametyped via GetClosureDeclarations, conservative when dirty (193f076fdd)
native / trap C++ localson the operand stack by conventionbridge into C++sameoperand stack, conservative

5.2 Target rules

A word is a root if and only if the collector is told it is a reference at the safepoint where the collection happens. No word is ever relocated on a guess.

  1. Declared locals. Precise by declaration, as today. Both backends already lay frames out from declarations (4b67aa39ba), and entry zeroing makes a not-yet-written slot hold 0. Liveness is not required: a dead local that holds a reference is retention, not corruption, and the language makes it visible.
  2. The and/or slot. Typed per pc by the shared analysis (§6.1): a reference where the join merges references, an integer otherwise. It is no longer conservative.
  3. Self. frame->mem[0] is the only root. INSTANCE_MEM is a derived cache that is never scanned and is reloaded after every GC-reachable call. That rule exists today; the design states it as an invariant and the verifier checks it.
  4. Operand stacks. Typed per (method, pc) by the shared analysis. A depth is marked and relocated only if it is a reference. Integers are never touched, which fixes §3.3.
  5. Pending values in JIT code. A reference goes to a reference-only spill slot, which is typed, scanned precisely, zeroed at entry and cleared after reload. An integer or float goes to slots the collector never scans. This fixes §3.1 and §3.3 in compiled code.
  6. Scratch saves. These go to non-scanned slots. A scratch value is never a reference the collector needs: no collection can happen between a save and its restore inside one instruction.
  7. Saved frame words and float temps. Never scanned.
  8. Closure captures. Unchanged for roots. The capture block gets a type of its own in phase 6 (the heap side), which retires 193f076fdd's conservative dirty-block path.
  9. Native and trap C++ code. The convention stays: a reference held across an allocation lives on the operand stack. The analysis types an instruction's inputs as of the instruction's start, and a trap that pushes intermediate references must declare them. Phase 2 audits the traps; the verifier runs report-only until that audit is done.

5.3 Conservative scanning afterwards

There is none in frame or stack root discovery. The old scan survives in one place: the audit mode OBJECK_GC_ROOTS=audit, which runs the conservative scan beside the precise one and reports any allocated word the conservative scan would mark that the precise scan did not. That is the migration safety net. During phases 1-4 it reports, and it never changes what is marked.

6. How live references are reported

6.1 One shared analysis: RefMap (in jit_common)

The bytecode does not carry reference types on the stack. LOAD_INT_VAR loads objects and integers alike, and Int[] and object arrays share INT_TYPE. So the first piece is an abstract interpretation over a method's bytecode that tracks, for each operand-stack depth and the and/or slot, one of {int, ref, float, unknown}:

sourceresult
LOAD_INT_VARthe slot's declared kind
NEW_*ref
LOAD_INST_MEMref
MTHD_CALLthe callee's return type
arithmeticint
array element loadsthe array's declared element kind, where the declaration knows it
joinsthe meet of the incoming kinds

The analysis runs once per method on first need and is cached. It has three consumers: the interpreter (read lazily at collection time), AMD64 (at compile time) and ARM64 (at compile time).

unknown is the open question. Phase 1 measures how often it occurs across the regression suite and the library before anything depends on it. Any residue gets a compiler-emitted annotation instead of a guess.

6.2 Interpreter: stack maps per (method, pc)

The map is read only when a collection walks the frame, and indexed by the instruction in progress, StackFrame::cur_ip (see the status note at the top). Recording it is one store on the dispatch loop's default path, which only instructions that can collect take, and one at the safepoint poll.

6.3 JIT: three options compared

A. stack maps per safepointB. clear-after-reload, keep the windowC. typed spill areas (recommended)
mutator costone store per GC-reachable call (the site id into a frame word)one store per spilled valueone store per spilled reference (the clear)
precisionexact, including livenessretention fixed; integers still rewritten (§3.3)exact for spills; locals by declaration
failure mode of a bugwrong map, so a live object is missed: corruptionmissed clear: retentionmissed clear: retention; wrong slot class: caught by roots verify
new machinerysite-id word, per-method tables, lookup in CheckJitRoots/FixupRootsnonesplit TMP_* into TMP_REF_* and TMP_VAL_*; a RegInstr ref bit from RefMap
backendsboth, plus table encodingbothboth; frame layout constants change in both .h files

B is the minimum patch. It is rejected as a root model because it keeps the conservative window and does not fix §3.3.

A is the most general, but a wrong map entry fails toward corruption, and it adds a table format to two hand-written backends.

C gets the same safety from a smaller change, and every bug it can have fails toward retention. Recommendation: C. A can come later if liveness inside long frames ever measures as a problem.

Under C, the collector walks each JIT frame as: the declared slots (typed), the and/or slot (typed by RefMap at the frame's current pc), then the TMP_REF_* slots (typed ref, 0 when empty). Nothing else. The and/or slot needs the pc, and this design has not yet confirmed that a JIT frame records it at every GC-reachable call. Phase 1 checks this. If it does not, phase 3 adds one store of the instruction index per such call, which is option A's cost for this one slot only.

7. Phases

Each phase is one PR against master. The full regression suite, the GC verifier legs and the stress tests (gc_*_stress, jit_gc_stress, gc_mt_small_nursery_stress, the nightly verify-major and --nursery configurations) stay green. Before/after numbers are recorded in each PR (§8).

phasecontentfixesrisk
0Fix --jit=off so it disables native compilation too, or rename its meaning; the differential's s0/off depends on it. Add the measurement baseline (§8).the evidence gap in §2.1low
1RefMap analysis (core/vm/arch/ref_map.h, header-only so no build file changes) plus the OBJECK_GC_ROOTS=audit report-only mode. Report how often unknown occurs per suite. No behaviour change.nothing yet; measures the designlow
2Interpreter operand stacks and the interpreted and/or slot marked and relocated by RefMap; audit traps that hold references across allocations. Lands gc_int_not_relocated (interpreter configurations).§3.3 in interpreted code; retires 9cdc656566's interpreter halfmedium
3AMD64: typed spill areas (TMP_REF_* / TMP_VAL_* / TMP_SCR_*), the kind read from RefMap at the spill site, clear after reload, entry zeroing of the reference slots; CheckJitRoots/FixupRoots walk them by type on x64. Lands jit_stale_spill_slot at s3 and the JIT configurations of gc_address_int_pending. Built (see the status note).§3.1 paths 1 and 2, §3.3 compiled, on AMD64medium-high (frame constants)
4ARM64: the same. Compile-checked locally with cl /Zs and the cross toolchain; proven on CI's arm64 legs. Built (see the status note).the same, on ARM64medium-high
5Remove JIT_TMP_LOOK_BACK and every CheckUntypedSlot/FixupSlot call on frame and stack words. Delete the dead JIT inliner (ProcessInlineMethod, CanInlineMethod, extra_inline_space). Remove the symptom code made redundant: the conservative and/or marking (9cdc656566) and the window spill comment paths in the loop safepoints (613e68a63f/8617fda009, now TMP_REF_*). Demote the conservative-word class validation (181d478f70) and the FixupSelf checks (4542ec7e41) to verifier assertions.the whole classmedium
6Heap side: a reference type for closure capture blocks, retiring 193f076fdd's conservative dirty-block path. Separately, stable instance IDs.193f076fdd(b); ID identitymedium

5d92132687's GC_TRACED_BIT is kept either way. It makes typed tracing of a closure happen exactly once, which stays useful with precise roots.

What v2026.10.2 needs

Recommendation: none of phases 1-6. Ship 10.2 without them.

If one item must go in, take phase 0's --jit=off fix. It is small and VM-only, and it makes the nightly differential's "interpreter" column mean what it says.

The one-line entry-zeroing change would also be safe, but it fixes one path of one symptom, which is the pattern this design replaces. It stays on the evidence branch unless the user decides otherwise.

8. Measurement

Recorded before and after phases 2, 3 and 5. Precise scanning should lower mark work (no IsAllocated set lookups per untyped word); the map read is per frame.

9. Tests

testcatcheslands in
jit_stale_spill_slot (rewritten without native, so --jit=off is a true control; checks both the s0 and s3 paths)§3.1phase 3 (AMD64), extended in 4
gc_int_not_relocated, the §3.3 reproducer, all four configurations at --nursery=64k§3.3phase 2 (interpreter), phase 3 (JIT)
jit_inline_local_rootnothing today (§3.2)not landed; phase 5 deletes the code it would test
gc_root_coverage, randomized (below)the whole classphase 1 runs it report-only; it gates from phase 2

gc_root_coverage generates, from a seed (printed on failure, reproducible with --seed), methods that:

It checks every value afterwards: integers unchanged, objects with their fields intact. It runs under OBJECK_GC_VERIFY=checkmark with --nursery=64k, at s0 and s3, with --jit=off and with OBJECK_JIT_THRESHOLD=1, plus a two-thread variant that parks at loop polls. The fixed seeds run in the regression suite, and the nightly runs fresh seeds. During phase 1 OBJECK_GC_ROOTS=audit must report zero differences across the suite before phase 2 starts.

10. Not established

One link to the harness is real: that differential's s0/off column is not interpreter-only (§2.1), so "passes with the JIT off" has meant less than it appeared. #1443 was not reproduced here.