Skip to content

Runtime internals

This page is a map - not a specification - of what happens between gos run and main returning. Each section links to the crate that owns the stage so a new contributor can find the real source.

Stages

source.gos
   │
   ▼
┌──────────────┐  gossamer-lex        tokens + source map
│  Lexing      │
└──────────────┘
   │
   ▼
┌──────────────┐  gossamer-parse      AST (items + uses)
│  Parsing     │  gossamer-ast        + diagnostics
└──────────────┘
   │
   ▼
┌──────────────┐  gossamer-resolve    name resolution, imports
│  Resolution  │                      path → DefId mapping
└──────────────┘
   │
   ▼
┌──────────────┐  gossamer-types      type inference, trait solve,
│  Type check  │                      exhaustiveness
└──────────────┘
   │
   ▼
┌──────────────┐  gossamer-hir        lowered program tree
│  HIR lower   │                      (match-desugars, for → loop)
└──────────────┘
   │
   ▼
┌──────────────┐  gossamer-interp     register-based bytecode VM;
│  Evaluation  │                      the LLVM and Cranelift backends
└──────────────┘                      live in gossamer-mir / -codegen-*.

Evaluator

The register-based bytecode VM in gossamer-interp is the sole gos run / gos test engine. It:

  1. Accepts an HirProgram.
  2. Compiles every top-level function and inherent-impl method to a register-machine FnChunk, registered under both the unqualified (foo) and type-qualified (Type::foo) names.
  3. Registers builtin callables for stdlib functions (env::args, time::sleep, json::parse, …) and variant constructors for every user enum.
  4. Executes the compiled bytecode in a register machine, keeping locals in per-frame register files. Every construct - closures, select, defer, or-patterns, goroutines, custom iterators - is lowered to native bytecode; nothing is interpreted from HIR.

Struct values are Rc<Vec<(Ident, Value)>>. Field assignment runs through a copy-on-write helper that allocates a fresh Rc so alias bindings never observe each other's mutations.

Memory management

Compiled programs manage memory deterministically with a Swift-like model - reference counting plus arenas, and no tracing collector:

  • Reference counting (gossamer-runtime::c_abi::rc). Recursive heap enums and runtime containers carry an intrusive [RcHeader | payload] header. Codegen emits balanced retain/release pairs; a strong count hitting zero releases the value's reference-counted children iteratively and frees the payload. Weak references follow the Swift-ARC model.
  • Cycle collection. The compiled tiers run a Bacon-Rajan trial-deletion pass over suspected cycle roots, both on demand (runtime::collect_cycles()) and automatically under allocation pressure. The bytecode VM backs values with Arc and does not collect cycles, so collect_cycles() is a no-op there: a strong reference cycle leaks under gos run but is reclaimed under gos build.
  • Aggregate reclamation. Structs / tuples / arrays are heap-allocated via gos_rt_aggr_alloc (plain zeroed malloc) and freed by the MIR drop pass via gos_rt_aggr_free at scope exit.
  • Arenas. arena { } blocks bump-allocate and free wholesale at every block exit.

The bytecode VM piggy-backs on Rust's Rc / Arc for object lifetime; its output is the semantic oracle the compiled tiers match, cross-checked by the tier-parity suite and the VM-vs-LLVM-AOT differential.

Scheduler

gossamer-runtime::sched is the work-stealing M:N scheduler. Every Gossamer binary links it through libgossamer_runtime.a, so go expr in compiled code (and the bytecode VM) lands on the same shared pool. The pool size defaults to num_cpus(), overridable via GOSSAMER_MAX_PROCS=N or runtime::set_max_procs(n) from user code.

Deferred JIT promotion reports compile time, emitted native-code bytes, and process RSS through VM diagnostics. Set GOS_JIT_MAX_CODE_BYTES=N to reject a promotion whose retained code would exceed N bytes for that VM; set GOS_JIT_MAX_RSS_MB=N to reject a promotion before compilation when the process is already at its RSS budget. A rejected artifact releases its MIR snapshot and remains on bytecode.

A MultiScheduler owns:

  • one work-stealing deque per worker M (a crossbeam_deque::Worker),
  • a global injector (crossbeam_deque::Injector) for cross-thread pushes and the netpoller's wakeup path,
  • a parked map keyed by Gid for goroutines suspended on I/O, channels, mutexes, sleeps, or the blocking-syscall pool,
  • a watchdog thread that bumps the cooperative preempt phase every 5 ms and signals SIGURG to a worker that's been running more than 100 ms.

Goroutines

go expr(args) is a real stackful coroutine. Construction:

  1. gossamer_runtime::sched_global::spawn(closure) allocates a 1 MiB corosensei::Coroutine stack (override: GOSSAMER_GOROUTINE_STACK=N). The stack is an mmap reservation fronted by a guard page; the OS commits pages on first touch, so resident memory tracks the depth a goroutine actually uses rather than the reservation. See Goroutine stack model.
  2. The coroutine's entry shim publishes its Yielder pointer to a shared slot, sets the worker's TLS yielder, then runs closure.
  3. The scheduler wraps the coroutine in a GoroutineTask whose step() calls coroutine.resume(). Result Yield → Step::Yield; result Return → Step::Done.

When user code blocks (channel recv on empty, mutex contention, time::sleep, net::TcpStream::read returning WouldBlock, filesystem syscall via blocking_pool::run), the helper calls sched_global::park(reason, |parker| { register parker.gid with the wakeup source }) and then gossamer_coro::suspend(). The worker M sees Step::Yield plus a pending-park flag and moves the task into MultiScheduler::parked keyed by gid. The wakeup source (poller readiness, channel send, mutex unlock, blocking-pool worker, ...) calls MultiScheduler::unpark(gid) which pushes the task back onto the injector. Any free worker picks it up and resumes the coroutine - possibly on a different OS thread than the one it suspended on.

A blocked goroutine costs only the few KiB of stack its shallow frame actually touched - not an OS thread, and not the 1 MiB reservation. Thousands of concurrent goroutines stay in the tens of MiB of resident memory; the 1 MiB reservations consume address space (abundant on 64-bit) that never becomes resident until used. The Goroutine stack model note explains why this guard-page / lazy-commit scheme was chosen over Go-style copyable stacks.

The wake-before-park race window (where unpark(gid) arrives before the goroutine has actually been moved into parked) is closed by a pre_unpark set: if unpark(gid) finds the gid not yet parked, it records the gid; the worker about to park the task observes the pre-unpark and immediately re-ejects the task to the injector instead of leaving it parked.

Preemption

Scheduling uses cooperative safepoints with watchdog-requested preemption.

A goroutine yields the worker M at safepoints:

  • every park point - channel send/recv, select, mutex contention, time::sleep, scheduler-aware network reads, and core filesystem operations,
  • function-call / scheduler-step boundaries, where the worker can reclaim the coroutine between step() invocations,
  • amortized loop back-edge polls. LLVM and Cranelift poll every 1,024 taken backedges and suspend the current coroutine when the watchdog phase changes. The bytecode VM yields its OS worker every 1,024 taken backedges.

The watchdog thread (sched::multi::watchdog_loop, 5 ms tick) escalates against a worker that has not reached a safepoint:

  • after ~10 ms it bumps the global preempt phase (preempt::request_yield_all); the next preempt::should_yield poll at any safepoint returns true and the goroutine yields,
  • after ~100 ms it sends a real OS signal to that worker's thread - SIGURG on Unix, a QueueUserAPC on Windows. The signal does not itself context-switch; it flips the yield flag promptly and, more importantly, interrupts a blocking syscall the worker is stuck inside (the kernel returns EINTR).

The compiled hot path is a decrement and branch; the runtime call occurs only when the counter expires, preserving a bounded polling cost. The native scheduler has a one-worker fairness regression that proves a call-free loop hands control to a runnable peer. The VM uses a separate bounded OS-thread pool, so its backedge poll yields an OS worker rather than suspending a stackful coroutine. A VM configured with one goroutine worker can still be monopolized by one nonterminating task; replacing that pool with resumable VM frames remains a tracked limitation.

Not every host operation is scheduler-aware yet. Core filesystem file and path I/O, HTTP client work, channels, timers, and socket readiness are routed or parked. Specialty filesystem bridges, some process pipe operations, compression, terminal calls, and third-party binding code still require effect-ledger audit.

Netpoller

gossamer-runtime::sched::poller::OsPoller wraps mio (epoll on Linux, kqueue on macOS / BSD, IOCP on Windows). One dedicated gos-netpoller OS thread blocks on OsPoller::poll(50 ms) and dispatches each readiness event to the goroutine that registered for it via register_waker(gid, closure). Default closure: just scheduler().unpark(gid). Timers (time::sleep, http::Client::do_request deadlines) ride the same wheel.

HTTP server

gossamer-std::http::server::run and the compiled-tier gos_rt_http_serve both:

  • bind a non-blocking TcpListener,
  • park on the netpoller for accept readiness,
  • spawn each accepted connection as a goroutine via sched_global::spawn,
  • read / write under the netpoller - WouldBlock parks the goroutine; the worker thread immediately picks up another connection.

Graceful shutdown is driven by:

  • GOSSAMER_HTTP_MAX_REQUESTS=N - env var, stop after N requests.
  • gossamer_interp::set_http_max_requests(N) - safe-Rust test hook.
  • config.shutdown: Arc<AtomicBool> - for in-process callers.

Panic recovery

panic!(msg) in user code returns RuntimeError::Panic(msg) from the evaluator. The native HTTP server catches that per-request, logs it, and returns a 500. A panic inside a goroutine body unwinds the coroutine's stack and propagates to the worker M's resume site - the worker exits with the panic, but other goroutines on other workers continue running. A program-wide panic handler can be installed via panic::set_hook from user code.

Where each stage is tested

Stage Test location
Lexing gossamer-lex/tests/
Parsing gossamer-parse/tests/
Resolution gossamer-resolve/tests/smoke.rs
Type check gossamer-types/tests/typeck.rs, tests/exhaustiveness.rs
HIR lower gossamer-hir/tests/lower.rs
Interpreter gossamer-interp/tests/{eval,run_pass,vm,http_end_to_end}.rs
Stdlib gossamer-std/src/* (#[cfg(test)] modules)
Driver gossamer-driver/tests/
CLI gossamer-cli/tests/cli.rs