Overall Architecture
The Karkain compilation pipeline transforms source text into machine code through a series of well-defined stages.
Pipeline diagram
Source (.kark)
│
▼
Lexer (pkg/lexer)
│ tokens
▼
Parser (pkg/parser)
│ AST
▼
Macro Expander
│
▼
Borrow Checker (Phase 51)
│
▼
Type Checker (pkg/sema)
│ typed AST
▼
HIR (pkg/ir/hir)
│
▼
Typed SSA IR (pkg/ir/ssa)
│
▼
Optimizer (Mem2Reg → FoldConst → CSE → DCE → LICM)
│
▼
SSA Verifier
│
├──► C Emitter (pkg/codegen) ──► GCC/Clang ──► Native Binary
│
├──► WASM Backend (pkg/wasm) ──► wasmtime
│
├──► Native Builder (Object → Linker → Executable)
│
├──► GPU Backend (planned: WGSL / SPIR-V / OpenCL)
│
└──► Quantum Backend (planned: OpenQASM 3.0 / QIR)
Two engines
The pipeline is realized by two engines that produce byte-identical output for the supported language subset.
Go front end
The Go front end is the primary engine. It owns the full feature set: SSA optimization, DWARF debug information, SIMD vector types, the concurrency runtime, profiling instrumentation, cross-compilation, and every advanced backend.
Package |
Responsibility |
|---|---|
|
Tokenizer: keywords, literals, block-comment nesting (Go-only) |
|
Recursive-descent parser, AST with 20+ node types, macro expansion |
|
Resolver with scope tracking, borrow checker (Phase 51), const tracking (Phase 112) |
|
C23 emitter with embedded runtime preamble, deterministic
|
|
High-Level IR: typed intermediate representation between AST and SSA |
|
Typed SSA IR, dominance tree, liveness analysis, optimization pipeline, verifier |
Self-hosted kcc
The self-hosted compiler (src/compiler/*.kark) implements the same
pipeline — lex, parse, type check, C code generation — in Karkain
itself. It is assembled into C23 and compiled with GCC. Since Phase 97
kcc is the default engine for karkain check/build/run/
test. The Go front end remains the fallback for the LSP, test runner,
and features not yet ported.
Package layout
Package |
Description |
|---|---|
|
Tokenizer |
|
Recursive-descent parser and AST definitions |
|
Semantic analysis: resolver, borrow checker, type checker |
|
C23 code generation, native builder, linker, DWARF, profiling |
|
High-Level IR (typed nodes, 20 expression kinds, 16 statement kinds, 19 type kinds) |
|
Typed SSA IR, optimization pipeline, dominance tree, liveness |
|
Incremental compilation cache (content-addressed, whole-assembly) |
|
CPU reference backend (correctness oracle) |
|
GPU backend (WGSL emission) |
|
NPU abstraction + 5 vendor adapters (Intel, Qualcomm, Apple, AMD, Arm) |
|
Dependency-free WASM binary v1 emitter with WASI runtime |
|
Runtime compilation gates for freestanding, core, and concurrency layers |
|
CLI commands, engine routing, diagnostics rendering |
|
Package manager: resolver, lockfile, fetch cache |
|
Line-index / excerpt utilities for diagnostics |
|
Module graph and import resolution |
|
Structured diagnostic types |
|
Language Server Protocol implementation |
|
Math IR (Phase 71) |
|
Tensor IR (Phase 72) |
|
JIT compilation infrastructure |
|
Test runner framework (KTF-001) |
|
Cross-compilation target model (arch, OS, canonical triples) |
|
Three-stage self-hosting bootstrap pipeline |
|
Karkain-owned DWARF 4 emitter and self-hosted parser |
Transitional architecture
Several subsystems exist in a transitional state where only the Go front end implements the feature and kcc parity is a documented future work item:
SSA optimization — only the Go front end lowers to SSA and runs the optimizer pipeline; kcc generates C directly from AST.
DWARF debug information — emitted by the Go codegen; kcc does not produce debug sections.
Concurrency runtime — embedded by the Go codegen when
usesConcurrencyis true; kcc supports the surface syntax (lexed, parsed) but the C helper emission is a documented post-107 boundary.SIMD vector types — the Go codegen emits typed lane vectors and runtime helpers; kcc preserves the syntax as inert code.
Cross-compilation (Phase 111) — the Go codegen handles
--target <triple>with cross-gcc toolchain selection; kcc does not implement target switching.Profiling / trace hooks — emitted by the Go codegen; kcc does not instrument generated code.
Native linker — the Go codegen’s
NativeBuilderproduces object files and linked executables; kcc always shells out to GCC for linking.
These boundaries are intentional: the Go engine serves as the reference implementation, and kcc parity is verified incrementally through conformance and parity test gates.
Compiler independence
Phase 120 established the first compiler component written entirely in
Karkain and owned by the self-hosted compiler: the KIR emitter
(src/compiler/kir.kark), gatewayed by kcc kir / karkain kir.
See KIR (Karkain IR) for its contract. The dependency inventory
(docs/inventory/compiler-dependencies.json) classifies every compiler
component as Go-only, Karkain-owned, or dual, and records KEEP/MIGRATE
targets on the self-hosting path. Phase 121 (“Compiler Independence
Foundation”) extends this: one Karkain-owned component from the inventory
becomes measurably more real, more integrated, and more responsible in
the actual kcc pipeline, with an evidence report instead of a plan.