The Self-Hosted kcc Engine
The self-hosted compiler (kcc) is a Karkain-to-C23 transpiler
written entirely in Karkain. It compiles its own source code through a
bootstrap pipeline and is the default engine for all core CLI commands.
Source files
The compiler lives in src/compiler/ and consists of nine .kark
source files plus two generated C artifacts:
File |
Responsibility |
|---|---|
|
Tokenizer: keywords, literals, operators, identifiers |
|
Recursive-descent parser, AST construction, macro expansion |
|
AST node definitions and accessor helpers |
|
Symbol tables, scope management, name resolution |
|
Two-pass type checker ( |
|
Conservative static type inference ( |
|
C23 code emission (mirrors |
|
Driver: |
|
Standard library builtins for the self-hosted engine |
|
Generated C artifact (output of transpilation) |
|
Generated C runtime (output of transpilation) |
Assembly
When the Go front end compiles kcc, it assembles all .kark
sources into a single C23 file (src/compiler/main.c or
main.c23), which GCC links into the kcc binary. The assembly
step concatenates lexer.kark, parser.kark, ast.kark,
sema.kark, checker.kark, atypes.kark, codegen.kark,
stdlib.kark, and main.kark in dependency order, then appends
main.kark last (it contains main()). The file
loadSourceWithSiblings() in main.kark mirrors this: it reads
all .kark files in the same directory and concatenates them, placing
the target file last.
Pipeline
The self-hosted compiler implements the same four-stage pipeline as the Go front end:
Lex —
tokenize(source)produces a token array.Parse —
parse(tokens, source)produces an AST. Parser errors are reported viaprintParserErrors(state).Type check —
typeCheckProgram(ast)runs the two-pass checker. Errors surface aserror[K1XX]diagnostics. When any are present the file is NOT reported as[ok]and the CLI maps the result to exit code 3.Codegen —
generate(ast, target, fileBaseName)produces C23 source code which is written to a.c23file alongside the source.
CLI commands in main.kark:
check <file.kark>— lex → parse → type-check only.build <file.kark> [--target c23]— full pipeline to C23 output.run <file.kark>— transpile →gcc→ execute → cleanup.test <path> [filter]— discover*_test.kark, per-test driver synthesis, compile+run, structured summary output.lsp— placeholder for the self-hosted LSP.
CLI integration
Package: pkg/cli/kcc_engine.go
The Go CLI routes commands through kcc via three entry points:
KCCCheckCommand— assembles source (manifest dependencies + siblings), runs the Go-side multi-file syntax preflight, NPU target preflight, then invokeskcc checkin a temp sandbox.KCCBuildCommand— same assembly + preflight, thenkcc buildin a temp sandbox. Unless compile-only, links with GCC.KCCRunCommand— same assembly + preflight, thenkcc run(transpile → gcc → execute) in a temp sandbox.
The KCCRunCommand wraps kcc’s run path because the self-hosted
engine shells out to gcc and system(), which require the binary
to be on the host filesystem. The Go CLI manages sandbox creation and
cleanup.
Engine selection
Since Phase 97 kcc is the default engine. EngineFromEnv()
returns EngineKCC for an empty KARKAIN_ENGINE or any value
other than go/Go/GO. The --engine go|kcc flag provides
explicit override.
Staleness detection
kccStale() compares the modification time of kcc.exe against
every src/compiler/*.kark file. When any source is newer, the
binary is rebuilt via the bootstrap pipeline to prevent silent incorrect
output.
Phase history
Phase |
Milestone |
|---|---|
88 |
Self-hosted compiler foundation: lex/parse/AST/sema/codegen in Karkain, 7 representative test programs, build script |
95 |
kcc is the primary engine for lex + parse + C codegen;
|
96 |
kcc owns the test runner: |
97 |
kcc is the default engine ( |
99 |
Type checker certified: |
107 |
Bootstrap identity verified (stage2 == stage3 bitwise identical,
SHA |
Known limitations
Stage-2 OOM on ~4 GB hosts — The full compiler source tree (approximately 214 KB of
.karksource) may exhaust memory during Stage 2 compilation. This is a documented environmental class, not a defect. The low-memorycheckpath still passes.String-array print differs — kcc prints string arrays as
[x]while the Go engine prints["x"]. This is an accepted divergence in the test output formatting, not a semantic difference.PowerShell BOM issue — PowerShell 5.1
Set-Content -Encoding UTF8writes a BOM that breaks mid-assembly module loads. Standard library.karkfiles must be written as UTF-8 without BOM.SSA optimization — kcc generates C directly from AST without SSA lowering. Optimized builds use the Go front end.
DWARF / profiling / concurrency C helpers — not emitted by kcc; these are Go-only features.