Metrics · regenerated nightly

Where krusty stands against kotlinc

Every number comes from automated runs; nothing is picked by hand. A source that could not be read says so instead of showing a number.

Headline metric

Byte-identical classes on real projects

krusty aims to emit .class files byte for byte identical to kotlinc's. An identical class needs no tests to trust: it is exactly what kotlinc would have shipped, which matters most for code with few or no tests. A class counts only when the whole file matches.

Each project is built from a pinned commit with its own Gradle build, once with kotlinc and once with the krusty Gradle plugin. A compilation is one Kotlin/JVM compile task of the reference build, main and test source sets alike. krusty built it when it compiled it itself. When it did not, every class of that compilation counts as different, and the reason is one of: left to kotlinc (the build uses a compiler plugin or argument krusty does not implement yet, so the plugin hands the task to kotlinc), failed (krusty reported an error), or not run (a task it depends on failed first). The run's summary artifact lists every reason (run of 2026-11-09).

Over time

Trend

One point per corpus run that measured all 13 listed projects, each measure on its own scale. Partial runs are in the table below.

Time and memory

Build time and peak RSS

The same two builds of each project, each alone on its own runner after a warm-up build, both offline from the same caches. Each cell shows krusty's build against kotlinc's: −40% means 40% less. Whole builds are compared only where both did the same work: both Gradle builds succeeded and krusty compiled something with no compilation failed or skipped. Elsewhere one build stopped early or ran different tasks, so its numbers are shown but marked not comparable; results recorded before the corpus kept the build outcomes are marked comparability unknown. Whole-build wall time barely moves yet, because krusty builds only 15.1% of the compilations: kotlinc still compiles the rest in the krusty build, and Gradle's own work is the same in both. Compile time of what krusty built compares only the compilations krusty compiled itself. Compiler RSS is the Kotlin daemon's peak in the kotlinc build and krusty's in the other; where krusty built little, the Kotlin daemon still runs in the krusty build, so build RSS is the fairer memory figure.

Kotlin's own test suite

codegen/box per Kotlin version

Every applicable case of JetBrains' codegen/box tests, compiled by krusty and by the matching kotlinc. Conformance is the share of cases whose box() returns "OK". The byte-equality gauge is a progress measure, not a share of identical classes: each class counts its bytes before the first difference from kotlinc's, over the longer of the two files, and a failing case counts zero; 100% means every class is identical. See docs/TEST_HARNESS.md.

krusty itself

Coverage floor of krusty's own tests

The share of krusty's own source its test suite exercises, as the floor the coverage gate enforces on every change (coverage-baseline.json). The measured coverage of a change is at least this.

Not measured yet

Coming next