Benchmarks
A first look at the numbers. The full, reproducible benchmark suite will be published here.
Scores are weighted B-cubed F1 (log-TF weighting) on equivalence classes of the vocabulary, always against the official Snowball stemmer for the same language. Corpora: general text and public-domain scripture.
Polish, scripture (Biblia Jakuba Wujka, 1923)
| System | F1 (full vocabulary) | F1 (held-out roots) | Size (gzip JS) |
|---|---|---|---|
| Snowball Polish | 78.62% | 83.55% | ~12.5 KB |
LemmaJet sm-balanced | 90.69% | 93.74% | 11.8 KB |
LemmaJet md-balanced | 96.58% | 98.05% | 27.7 KB |
English
| System | F1 (general, 118,947 words) | Δ vs Snowball | Size (gzip JS) |
|---|---|---|---|
| Snowball English (Porter2) | 89.64% | — | ~5.5 KB |
LemmaJet xxs-precision | 93.01% | +3.37 | 1.5 KB |
LemmaJet md-balanced | 98.42% | +8.78 | 25.4 KB |
German, Spanish, French, Czech
Gain over each language’s own Snowball stemmer at the md-balanced profile:
| Language | General | Scripture |
|---|---|---|
| German | +10.29 pp | +15.63 pp |
| Spanish | +20.09 pp | +14.28 pp |
| French | +15.83 pp | +20.05 pp |
| Czech | +14.90 pp | +23.38 pp |
Speed
A proper microbenchmark is still to come, but early informal checks are encouraging: in Python expect LemmaJet to be over 2× faster than Snowball, and in JavaScript at least as fast (in a quick Node run it was well ahead). Further optimizations are planned, so these numbers should only improve.
Caveats
- Czech general-text figures predate the enlarged Czech corpus and will be refreshed.
- Speed figures above are indicative, not a published benchmark.
- Numbers come from the project’s internal benchmark documents and may be refreshed as models are rebuilt.