Laboratory · Gate C · ?!
Filed against the 2026-08-29 net
Published 2026-08-29
The learned evaluator lost 143 Elo
-143.1 ±40.5 Elo.
100 games · 50 000 nodes/move · fixed-N
Result: Black was unconvinced.
Hypothesis
Two evaluations, one search: would the playing 768×2×256 net — trained on 20 million quiet CC0 Lichess cloud evals, depth-12 labels — beat handcrafted PeSTO at the same node budget?
Experiment
Same search implementation. Fixed 50 000 nodes a move. 100 games, colours swapped on the fifty-opening suite. LEARNED is the masthead eval; PeSTO is the Gate C opponent and remains a comparison toggle. This is a fixed-N match, not a terminated SPRT.
Result
fixed-N: −143.1 ±40.5 Elo @ 50000 nodes, 100 games, LLR −2.33 (inconclusive; SPRT unterminated)
What failed
LEARNED scored 1 win, 59 draws, 40 losses. LLR −2.33 against bounds ±2.94 is inconclusive; SPRT unterminated. Gate A at the same cap was 0.0 Elo (17–66–17, every pair 1–1), so the −143 is not a colour or adjudication artefact.
What I learned
Honest Elo: the deltas are rigorous, the anchor is declared. A loss at the spec cap is still a result. The earlier −100 at 1 000 nodes was the v1 128; do not compare the two caps. The 2200 on the glass is a club-strength anchor, not a CCRL listing.
The evaluations meet
LEARNED FALLS −143 ELO AT 50 000 NODES
Two evals, one search. Handcrafted PeSTO against the playing 768×2×256 net (nnue-lichess-cc0-768x2x256-32-1-2026-08-29) trained on 20 million quiet CC0 Lichess cloud evals — depth-12 labels, three epochs, hold-out r vs Stockfish 0.64, r vs PeSTO 0.70. LEARNED is the masthead eval; PeSTO remains a comparison toggle and the Gate C opponent. This column is a fixed-N match at 50 000 nodes a move, 100 games, colours swapped on the fifty-opening suite — not a terminated SPRT. 1 win, 59 draws, 40 losses. LLR −2.33 against bounds ±2.94 is inconclusive; the ±40.5 Elo bar is the 95% interval on that fixed sample. Gate A at the same cap was 0.0 Elo (17–66–17, every pair 1–1), so the −143 is not a colour or adjudication artefact. The earlier −100 at 1 000 nodes was the v1 128; do not compare the two caps. The 2200 on the glass is a club-strength anchor, not a CCRL listing.
fixed-N: −143.1 ±40.5 Elo @ 50000 nodes, 100 games, LLR −2.33 (inconclusive; SPRT unterminated)
nnue-lichess-cc0-768x2x256-32-1-2026-08-29 · openings-v1 · 50000 nodes/move · 100 games · fixed-N
Honest Elo: the deltas are rigorous, the anchor is declared. A loss at the spec cap is still a result.
Elo over commits