When analyzing a Carlsen versus Vachier-Lagrave game, I noticed MadChess 3.0 Beta struggling to find Magnus Carlsen’s crushing 23rd move, d6, in the following position.
2r2rk1/1ppq2b1/1n6/1N1Ppp2/p7/Pn2BP2/1PQ1BP2/3RK2R w K - 0 23
In the engine output displayed by the Hiarcs Chess GUI, I noticed MadChess kept restarting its search of ply 18. It indicated it was searching the first move (of 47 legal moves), second move, third move, etc… then would restart searching the first move again, still on ply 18. It restarted searching the first move of ply 18 numerous times. I suspected this was due to search instability caused by aspiration windows: searching a window centered around the previous ply’s score (ply 17 for this position) causes a fail high, the window is expanded in the direction of the fail high, sometimes (bizarrely) this causes a fail low, the window is expanded in the direction of the fail low, causing a fail high, low, high, etc… until the window is wide enough in both directions to return a stable score.
For this position, with aspiration windows enabled (by default they are), MadChess took four minutes to find Carlsen’s move. Even though MadChess is single-threaded, it has a non-deterministic search because it generates random Zobrist position keys (for the initial key, piece squares, side-to-move, castling rights, and en passant squares). Due to this non-determinism, in some cases restarting MadChess and searching the position again caused MadChess to recommend sub-optimal moves, unable to find Carlsen’s move even after 18 minutes, when I gave up.
I’ve long suspected aspiration windows are worthless, so I disabled them, produced a new build of MadChess, restarted the Hiracs Chess GUI, and searched the position again. MadChess found Carlsen’s move in 45 seconds. Restarting MadChess and searching the position again reliably found Carlsen’s move at ply 18 within 40 – 50 seconds.
Knowing the performance of a chess engine on one particular position is not indicative of how it performs in games against a variety of opponents, I decided to remove the aspiration window code (not merely disable it), then run a gauntlet tournament against ten opponents. I removed the following code.
The tournament confirmed my suspicions. MadChess performed 9 Elo stronger without aspiration windows.
See my Are Aspiration Windows Worthless? post on the TalkChess forum for a discussion of this blog post.
| Feature | Category | Date | Commit1 | WAC2 | Elo Rating3 | Improvement |
|---|---|---|---|---|---|---|
| Remove Aspiration Windows | Search | 2020 Dec 20 | 4b7963b | 290 | 2530 | +9 |
| Time Management | Search | 2020 Dec 19 | d143bb5 | 286 | 2521 | +8 |
| Crash Bug | Search | 2020 Aug 29 | 2d855ec | 288 | 2513 | +0 |
| King Safety | Evaluation | 2020 Aug 16 | 6794c89 | 288 | 2513 | +63 |
| Eval Param Tuning | Evaluation | 2020 Jul 23 | bef88d5 | 283 | 2450 | +30 |
| Late Move Pruning | Search | 2020 Feb 08 | 6f3d17a | 288 | 2420 | +29 |
| Piece Mobility | Evaluation | 2020 Feb 01 | 5c5d4fc | 282 | 2391 | +62 |
| Passed Pawns | Evaluation | 2018 Dec 27 | 103 | 279 | 2329 | +119 |
| Staged Move Generation | Search | 2018 Dec 15 | 93 | 275 | 2210 | +39 |
| History Heuristics | Search | 2018 Dec 03 | 84 | 275 | 2171 | +28 |
| Eval Param Tuning | Evaluation | 2018 Nov 24 | 75 | 272 | 2143 | +47 |
| Sophisticated Search Material and Piece Location |
Baseline | 2018 Nov 08 | 58 | 269 | 2096 | 0 |
- GitHub commit (hash) or Subversion source code revision (integer)
- Win At Chess position test, 3 seconds per position
- Bullet chess, 2 min / game + 1 sec / move
I work as a Software Architect at an insurance company. We're building a modern tech stack in the cloud using C#, Azure, ASP.NET Core, Angular, TypeScript, etc.