MadChess 3.0 Beta 4b7963b (Remove Aspiration Windows)

When analyzing a Carlsen versus Vachier-Lagrave game, I noticed MadChess 3.0 Beta struggling to find Magnus Carlsen’s crushing 23rd move, d6, in the following position.

2r2rk1/1ppq2b1/1n6/1N1Ppp2/p7/Pn2BP2/1PQ1BP2/3RK2R w K - 0 23

In the engine output displayed by the Hiarcs Chess GUI, I noticed MadChess kept restarting its search of ply 18. It indicated it was searching the first move (of 47 legal moves), second move, third move, etc… then would restart searching the first move again, still on ply 18. It restarted searching the first move of ply 18 numerous times. I suspected this was due to search instability caused by aspiration windows: searching a window centered around the previous ply’s score (ply 17 for this position) causes a fail high, the window is expanded in the direction of the fail high, sometimes (bizarrely) this causes a fail low, the window is expanded in the direction of the fail low, causing a fail high, low, high, etc… until the window is wide enough in both directions to return a stable score.

For this position, with aspiration windows enabled (by default they are), MadChess took four minutes to find Carlsen’s move. Even though MadChess is single-threaded, it has a non-deterministic search because it generates random Zobrist position keys (for the initial key, piece squares, side-to-move, castling rights, and en passant squares). Due to this non-determinism, in some cases restarting MadChess and searching the position again caused MadChess to recommend sub-optimal moves, unable to find Carlsen’s move even after 18 minutes, when I gave up.

I’ve long suspected aspiration windows are worthless, so I disabled them, produced a new build of MadChess, restarted the Hiracs Chess GUI, and searched the position again. MadChess found Carlsen’s move in 45 seconds. Restarting MadChess and searching the position again reliably found Carlsen’s move at ply 18 within 40 – 50 seconds.

Knowing the performance of a chess engine on one particular position is not indicative of how it performs in games against a variety of opponents, I decided to remove the aspiration window code (not merely disable it), then run a gauntlet tournament against ten opponents. I removed the following code.

The tournament confirmed my suspicions. MadChess performed 9 ELO stronger without aspiration windows.

See my Are Aspiration Windows Worthless? post on the TalkChess forum for a discussion of this blog post.

 

Feature Category Date Rev1 WAC2 ELO Rating3 Improvement
Remove Aspiration Windows Search 2020 Dec 20 4b7963b 290 2530 +9
Time Management Search 2020 Dec 19 d143bb5 286 2521 +8
Crash Bug Search 2020 Aug 29 2d855ec 288 2513 +0
King Safety Evaluation 2020 Aug 16 6794c89 288 2513 +63
Eval Param Tuning Evaluation 2020 Jul 23 bef88d5 283 2450 +30
Late Move Pruning Search 2020 Feb 08 6f3d17a 288 2420 +29
Piece Mobility Evaluation 2020 Feb 01 5c5d4fc 282 2391 +62
Passed Pawns Evaluation 2018 Dec 27 103 279 2329 +119
Staged Move Generation Search 2018 Dec 15 93 275 2210 +39
History Heuristics Search 2018 Dec 03 84 275 2171 +28
Eval Param Tuning Evaluation 2018 Nov 24 75 272 2143 +47
Sophisticated Search
Material and Piece Location
Baseline 2018 Nov 08 58 269 2096 0
  1. GitHub (hash) or Subversion (integer) source code revision
  2. Win At Chess position test, 3 seconds per position
  3. Bullet chess, 2 min / game + 1 sec / move
Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *