MadChess 3.0 Beta 4d22dec (Endgame Eval Scaling)

I improved MadChess 3.0 Beta’s detection of drawn endgames. The IsPawnlessDraw method scores the following positions as drawn. Though it continues to search moves for a swindle (opponent mistake that makes a drawn game winnable).

  • 2N vrs <= 1 Minor
  • Q vrs 2B
  • Q vrs 2N
  • Q vrs Q
  • Q vrs R + Minor
  • R vrs R + <= 1 Minor
  • Q vrs 2R
  • 2R vrs R + Minor
  • 2R vrs 2R

Testing revealed considering R vrs <= 2 Minors a draw increased evaluation error and caused the engine to play weaker. I left that endgame in the IsPawnlessDraw function as commented out code (explaining the regression) to thwart any temptation to add it later.

In addition, I added a DetermineEndgameScale method that scales down the score of difficult to win endgames.

  • Winning side has no pawns and is up by a bishop or less.
    • Winning side has a rook or more.
    • Winning side has less than a rook.
  • Sides have opposite colored bishops and no other pieces.
  • All other endgames are scaled by winningPawnCount.

Also, I added a GetTotalScore method that scales down the score as games approach a draw by 50 moves (100 ply) without a capture or pawn move.

Finally The GetStaticScore method brings together the entire evaluation calculation. See the Evaluation.cs source code file for full details. Here’s the code in simplified form.

These code changes increased the playing strength of MadChess 3.0 Beta by 12 Elo. MadChess has crossed the 2600 Elo threshold, at least at bullet chess (2 min / game + 1 sec / move). To date, I have tested MadChess 3.0 Beta exclusively at bullet time control. I was curious how MadChess would perform given more time per game. Of course I’d give its opponents equal additional time. Wouldn’t this benefit both engines equally? Well, chess engines do not scale equally with time. Would MadChess 3.0 Beta or its opponents benefit more from the additional time? Or would it be a wash? That is, they’d scale equally and MadChess 3.0 Beta would achieve the same blitz rating as bullet rating?

MadChess has crossed the 2600 Elo threshold at bullet time control.

It turns out, similar to previous versions, MadChess scales better per time than its opponents. Its blitz chess rating is 2638 Elo. I have started a tournament with rapid time controls (14 min / game + 7 sec / move), however, I won’t know the results for a month or so. Unlike bullet and blitz, I do not have a database of chess engine games at rapid time control. Therefore I cannot run a gauntlet tournament pitting MadChess 3.0 Beta against ten other engines with established ratings. I must run an all-play-all round robin tournament of 48 engines, including MadChess 3.0 Beta, to establish ratings.

MadChess scales better per time than its opponents. Its blitz chess rating is 2638 Elo.

My priority now is to ensure MadChess 3.0 Beta has feature parity (UCI_LimitStrength, MultiPV, etc) with the last release of MadChess, 2.2. Once that’s complete, I’ll release MadChess 3.0.

 

Feature Category Date Commit1 WAC2 Elo Rating3 Improvement
Endgame Eval Scaling Evaluation 2021 Apr 08 4d22dec 286 2604 +12
Bishop Pair Evaluation 2021 Mar 14 2960ec9 285 2592 +22
Position Cache Optimization Search 2021 Feb 23 42d7702 286 2570 +8
Move Generation Optimization Search 2021 Feb 17 22002dc 287 2562 +12
PVS and Null Move Search 2021 Feb 09 f231dac 285 2550 +20
Remove Aspiration Windows Search 2020 Dec 20 4b7963b 290 2530 +9
Time Management Search 2020 Dec 19 d143bb5 286 2521 +8
Crash Bug Search 2020 Aug 29 2d855ec 288 2513 +0
King Safety Evaluation 2020 Aug 16 6794c89 288 2513 +63
Eval Param Tuning Evaluation 2020 Jul 23 bef88d5 283 2450 +30
Late Move Pruning Search 2020 Feb 08 6f3d17a 288 2420 +29
Piece Mobility Evaluation 2020 Feb 01 5c5d4fc 282 2391 +62
Passed Pawns Evaluation 2018 Dec 27 103 279 2329 +119
Staged Move Generation Search 2018 Dec 15 93 275 2210 +39
History Heuristics Search 2018 Dec 03 84 275 2171 +28
Eval Param Tuning Evaluation 2018 Nov 24 75 272 2143 +47
Sophisticated Search
Material and Piece Location
Baseline 2018 Nov 08 58 269 2096 0
  1. GitHub commit (hash) or Subversion source code revision (integer)
  2. Win At Chess position test, 3 seconds per position
  3. Bullet chess, 2 min / game + 1 sec / move

MadChess 3.0 Beta 2960ec9 (Bishop Pair)

I improved MadChess 3.0 Beta’s evaluation function by adding middlegame and endgame evaluation parameters for bishop pair.

Tuning code indicated the bishop pair parameters immediately reduced evaluation error when examining a database consisting of approximately 54,000 Grandmaster games (both players 2600 Elo or stronger). I ran the Particle Swarm Optimization tuner on all evaluation parameters and it further reduced evaluation error.

This improved evaluation function increased the playing strength of MadChess 3.0 Beta by 22 Elo.

 

Feature Category Date Commit1 WAC2 Elo Rating3 Improvement
Bishop Pair Evaluation 2021 Mar 14 2960ec9 285 2592 +22
Position Cache Optimization Search 2021 Feb 23 42d7702 286 2570 +8
Move Generation Optimization Search 2021 Feb 17 22002dc 287 2562 +12
PVS and Null Move Search 2021 Feb 09 f231dac 285 2550 +20
Remove Aspiration Windows Search 2020 Dec 20 4b7963b 290 2530 +9
Time Management Search 2020 Dec 19 d143bb5 286 2521 +8
Crash Bug Search 2020 Aug 29 2d855ec 288 2513 +0
King Safety Evaluation 2020 Aug 16 6794c89 288 2513 +63
Eval Param Tuning Evaluation 2020 Jul 23 bef88d5 283 2450 +30
Late Move Pruning Search 2020 Feb 08 6f3d17a 288 2420 +29
Piece Mobility Evaluation 2020 Feb 01 5c5d4fc 282 2391 +62
Passed Pawns Evaluation 2018 Dec 27 103 279 2329 +119
Staged Move Generation Search 2018 Dec 15 93 275 2210 +39
History Heuristics Search 2018 Dec 03 84 275 2171 +28
Eval Param Tuning Evaluation 2018 Nov 24 75 272 2143 +47
Sophisticated Search
Material and Piece Location
Baseline 2018 Nov 08 58 269 2096 0
  1. GitHub commit (hash) or Subversion source code revision (integer)
  2. Win At Chess position test, 3 seconds per position
  3. Bullet chess, 2 min / game + 1 sec / move

MadChess 3.0 Beta 42d7702 (Position Cache Optimization)

Quoting from my Pull Request #12:

Converted Cache class’ _positions field from a jagged array to a flat array. The flat array is more memory efficient than a jagged array. Jagged arrays have a .NET object header for each sub-array (for garbage collection tracking of reachable-from-root). This enables more positions to be stored per megabyte of memory. The code in this PR stores 65,536 positions per MB = 8,388,608 positions for a typically sized 128 MB cache.

Added stats to track cache hit %, best move found in cached position %, and count of invalid best moves. Stats are displayed when the debug on command is sent.

Experimented condensing CachedPosition struct from two ulongs to a single ulong. Only 15 bits were available for a partial key. Testing showed the partial key caused too many invalid best moves to be extracted from cached positions. Performed weaker than leaving CachedPosition as a struct with a full key.

Here’s my implementation of the Cache class.

Here’s my implementation of the associated CachedPosition struct and CachedPositionData static class.

This improved cache code increased the playing strength of MadChess 3.0 Beta by 8 Elo.

 

Feature Category Date Commit1 WAC2 Elo Rating3 Improvement
Position Cache Optimization Search 2021 Feb 23 42d7702 286 2570 +8
Move Generation Optimization Search 2021 Feb 17 22002dc 287 2562 +12
PVS and Null Move Search 2021 Feb 09 f231dac 285 2550 +20
Remove Aspiration Windows Search 2020 Dec 20 4b7963b 290 2530 +9
Time Management Search 2020 Dec 19 d143bb5 286 2521 +8
Crash Bug Search 2020 Aug 29 2d855ec 288 2513 +0
King Safety Evaluation 2020 Aug 16 6794c89 288 2513 +63
Eval Param Tuning Evaluation 2020 Jul 23 bef88d5 283 2450 +30
Late Move Pruning Search 2020 Feb 08 6f3d17a 288 2420 +29
Piece Mobility Evaluation 2020 Feb 01 5c5d4fc 282 2391 +62
Passed Pawns Evaluation 2018 Dec 27 103 279 2329 +119
Staged Move Generation Search 2018 Dec 15 93 275 2210 +39
History Heuristics Search 2018 Dec 03 84 275 2171 +28
Eval Param Tuning Evaluation 2018 Nov 24 75 272 2143 +47
Sophisticated Search
Material and Piece Location
Baseline 2018 Nov 08 58 269 2096 0
  1. GitHub commit (hash) or Subversion source code revision (integer)
  2. Win At Chess position test, 3 seconds per position
  3. Bullet chess, 2 min / game + 1 sec / move

MadChess 3.0 Beta 22002dc (Move Generation Optimization)

Rather than repeat myself, I’ll explain my recent code update by copying the text of my Pull Request #10 here:

Improved detection of pieces pinned to own king by sliding attackers. Previous implementation only found potentially pinned pieces (because the pieces were on the same file, rank, or diagonal as the sliding attacker). The new implementation finds all actually pinned pieces.

This speeds up resolution of pseudo-legal moves to legal moves by eliminating unnecessary calls to Board.IsSquareAttacked(kingSquare) in Board.IsMoveLegal method. Its benefit is limited though because many pseudo-legal moves never are examined for legality because a beta cutoff occurs before the move is searched.

Eliminated unnecessary call to Board.PlayNullMove and Board.UndoMove in Board.IsMoveLegal method. Instead of actually making a null move, the code flips side-to-move, sets a few other properties of the position, calls IsSquareAttacked(kingSquare), then restores original property values.

I experimented with eliminating the remaining call to Board.PlayMove and Board.UndoMove in Board.IsMoveLegal method to determine if this speeds up resolution of pseudo-legal moves to legal moves. I did this by intersecting pre-calculated moves from the destination (To) square to the opponent king. And by detecting pieces pinned to the opponent king by own sliding attackers. Moving such pieces in a direction other than the attacking ray creates a discovered check. Unfortunately, this code was slower than simply moving the piece and calling Board.IsSquareAttacked. Board.IsSquareAttacked uses pre-calculated moves (magic bitboards) to determine if own king is attacked after moving the piece, constituting an illegal move. Did not include in this PR.

Also, I experimented combining legality checking with playing a move in a Board.PlayMoveIfLegal method. Unfortunately this causes numerous complications with futility-pruned moves. MadChess never prunes moves that deliver check, moves aren’t known to deliver check until they’re played, so this must be detected after the fact (which complicates undoing the move and restoring board state) or detected prior to playing the move (which already has been proven slower). Ugly and prone to bugs. Did not include in this PR.

Also adjusted how nodes are counted- affecting Node Per Second (NPS) metric- to a more honest measurement. A node is counted only in…

  1. Board.PlayMove method
  2. Board.PlayNullMove method (because this changes side-to-move plus a few other properties and advances position index)

2562 +/- 16 Elo at bullet chess.

 

Feature Category Date Commit1 WAC2 Elo Rating3 Improvement
Move Generation Optimization Search 2021 Feb 17 22002dc 287 2562 +12
PVS and Null Move Search 2021 Feb 09 f231dac 285 2550 +20
Remove Aspiration Windows Search 2020 Dec 20 4b7963b 290 2530 +9
Time Management Search 2020 Dec 19 d143bb5 286 2521 +8
Crash Bug Search 2020 Aug 29 2d855ec 288 2513 +0
King Safety Evaluation 2020 Aug 16 6794c89 288 2513 +63
Eval Param Tuning Evaluation 2020 Jul 23 bef88d5 283 2450 +30
Late Move Pruning Search 2020 Feb 08 6f3d17a 288 2420 +29
Piece Mobility Evaluation 2020 Feb 01 5c5d4fc 282 2391 +62
Passed Pawns Evaluation 2018 Dec 27 103 279 2329 +119
Staged Move Generation Search 2018 Dec 15 93 275 2210 +39
History Heuristics Search 2018 Dec 03 84 275 2171 +28
Eval Param Tuning Evaluation 2018 Nov 24 75 272 2143 +47
Sophisticated Search
Material and Piece Location
Baseline 2018 Nov 08 58 269 2096 0
  1. GitHub commit (hash) or Subversion source code revision (integer)
  2. Win At Chess position test, 3 seconds per position
  3. Bullet chess, 2 min / game + 1 sec / move

MadChess 3.0 Beta f231dac (PVS and Null Move Improvements)

I improved MadChess 3.0 Beta’s Principal Variation Search (PVS) by consolidating two separate re-searches into one. In addition, I changed the null move implementation so it reduces the search horizon more aggressively if the current position’s static score far exceeds beta.

Previously, if a PVS search failed high, the engine would confirm the fail high first by restoring beta from bestScore + 1 to its original value and re-searching. If that also failed high, it restored the search horizon from a reduced value to its original value and re-searched. If that failed high the engine took a beta cutoff.

I simplified the code to a single re-search.

Null move now reduces more aggressively if the current position’s static score far exceeds beta.

This increased the playing strength of MadChess 3.0 Beta by 20 Elo.

 

Feature Category Date Commit1 WAC2 Elo Rating3 Improvement
PVS and Null Move Search 2021 Feb 09 f231dac 285 2550 +20
Remove Aspiration Windows Search 2020 Dec 20 4b7963b 290 2530 +9
Time Management Search 2020 Dec 19 d143bb5 286 2521 +8
Crash Bug Search 2020 Aug 29 2d855ec 288 2513 +0
King Safety Evaluation 2020 Aug 16 6794c89 288 2513 +63
Eval Param Tuning Evaluation 2020 Jul 23 bef88d5 283 2450 +30
Late Move Pruning Search 2020 Feb 08 6f3d17a 288 2420 +29
Piece Mobility Evaluation 2020 Feb 01 5c5d4fc 282 2391 +62
Passed Pawns Evaluation 2018 Dec 27 103 279 2329 +119
Staged Move Generation Search 2018 Dec 15 93 275 2210 +39
History Heuristics Search 2018 Dec 03 84 275 2171 +28
Eval Param Tuning Evaluation 2018 Nov 24 75 272 2143 +47
Sophisticated Search
Material and Piece Location
Baseline 2018 Nov 08 58 269 2096 0
  1. GitHub commit (hash) or Subversion source code revision (integer)
  2. Win At Chess position test, 3 seconds per position
  3. Bullet chess, 2 min / game + 1 sec / move