Experiment 2

Summary

Use 16 PSQT buckets and layer stacks for SmallNet.

Description

Setup

Continued from Experiment 1.

Use (pos.count<ALL_PIECES>() - 1) / 2. Edit model/lightning_module.py, model/model.py and training_data_loader.cpp accordingly.

Set max epoch to 400.

Local Testing

• 10,000 fixed games

• Openings: UHO_Lichess_4852_v1

• Time control: 10+0.1, nodestime=69 (simulated 1+0.01)

• Engine options: Threads=1, Hash=16

Epoch 100

Results of Small16Bucket-ep100 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -2.02 +/- 4.74, nElo: -2.90 +/- 6.81
LOS: 20.21 %, DrawRatio: 41.26 %, PairsRatio: 0.98
Games: 10000, Wins: 3149, Losses: 3207, Draws: 3644, Points: 4971.0 (49.71 %)
Ptnml(0-2): [332, 1150, 2063, 1154, 301], WL/DD Ratio: 2.08

Epoch 150

Results of Small16Bucket-ep150 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -3.30 +/- 4.72, nElo: -4.76 +/- 6.81
LOS: 8.52 %, DrawRatio: 43.42 %, PairsRatio: 0.96
Games: 10000, Wins: 3183, Losses: 3278, Draws: 3539, Points: 4952.5 (49.53 %)
Ptnml(0-2): [351, 1089, 2171, 1082, 307], WL/DD Ratio: 2.17

Epoch 200

Results of Small16Bucket-ep200 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -6.85 +/- 4.68, nElo: -9.97 +/- 6.81
LOS: 0.21 %, DrawRatio: 42.94 %, PairsRatio: 0.89
Games: 10000, Wins: 3091, Losses: 3288, Draws: 3621, Points: 4901.5 (49.02 %)
Ptnml(0-2): [328, 1180, 2147, 1051, 294], WL/DD Ratio: 2.09

Epoch 250

Results of Small16Bucket-ep250 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: 0.03 +/- 4.75, nElo: 0.05 +/- 6.81
LOS: 50.57 %, DrawRatio: 40.72 %, PairsRatio: 1.00
Games: 10000, Wins: 3208, Losses: 3207, Draws: 3585, Points: 5000.5 (50.01 %)
Ptnml(0-2): [315, 1170, 2036, 1157, 322], WL/DD Ratio: 2.24

Epoch 300

Results of Small16Bucket-ep300 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -3.02 +/- 4.77, nElo: -4.32 +/- 6.81
LOS: 10.69 %, DrawRatio: 42.06 %, PairsRatio: 0.96
Games: 10000, Wins: 3179, Losses: 3266, Draws: 3555, Points: 4956.5 (49.56 %)
Ptnml(0-2): [347, 1132, 2103, 1097, 321], WL/DD Ratio: 2.17

Epoch 350

Results of Small16Bucket-ep350 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: 0.42 +/- 4.75, nElo: 0.60 +/- 6.81
LOS: 56.84 %, DrawRatio: 40.86 %, PairsRatio: 1.00
Games: 10000, Wins: 3227, Losses: 3215, Draws: 3558, Points: 5006.0 (50.06 %)
Ptnml(0-2): [310, 1169, 2043, 1155, 323], WL/DD Ratio: 2.31

Epoch 400

Results of Small16Bucket-ep400 vs master (10+0.1, 1t, 16MB, UHO_Lichess_4852_v1.epd):
Elo: -0.17 +/- 4.65, nElo: -0.25 +/- 6.81
LOS: 47.08 %, DrawRatio: 42.18 %, PairsRatio: 1.00
Games: 10000, Wins: 3203, Losses: 3208, Draws: 3589, Points: 4997.5 (49.98 %)
Ptnml(0-2): [295, 1152, 2109, 1151, 293], WL/DD Ratio: 2.28

fishtest

Epoch 250 (nn-e1e21c0c34ff.nnue)

STC

Stockfish Testing Frameworktests.stockfishchess.org

LLR: -2.93 (-2.94,2.94) <0.00,2.00>
Total: 83040 W: 21475 L: 21613 D: 39952
Ptnml(0-2): 272, 9903, 21288, 9805, 252

Result: FAILED

Epoch 350 (nn-c98a3dc5ad4b.nnue)

STC

Stockfish Testing Frameworktests.stockfishchess.org

LLR: -2.94 (-2.94,2.94) <0.00,2.00>
Total: 61440 W: 15897 L: 16068 D: 29475
Ptnml(0-2): 190, 7333, 15865, 7122, 210

Result: FAILED