Gems of Wars RNG is currently broken

[Saltypatra]

I’ve dug up some old tests we did on the AI luck when it comes to RNG, so please feel free to review these if you’re interested. As I said, these tests are a little older, but they do show the lengths we go too, and how we look into our AI and RNG.

@Sirrian

So we actually ran some tests on the AI last week, because we’re hearing a few complaints here… we simulated input from the human player, and ran 816 matches overnight on one of our console test kits. The simulation uses the AI to determine the human player’s move, but that only provides input for which Gems to switch & spells to cast… in ALL other respects the actual Gem Board can’t tell whether that input comes from a Human Player or an AI player.

For the record:
a) This was done with all cheating in the human’s favor disabled
b) We define a “lucky” drop as one of the following events (relative weighting is show in brackets):

• A Skull drops in to form a 4+ Skull match, when that skull was not previously visible on the board (5 pts)
• A Gem drops in to form a 4+ match, when that gem was not previously visible on the board (3 pts)
• A Skull drops in to a space where the opponent then immediately gets a 4/5-of-a-kind Skulls (4 pts)
• A Gem drops in to a space where the opponent then immediately gets a 4/5-of-a-kind (2 pts)
• A Skull Gem drops in to set up a Skull match for the opponent (3 pts)

Here are the results from 816 Games

LUCK SCORES (based on weightings above)

• Average Player Luck Score: 24.3
• Average AI Luck Score: 25.0
• Difference in Luck Scores - the AI was on average 2.9% luckier than the human player

LUCKY GAMES (number of games where player had a higher luck score)

• Games with Equal Luck: 4
• Games where the Human was Luckier: 414
• Games where the AI was Luckier: 398
• Difference in Lucky Games: The Human player was luckier in 4% more games

Now that’s not a conclusive result, sample size could certainly be bigger… and obviously we could change the weightings… but I think we can see a clear trend that both sides are receiving fairly similar “luck scores” in the games we ran.

@Sirrian

So we actually ran some tests on the AI last week, because we’re hearing a few complaints here… we simulated input from the human player, and ran 816 matches overnight on one of our console test kits. The simulation uses the AI to determine the human player’s move, but that only provides input for which Gems to switch & spells to cast… in ALL other respects the actual Gem Board can’t tell whether that input comes from a Human Player or an AI player.

For the record:
a) This was done with all cheating in the human’s favor disabled
b) We define a “lucky” drop as one of the following events (relative weighting is show in brackets):

• A Skull drops in to form a 4+ Skull match, when that skull was not previously visible on the board (5 pts)
• A Gem drops in to form a 4+ match, when that gem was not previously visible on the board (3 pts)
• A Skull drops in to a space where the opponent then immediately gets a 4/5-of-a-kind Skulls (4 pts)
• A Gem drops in to a space where the opponent then immediately gets a 4/5-of-a-kind (2 pts)
• A Skull Gem drops in to set up a Skull match for the opponent (3 pts)

Here are the results from 816 Games

LUCK SCORES (based on weightings above)

• Average Player Luck Score: 24.3
• Average AI Luck Score: 25.0
• Difference in Luck Scores - the AI was on average 2.9% luckier than the human player

LUCKY GAMES (number of games where player had a higher luck score)

• Games with Equal Luck: 4
• Games where the Human was Luckier: 414
• Games where the AI was Luckier: 398
• Difference in Lucky Games: The Human player was luckier in 4% more games

Now that’s not a conclusive result, sample size could certainly be bigger… and obviously we could change the weightings… but I think we can see a clear trend that both sides are receiving fairly similar “luck scores” in the games we ran.

@GoldPhoenix0

Just to confirm that our code is working correctly, we have been performing further tests using the same test conditions as the previous test, but on Normal Difficulty in a mode which is not Ranked PvP. This here would be the difference between playing the rest of the game on normal difficulty, and then playing Ranked PvP, where the Combo Breaker is no longer enabled.

Over the past 90 hours, we have produced and analysed 2870 unique battles using the same scoring system according to @Sirrian’s initial post to produce the following results:

LUCK SCORES (based on weightings in the initial post)

Average Player Luck Score: 40.107 (vs 23.153)
Average AI Luck Score: 19.989 (vs 22.982)
Difference in Luck Scores - the human player was on average over 100% luckier than the AI player

LUCKY GAMES (counting number of games and the associated luck score)

Games with Equal Luck: 1
Games where the Human was Luckier: 2870
Games where the AI was Luckier: 0
Difference in Lucky Games: The Human player was equal or luckier in every game

CASCADES (number of cascades which occurred in a match)

Average Player Cascades: 39.532 (vs 36.534)
Average AI Cascades: 30.655 (vs 36.894)
Difference in Cascades - the Human Player had on average 38.40% more games with more cascades than the AI player

4 OF A KIND MATCHES (number of 4 of a kind matches performed by each player)

Average Player 4 of a Kind Matches: 5.784 (vs 5.744)
Average AI 4 of a Kind Matches: 2.893 (vs 5.514)
Difference in 4 of a Kind Matches - the human player had on average 38.40% more games with more 4 of a Kind Matches than the AI player

5+ OF A KIND MATCHES (number of 5+ of a kind matches performed by each player)

Average Player 5+ of a Kind Matches: 2.252 (vs 2.217)
Average AI 5+ of a Kind Matches: 1.000 (vs 2.136)
Difference in 5+ of a Kind Matches - the human player had on average 35.61% more games with more 5+ of a Kind Matches than the AI player

BOARD MATCHES (number of board moves which each player has taken)

Average Player Board Moves: 24.580 (vs 26.49)
Average AI Board Moves: 24.332 (vs 26.734)
Difference in Board Moves - the Human player had on average 8.48% more games with more board matches than the human player

SPELLS CAST (number of spells cast by each player)

Average Player Spells Cast: 7.262 (vs 6.577)
Average AI Spells Cast: 3.737 (vs 5.627)
Difference in Spells Cast - the human player had on average 44.11% more games with more spells cast than the AI player