Hey folks, Monday here, no further info yet sorry.
There’s been at least a couple of really good posts here, particularly by @Kezef who has done a great job of articulating part of the situation.
@Fourdottwoone and @Cronus were talking about new thresholds implemented for Treasure Maps causing these bans and asked for confirmation.
Treasure Hunt and Treasure Map detections are actually the oldest detection types and thresholds in the game - these have been in since before we worked on the newer cheat detection tool in the past few years. They’ve been in for over 5 years (because I started working here 5 years ago and these detections already existed).
From what I understand of the information, the issue occurred because the threshold specifically wasn’t updated successfully to account for Kris Kringle’s spell under all conditions. Kris gives guaranteed Treasure Maps and you can have more than one on a Team. If you blitz through Treasure Maps in a single battle - no problem - you blitz through every single one of let’s say 100 battles - you got banned. It was the literal definition of a bug, the system didn’t operate the way we all expected it to under a specific set of conditions, exacerbated by Kris.
Kris Kringle didn’t cause the issue - but he sped it up. I think the whole situation is a double edged sword. Kris Kringle sped the issue up meaning it hit enough players in a short amount of time that we all noticed - because it was a huge issue. But regardless, this probably would have become an issue at some point, albeit maybe a less noticeable one - which would have been far more insidious.
I’ve done a bunch of checks to make sure no one was banned before or after this incident for the same reason as the “Kris Kringle” Christmas bug. So far everything I’ve seen looks ok. I haven’t called it on my checks yet because well, I guess I’m a worry wort, I want to be 300% sure everything is ok before I call it done for myself. The devs have also done their checks obviously and they are satisfied it was fixed over Xmas except for the 3 players they found that had queued bans that we missed from 7 minutes outside the fix window. These players have now also been sorted out.
I handle all the cheating ban appeals myself and I haven’t seen anyone come through ban appeals prior to this for the reasons that caused the bug - definitely especially keeping a closer eye out for this moving forward for awhile. But I’m 99.9999999% sure no one has appealed a ban relating to this prior to Christmas time.
I also checked the currently banned list for players who are banned for reasons relating to Treasure Maps and who haven’t writeen ban appeals and haven’t spotted anything unexpected yet - but I’m still going through it to be as sure as possible.
I plan to talk to the programmers again on Wednesday (when I have my next call with them) to ask for more info to assist me with my checks, although again, everything should be fine by this stage, I’m just continuining to dig into it for my own sanity.
Someone also asked why the autobans weren’t turned off as soon as Nimhain replied on the forums here - it was because she noticed it at midnight - 1am on Christmas morning and turning the autoban off required a senior engineer, as soon as a senior engineer was reached they began working on shutting it down.
Basically we planned the holiday weekend in a way to minimise any potential issues popping up - nothing new or spicy was deployed close to Christmas. Unfortunately, despite that, this one got by us and we weren’t expecting Christmas morning to go like that, we had no reason to suspect anything could happen that day, we wrapped Christmas weekend up in bubblewrap and made the thing fool proof- jokes on us, Murphy’s Law and all that.
I should probably also personally apologise although there’s nothing I could have really done about it, I was the CX Agent down for holiday coverage and emergencies that week, but I ended up being out of action for the whole week as I caught covid (I had a good run, almost 3 years without catching it), so it took me a long time to realise things were blowing up in Gems of War and then I only managed to work for 3.5 hours that week due to being laid out in bed with the rona. I’m very grateful Nimhain was so on the ball despite her being on vacation herself and able to get in contact with the senior engineers on call. The team worked really hard, long and odd hours to get this resolved as quickly as they could. We are sorry it happened and that we couldn’t fix it faster than we did. I will be incredibly surprised if this happened again - we’re all on high alert now after having that bonfire lit under us.