You actually made me laugh out loud.
Thanks!
3 Likes
3 Likes
Thank you all for this thread, it delivers in such a humorous way that I cannot recall another thread on this board that comes close… Those analogies are priceless…
Pringles are kinda shaped like tennis ball components. I guess that help sells the joke.
And I do like pancakes. (And waffles.) And as long as the pancakes are good, I’ll keep coming back. Others might be happier finding another pizza joint. It is what it is.
I have said several times that our drop rates are staying the same, and we are changing the in game displayed drop rates to reflect that.
We have ALSO fixed the bug in the code for the future. With the drop rates staying as they were.
I hope that this has clarified everything.
4 Likes
Why couldn’t this have been done ages ago when the issue was first reported?
smol independant team asking for full aaa game price for a set of armor can’t afford testing stuff
A free game giving an option to spend $50 on cosmetic and in game bonuses. How dare they!
They should just charge $50 instead and then charge $13 a month to wear the armor like a Triple A publisher would! 
(Who still doesn’t do enough testing despite making millions each quarter.)
2 Likes
that is hilarious on so many levels.
I wonder how we ended up in situation where bugs that would be discovered within 10 minutes of automated testing by unpaid intern persist in game for years. /s
EDIT: added quote
Just because something can be discovered doesn’t mean it will be discovered. Writing bug-free code is hard, and expensive, and does not match well to the quarterly financial report cycle publishers want.
- The Hubble Space Telescope was launched with a mirror that didn’t meet specifications because during grinding one sensor was 1.3 mm out of alignment.
- The Ariane 5 launch in June 1996 failed because a 64-bit floating-point number was recast as an integer.
- The Mars Climate Orbiter is famous as a mission failure because one team of scientists used Imperial units, while another used metric.
- The Hitomi X-ray satellite broke apart shortly after launch because a single sensor reported faulty values, and the satellite’s internal code had a flipped sign necessary for correcting spins.
All of these projects received far more funding than does a single game development workshop, because the stakes were orders of magnitude higher. And they still failed to produce bug-free code, with the causes of failure being incredibly simple to explain—after the failure and weeks/months of investigation.
9 Likes
Well Factions are about a year old now for one. And I believe the drop rates being shown are actually only a few updates old at this point.
Idk how long you’ve been playing. But this current version is the least bugged version of GoW ever played. And every update continues that upward trend. While other much bigger game developers are headed in the opposite direction.
For instance, I couldn’t unlock the last Achievement of Child of Light tonight because it’s bugged. That game is just as old as GoW. 
Knowing the exact nature of the bug, it should be fairly easy to calculate the rates we are getting served now and see if they line up with sampling.
The internal drop tables would look something like this (for, say, Amanithrax in this example):
"Rewards": [{
"Type": "Troop",
"Chance": 0.7,
"RarityChance": [0, 0.59, 0.26, 0.11, 0.04, 0],
"Included": [6756, 6757, 6758, 6759],
}, {
"Type": "Treasure",
"Chance": 0.3,
"RarityChance": [0.25, 0.25, 0.25, 0.14, 0.08, 0.03],
"Included": [6114, 6115, 6116, 6117, 6118, 6119],
}]
When full drop rate data used to be sent to the client, the raw data appeared in a similar format. What essentially happened here is that the server does not make a distinction “troop” and “treasure” as different “types”… they are both “troops”. And the troop drop pools were simply combined. Thus, essentially, we got data that was rolled from tables like this:
"Rewards": [{
"Type": "Troop",
"Chance": 0.7,
"RarityChance": [0, 0.59, 0.26, 0.11, 0.04, 0],
"Included": [6756, 6757, 6758, 6759, 6114, 6115, 6116, 6117, 6118, 6119],
}, {
"Type": "Treasure (Also Troop)",
"Chance": 0.3,
"RarityChance": [0.25, 0.25, 0.25, 0.14, 0.08, 0.03],
"Included": [6756, 6757, 6758, 6759, 6114, 6115, 6116, 6117, 6118, 6119],
}]
Note that some of these now include IDs on their drop tables for which they have zero chance. These simply won’t drop on those tables - if you roll a “troop”, you can’t get a common or mythic because the respective rarity chances are zero. However, since both these are already displayed as chance of the total drop pool, the overall chances of mythic and common treasures should not change at all. Every other rarity should work out to be the average of their overall chances (with each rarity having an equal chance of two drops except common/mythic, which only have one), giving us an overall rarity distribution of:
| C | R | U | E | L | M |
|---|---|---|---|---|---|
| 7.50% | 48.80% | 25.70% | 11.90% | 5.20% | 0.90% |
When the displays are updated, we should see:
| Overall | Common | Rare | Ultra-Rare | Epic | Legendary | Mythic | |
|---|---|---|---|---|---|---|---|
| Troops | 45.80% | - | 24.40% | 12.85% | 5.95% | 2.60% | - |
| Treasures | 54.20% | 7.50% | 24.40% | 12.85% | 5.95% | 2.60% | 0.90% |
As far as at least treasures, this is very close to the midpoint of what was sampled in most instances and well within the confidence intervals for the 95% confidence level estimate over thousands of samples in all instances. Translation: Very likely that these are the “true rates” we have been experiencing up to this point, and this is what we should when the patch hits the server.
So I feel that the “how” was covered pretty well here, but the “why” wasn’t.
I didn’t mean to imply “no mistakes ever happen again”. Mistakes happen, and I understand that. The frustration comes from mistakes being repeated. I’m not talking about this or any specific particular game error. I’m talking about a pattern related to the cycle a game issue, particularly a drop rate issue, goes through on the forums that keeps happening.
Truthfully, this was a completely avoidable issue stemming from a process that was repeated many times - a misunderstanding with how something needed to be configured (or simple error) happened and then the chance to correct it was passed because everything was assumed to be correct without sampling the output. The second part is the big one, because in this instance only hundreds of samples needed to be checked to show that the treasure rate was far, far off what was being reported. Long before you even publicized the numbers, you’d have known it was far, far off what you intended to be configured. Before delves even released on live, even. Because by your own admission the drop rates shown up to this point are what was intended to be configured, it just never worked correctly (the optimization that apparently caused the error is older than delves themselves).
I feel like automated relatively small scale sampling is something that could be done a lot more. You don’t need to do a million-sample-proof every week, but every time a drop table changes, some thousands automatic sampling to verify things are in line with expectation even in the face of unknown variables would help insulate against issues like this in the future. You won’t be guaranteed to catch everything with sampling like this, but it would give another layer of protection/verification to an absolutely critical system. As noted, a sample as small as hundreds and very simple comparisons would have shown the portal output to be too different from what was configured as to not possible to be correct.
These things, in particular, could use work:
- Drops tables aren’t being scrutinized until after an issue is brought up. To this point, even, five years into the game. This has lead to repeated mistakes in the past. Troops missing from certain chests after they are supposed to be there, key types drop rates shifting suddenly, an entire rarity chance category being set to zero for weeks (and mythic rarity at that, and requiring direct support intervention to refund keys chasing it, leaving people out of the loop to just assume they were unlucky)… all these are things that have happened, some of them more than once. All had to be caught by the players and brought up to receive correction. Most of them with the backing of data that is no longer available because it is no longer sent to the client.
- The use of “nothing has changed” or “working as intended” before specifically checking with the people that might have changed something or seeing if it is actually working as intended, but simply relying on no specific knowledge anything that has changed and no strong indication things are not working as intended to make the assertion. While this is usually due to a breakdown in communication, speaking honestly, it makes it really hard to take these responses at face value.
- The perceived unwillingness to look at anything drop-rate or RNG related without a mountain of evidence and widespread discussion. (more on this)
To start off with, perceived discrepancies just kind of get written off as “RNG is RNG” and nothing more heavily than when drop rates is involved. I realize there are so many claims with so little to back up anything that I can see it being irritating that things that have been tested being brought up over and over again. Which is fine, most of the players would step in and back you up there on a lot of them… even more of us when we can prove things are. But then we move into issues where if even if some evidence is presented that might not strictly pass as “proof” that something wrong, but rather an “indication something might not be correct”, it is almost always still summarily written off.
Examples
As an example, see discussion about Adventure Board traitstone rates (here)in the 6 weeks or so following the patch where everyone got the same rewards. While recent samples have brought us more in line with what we were seeing prepatch in terms of traitstones (and thus, no strong indication we currently have a significantly higher traitstone appearance rate than we did in the previous version, which was the spirit of the report), this response was still incredibly concerning because of the “we didn’t change anything” (when things had to have changed to some degree even to keep them relatively the same, because deeds were added) the tone is more “you don’t have the samples to prove it” than “I checked with the team and they said that the traitstone rates before and after should be similar”. It was framed and reported as a bug, as in, something that might be unintended and therefore not known by whoever implemented, evidence was presented that things had changed significantly (not the strongest evidence, granted, but enough that players were having some pretty valid concerns), and it was dismissed as “not a bug” without any checks. Not “we didn’t intentionally change traitstone rates to be higher, they should actually be slightly lower now because we added more stuff” or “we will keep an eye on it as things develop” but “not a bug”.
On top of this you don’t disclose the rates and some the items in question either come so far apart or are grouped in such a way that makes it effectively impossible to even use sampling to submit as evidence that something is wrong. Orbs? Delve Chests? Adventure Boards? Well, since we don’t know what the rates are supposed to be, we can’t possibly be correct when we bring up something might be wrong. Makes it really easy to speak from a position of authority because it can’t be challenged, but it also has the habit of backfiring when the cracks are revealed. So when I see something like this, I’d love to be able to jump in and back you up. To be fair, this thread made an easily refutable and ridiculous assertion that was clearly hyperbole (obviously ascension orbs didn’t completely stop dropping, anyone getting a single one could say they didn’t) and some of the responses I read more in the tone of venting and I was going to step in myself and show the numbers involved and why streaking for even a month under the rates we think Ascension orbs drop at wouldn’t that uncommon compared to other events of similar probability. But when the dev response is “The amount of data players have access to is very small in comparison to us.”, with this issue still ongoing, my immediate thought was “Well, did you check it?”. And then I realize I just can’t in good faith defend orb drops being correct I can’t be sure that somehow adding tokens in explore didn’t mess with the rate of Ascension orbs based on the last selected difficulty (I’m not saying this happened, I’m pulling a ridiculous example of an interaction that I have no way to even begin to test even with months of evidence from multiple people and would be completely unintentional on your end and therefore you would have no way to know if it happened for sure either without actually checking the output side, which was readily apparent was not happening).
Even when the playerbase “wins” because of it, the entire situation ends up being a black mark. Like the whole fiasco where tributes were bugged for two years to compound the statue bonus for every pull. Even this is indicative of the same pattern - discussion happened, it was revealed something might be bugged, there was no way for us to tell 100% whether or not it was bugged and devs were pinged just to be sure, and everything stayed as it was with everyone thinking “must be working as intended or they would have fixed it by now” for two years before this (bug report when the payouts suddenly changed without warning) followed by this (“we didn’t change anything”) and then finally this (oops bugfix). But again, the entire issue would have been avoided by either sampling the output ahead of time and/or giving scrutiny to the issue when it was originally reported.
The one time I attempted to raise a drop rate issue DMs (no, I won’t show it here, but those involved know what I’m talking about) I was met with a strikingly similar pattern of outright denial followed by having to push too hard followed by eventual correction.
I’m not trying to scapegoat any one person here - the original thread where tribute payouts were being discussed and tried to be brought up goes all the way back before Salty or Kafka’s forum accounts even existed. Nor am I just dredging up past mistakes to hold them over you. I’m trying to point out the pattern that keeps repeating over and over and over again with little sign of improvement, and why it needs to change.
I suppose this post in and of itself honestly discussing the mistake that was made is indicative of some willingness to change for the better and I am thankful for that, but the rest of it still needs to change. We shouldn’t go back to a status quo where we deal with drop issues after they arise and reach critical mass, they need to be prevented proactively. We shouldn’t have players wanting to raise an issue needed to do so much work that it often isn’t worth it to pursue. We need you to have better tools so it isn’t a giant pain for you to even test if things are working correctly. This is what “improving your processes” should mean.
Right now this process isn’t just inelegant, it is downright disheartening. What I’m asking here is what can I do to make any evidence backed report about a discrepancy, especially a discrepancy with drop rate or RNG, stand out and not immediately get shot down. Preferably without needing to sample rewards from what represents hundreds or thousands of hours of player-time to gather the resources or weeks to months of real time to overcome time gates necessary to obtain the rewards to conduct the study in the first place. Again, I’m not asking for dozens of dev-hours to be diverted every time someone spent 100 gem keys and didn’t get their mythic. But feeling like we do have a line of communication for this sort of thing is very important because randomness and specifically random drops are central to this game. Sacrosanct, even. Because even a temporarily bugged drop rate can have permanent impact on a player’s experience and drop rates are tied directly to your monetization.
I’m sorry for the rant. I thought about trimming it down, but felt it was best to leave this unabridged version to best convey the utter frustration that is trying to convince someone to check a potential mistake when they think they have more or better information than you but are unwilling to check it because a bunch of other people are also trying to convince them they made a different but similar mistake twenty times a day. It is exhausting for all parties involved. Thats why the process needs to improve - starting with being able to recognize which reports are credible and having the tools to quickly investigate them.
So, onto the last part…
Keeping the current drop rates and fixing the bug is an acceptable outcome, IMO. I’m glad the underlying issue is getting fixed now, because leaving it in very likely could have led to issues when the next drop system was implemented and it was “forgotten” that it never worked properly. As a casual observation, a lot of issues seem to be caused by misunderstanding how a piece of code will use a particular piece of data, and I’m not really sure to solve that as an outside observer looking in… I could say more rigor and documentation, but really I have no idea how you distribute your workload that leads to these misunderstandings in the first place. I would have liked Guild Guardian style drops, but understand there might be upcoming features that make this less necessary as we move forward. If/when those are revealed, we can revisit the issue if need be. That is fine. It is the other stuff I feel can still be done better.
To sum up, I’m hoping for:
- A way for credible reports to be quickly supported or refuted with a reasonable degree of confidence through sampling with dev tools (Thus arming you with evidence to back your “working as intended” or “nothing has changed” without wasting a lot of your time and not making it feel like every drop/RNG issue is ignored and that we are wasting our time)
- Automatic sampling when drop tables change and on any new drop system ahead of time so that we are highly unlikely to even reach the point where a report needs to be made in the first place. (a way for you to catch mistakes before they make it this far)
- Disclosure of remaining undisclosed drop rates (Adventure Boards, Chaos Orbs, Tokens, Gnome Rates, Gnome Drops, Vault Rewards, Ingots from PvP, Daily Ingots, Troops in Event Shops, Delve Chests, Legendary Tasks, full troop pools and not just rarity distributions, and anything else I may have missed) as a good faith gesture that everything is working as intended. (a way for us, at least potentially, even if it is probably way too much work, to catch mistakes if they do make it this far)
The mistake isn’t that a bug happened, it was not correcting the circumstances that allowed it to happen, nor improving the process that needs to happen when attempting to get stuff like this looked at. Please recognize this.
23 Likes
All I want to know is, is this bug the cause of us getting a bajillion rare treasures while getting the rare troop to mythic is always the hardest troop of the 4? If so we are spending way more shards than intended to get all troops mythic.
Amen x1000000
Technically yes. There’s really two things to take into account here though:
1.) The originally intended drop rates would have required less shards to get all faction troops to mythic.
2.) The originally intended drop rates would have required much more shards to get hoard level to 100, hoard quality to 10, let alone any higher.
Lower hoard quality means less shards from runs, means less troops, means it might actually have taken longer to get the faction troops to mythic, despite requiring less shards. Personally, I prefer receiving more treasures than more troops, the latter only have very limited use once you do get them ascended to mythic.
7 Likes
I can explain the developer reluctance to look into complaints about drop rates. I used to work on a MMORPG and after EVERY release people would complain that accuracy was nerfed. Humans are terrible at dealing with true randomness, which is often very streaky. Invariably some player would miss 5 times in a row with 95% chance to hit and think the game was broken. Sure missing 5 times sucks and its not likely so they’d go to the boards and complain, look this happened to me, what are the chances!? Well 1/32,000 sounds unlikely, but when 30,000 players are playing concurrently, its happening to someone every minute.
I betting the devs hear the similar complaints about drop rates, so they won’t take them them seriously without a preponderance of evidence. Thank you Mithran for providing that evidence. I expect the next time there is a bug you’ll need to provide evidence again, but on the bright side…you, and you alone, have now gained a degree of credibility. Use it wisely!
5 Likes
Irongut had 99 attack and missed a devour last weekend. It’s best not to think about the exact odds… But instead remember that the computer is given a chance to miss… percentages don’t matter at that point.
Not if it only applies to the rares, which it seems like it does.
Mine only has 95 or 96 attack and I remember it whiffing 2x in a row in 1 match during that last event 
.
I’m just popping in to say that my new favorite word is now C R U E L M.
Also backing up the notion that if you’d publish the drop rates:
- Yes, you will get an influx of people who are mad that they see a 10% and made 10 pulls and didn’t get 1 thing.
- But players like me will take care of explaining just how silly they are through bad analogies.
- And occasionally players like Mithran will produce a 10-page essay that either:
- Completely verifies the drop rates are correct with a reasonable confidence measure.
- Demonstrates there is a bug, with reasonable confidence measure.
Just like he pointed out, there have been quite a few instances in this game where after months or years of condescending explanations of how an RNG works, there is a sudden “Ooops sorry guys the drop rates were wrong” update on that very thing. So I find it easier to assume that any unpublished rates actually do have a bug, because I have a demonstrably higher chance of being right about the GoW RNG being wrong than I do of getting the last 2 ascension orbs I need to craft Zuul’goth by the end of the year.
4 Likes