Chaos Portal drop rate displays are incorrect (and the math that shows why, and what to do about it)

Looks about right to me. Source: math phd, working as a statistics professor, whatever that’s worth.

Thanks for taking the time to look into this!

3 Likes

Well, the question was asked on stream, and apparently the answer is that they’ll be able to investigate in a couple weeks. Hopefully this feedback gets through in the meantime. Now it looks even more likely we will get a straight nerf (“correction” to the “intended” drop rates). In the meantime, I’ll highlight this, again:

Lets look at this a bit further.

Our 8k shard projections currently gets us to roughly hoard level 159 where the display rate estimate gets us to about about 128, a raw difference of 31 stat points. At 150% hoard stats, thats an additional 16 additional skill points difference (give or take a couple from rounding). So lets say the “average” delver (that is interested in progress, so they get all their troops to mythic) would be about 47 skill points lower for the same effort, and would need about 50% more shards (generally amounting to significantly more than 50% more effort, and also involving spending gems on delve events) to get those same skill points back. Thats a good chunk of the bonus that kingdom 14 (deeds) gives at this specific levels, but more than elite levels and medals combined. For players getting less shards per month (and thus less average xp per month, and less projected hoard level per faction), this stat gap is significantly smaller, but those generally aren’t the players attempting the difficult faction team runs. For players getting more than this, the raw stat gap is slightly bigger, and they cannot put any more effort into delving to correct the stats that are lost.

So while maybe not completely invalidating deeds and medals and elite level for the players at the top of the curve who are complaining about level 500 faction team runs, if the intent is to “chip away” at the whole 500 faction team problem with various systems, a nerf in this direction basically still regresses this issue by at least an update, possibly more.

Since on the dev side it was assumed the rates have always been the displayed rates, I know this can’t have fit into the long term “plan”, but all feedback is based on what they are currently, which is why I’m asking that the “true rates” be assessed before they are changed instead of just “bugfix, it was supposed to be this the whole time”.

Also, I’m glad to hear that this was at least brought up in the office, but it would still be nice to know if something is even planned at getting looked into on the forums, when it is brought up.

3 Likes

So if they adjust the rates to the displayed drop rates without implementing any counter fix to the decrease in treasure value, then what? Based on their response time on issues, players are going to be sitting on this reduction for quite awhile.

It’s interesting that they stated on stream that the rates look correct on the server. A straightforward way to code the droprates from your screenshot would be to call a random number [1,1000] and assign drops as appropriate. That we’re seeing wildly different rates suggests that the code is more convoluted, which there’s certainly precedent for (see: strange explosion rounding).

I wonder if the issue is with how the droprate algorithm interacts with generating batches of 10 troops/treasures. It’s not impossible to imagine code that performs correctly if the devs roll 10,000 drops simultaneously, but has issues (maybe due to truncation) with sets of 10. Hopefully whatever internal investigation is conducted will include an account opening portals as players do.

Anyway, it seems like the best solution (aside from switching to a guild guardian-style system…) would be to leave the drop rates as they are and change the stated rates to match what we’re currently seeing.

On a side-note, it was a bit discouraging to hear that there are no plans to allow conversion of lower rarity treasures. It would solve a huge problem, allowing souls to mitigate the enormous gold cost of using those treasures. If the devs don’t see this point of this, I worry that they don’t understand how useless the lower tier treasures are for the majority of players.

2 Likes

Then they are sending a pretty clear message, I’d say. Hopefully, they actually consider the ramification of fixing a long lasting “bug” and have a long term solution at least planned before they do. This has come back to bite them before. I just know they can’t have a plan at this point because they have already stated they weren’t aware there might be a problem until last week, and they won’t even be investigation into whether or not there is a problem for about two weeks. Guess that means they don’t think it is that important one way or the other?

The lower rarity treasures will be the hardest hit. The fewer of those you use now, the closer your projected “fixed” xp and current xp will be… but you’ll lose some in nearly every category.

Yeah, something I considered might be happening. However, while not done all that recently, there has been extensive sampling of other chest style drop in the game, many of which were done in lots of 50 or 10. There was no indication of this error at that time. Current drop rates for most chest types tend to “feel right” anecdotally, and, if you are on a losing streak, using a binomial probability calculation will generally show you aren’t that far off (I’ve seen some what would be <0.1% cumulative probability occurrences reported dealing with chest drops, but few and far between).

But also on this for every other random chest style drop (I’m not 100% sure on this one since it was never stated specifically, but we have been assuming) drops are calculated by first pulling a random number representing chance of category, then another random number for chance within the category, and so on, until an item is reached. For example, when opening gold keys, to get to an “Orc” card, you need to roll “troop”, then “common” then “(whatever the id of orc is from the list)”). It is possible the same algorithm used for other types of chests drops isn’t the same one as for portals. Meaning portals could use the same methodology but have an error where one of the numbers in the chain isn’t actually a new random number assigned.

I believe there is a simpler explanation here: different server files are actually being used.

I recall a similar issue with a kingdom being left out of tributes a while back: Blighted Lands does not give tribute - #11 by Sirrian

The basic probably was when they updated into using a new set of tables and didn’t clear the old ones out. On cursory inspection, everything looked right, but actually two different tables were being used.

This transition might have occurred when they implemented the drop tables in the first place. They used to send out data to the client that could be intercepted that contained sets of drop rates that were allegedly “outdated and incorrect” (even though the parsed data lined up with both what is being reported and what is being experienced on all normal chest types, and was used to catch multiple times when troops were inadvertently left off drop tables). However, if true, this means at some point at least two different versions of the same droprate table existed. The datamined versions of portal drops were posted here when delves were released (!) Spoiler Alert (!) -- [Any Details Provided are Subject to Change] (Part 1) - #6843 by Lyya, and line up exactly with the current stated drop rates.

Wishful thinking, but it could be that these rates here were the “outdated and incorrect” ones the whole time, and that file is still kicking around and somehow is the one being used to configure the display rates. And nobody noticed because all the other display rates were the same as the actual rates the whole time. Meaning the rates we are experiencing are the correct ones, the display rates are wrong.

I agree this would be the least-damaging and simplest short term fix. I still think long term the way that portals are interacted with (the ‘value’ of a chaos shard dropping sharply once you pull all troops from a faction to mythic) could be done in a way that incentivizes always trying to push forward a bit softcaps, rather than just treating the soft caps as hard caps as I’m sure most people do.

2 Likes

I’m new here so take with a grain of salt. But there’s got to be an economic reason behind all this, the drop rates being broken and no one doing anything about it, even after you (Mithran) embarrass them in front of everyone in the forums and hand-feed them several solutions.

I guess you must love this game to go through all this time and effort for people that created the Diabolist class a few days ago. People who implemented a 40% cleanse token of cedric. I hope your effort does not go to waste, but I’m skeptical.

Drop rates being broken actually happens quite regularly, most players just don’t notice. It’s always treated as a hush job, there isn’t any official acknowledgement unless there is absolutely no way around it, there isn’t any information once it gets fixed, the only viable answer for anyone bothering to inquire amounts to “we didn’t change anything”. The most recent example is [Not a bug] Adventure Board misconfigured for Traitstone tasks, I wouldn’t be surprised if anybody doing a task analysis over the past three weeks would find that distribution rates have changed significantly, despite “everything working correctly”.

I guess there are some legal consequences associated to admitting such mistakes. Like the few times event chests were actually unable to pull the kingdom mythic, as the grand prize in their gasha system. I just wish it wouldn’t make trying get these type of issues fixed such a miserable job, there’s a huge amount of effort involved to report it accurately enough to be considered beyond doubt, and the only official feedback you’ll at best get is that you are just imagining things.

I just wanted to add to that info. I have saved records for my last 8 sessions of opening 400 chaos portals + I went and open 1k chaos portals yesterday.
So, my total treasures from 4200 chaos portals (84k shards):
Common - 326 (7.8%)
Rare - 1036 (24.7%)
Ultra-Rare - 544 (13%)
Epic - 253 (6%)
Legendary - 102 (2.4%)
Mythic - 30 (0.7%)

The only significant difference with your data is the rate of Mythic treasures but I believe it’s a flaw of a method of compiling from different people - people much more inclined to post very lucky/unlucky data (we all can see a lot of that in new mythic threads), and 1 set of 13 Mythic treasures skewed your data too much.

That assumption may be incorrect. As this assumption has to be proved first and foremost, I did not read carefully anything that follows. Since all the drops are calculated server side, there is no clear and fool proof way to test the assumption. However, doing a simple runs test on a sequence of data might be used to prove or disprove the assumption. Since there are quite a few outcomes, I would estimate a sequence of 500 should be sufficient for 95-99% significance.

So, there are 3 possible ramifications of the results:

  1. The numbers are not random or not independent or not random and independent.
  2. There is a coding error server side with the calculations.
  3. Developers have intentionally or non-intentionally input wrong drop rates, so technically speaking, the app can be re-reviewed for compliance with Google play store rules, for example, which require disclosure of drop rate. I presume, the drop rates have to be correct, and not simply some arbitrary random numbers be put there claiming that this is our drop rate.

Since this has been brought up so much recently we are doing a more thorough investigation this week.

5 Likes

@Mithran the screen shot in your original post displaying the official claimed percentages, do you have your own estimate for all of those numbers? Nothing fancy, text is fine :stuck_out_tongue_winking_eye:

Those drops show the treasure pulls. Any chance you also have the troop pulls for those drops and could post them here? I’m curious whether the troops on their own follow the published rarity distribution within the troop pool (413/182/77/28), it might help to home in on the root cause. It feels unlikely that whatever is causing it is isolated to shard chests, token chest drop rates look highly suspicious too.

What are the token chest drop rates “supposed” to be? I have not seen them published anywhere, but I would like to know…

From these two guildmates:
97R/42UR/19E/13L (171 out of 390 total pulls)
123R/56UR/29E/15L (223 out of 500 total pulls, screenshots shown in OP)

Totals 215/98/48/28 (389 out of 890 total)
55.27%/25.19%/12.34%/7.2% (out of just troops)
(legendary rate of total pulls here is about 3.1%, the only one higher than the stated rates, everything else significantly lower)

Versus the displayed ratios of
59%/26%/11%/4% (out of just troops)

So very similar… until we get to legendaries, which would have the lowest degree of confidence. However, this sample here is in line with anecdotal evidence… in general, people finishing legendary ascensions way before the other ones, which is why the mythic +4 cost didn’t change much in my estimates under current rates above.

Avg shard costs to mythic (1 copy) under the stated rates is (roughly):
4400/4500/4200/4200
7140 for mythic legendary on average is far above the others (next one is avg 5200 to get the epic to mythic +4, < 5k avg for the others)
With the current rates, there are multiple reports of already having legendaries at mythic +4(or 6, or 8 as in the above sample set) before pulling the last rare or ultra-rare.

With the caveat that these are estimates - I don’t want people going around quoting these categorically as the “true rates”, I just know they are so far off the stated rates in some instances that the stated rates can’t possibly be correct - and the understanding that closer they are to 50%, the more confidence I have in their relative accuracy, heres all treasures recorded to date, in a simplified table (after adding my sample set and Neritar’s):

||C|R|U|E|L|M|Treasures|Portals|
| — | — | — | — | — | — | — | — | — | — |
|TOTALS|578|1917|1038|453|189|75|4250|7765|
|% of Treasures|13.60%|45.11%|24.42%|10.66%|4.45%|1.76%|100.00%|||
|% of Total|7.44%|24.69%|13.37%|5.83%|2.43%|0.97%|54.73%|||
|Displayed % of Treasures|25.00%|25.00%|25.00%|14.00%|8.00%|3.00%|100.00%|||
|Displayed % of Total|7.50%|7.50%|7.50%|4.20%|2.40%|0.90%|30.00%|||

Bolded row (% of total) are how they would display in game. @UKresistance Other stuff shown for reference.

Recalculating for additional samples, our avg expected hoard xp estimate per portal is now 30.35 (down from 31.85). Our avg hoard xp per portal under stated rates would still be 21.075. At a 400 portal pull (8k shards), this would mean the current hoard level estimates for “now” would be about 3-4 levels lower. Still a projected reduction of 27 hoard skill points before deeds and about 41 after kingdom level 14 for the same amount of shards used if we change from “sampled rates” to “displayed rates”. Still a pretty big gap, so my earlier statements still apply.

Yeah, it very well could be intentional, even though it does seem wonky for a “rarity” distribution to heavily favor the “ultra-rare” item, with the “common” and “rare” being significantly less common. Still waiting on that post they said they’d put on the forums detailing those rates. I promise I wont pick them apart unless they make an assertion that is completely out of whack with current observations, like cedrics token drop rates being stated to be anything lower than 35%, because there are already enough samples floating around to show that would be highly unlikely.

I don’t want to discourage drop rates being shared by the dev team, we need more of this. Well, not more of this this, but more of being able to independently verify stuff, and then hopefully being able to communicate that something is off and get a speedy (hopefully beneficial) resolution to an issue. The climate on all things drop rate seems to be “we don’t want you to know about us because you’ll constantly bother us that it is wrong”, but when it is wrong, with ample evidence to show it is, shouldn’t we be bringing it up?

9 Likes

Any news? The display rates are still the same a week later, a new faction is fast approaching and some of us are wondering if we will be dealing with a new reality for expectation of troops/treasures when opening up the next set of portals. If we are getting nerfed before then (or if we already have been, it won’t be widely noticed until Friday), it is probably better to know about it now.

Edit: So no news?

9 Likes

@Saltypatra @Sirrian
Sorry for the directness, but please provide any updates before the new faction event starts (even a nothing to report is an update or sorts)
Thank you

A nerf should really not even be on the table. It’s hard enough as it is to get enough rare treasures to level delves above 100, a nerf would give players even more reason to not care about high-level delving.

Nobody’s complaining about it being too hard to get faction troops and nobody is steamrolling delves with multiple hoards above level 200. If it ain’t broke, don’t fix it is a very viable response.

Just a quick update for you all.

3 Likes

:vulcan_salute:

Just letting you all know the status on that. :stuck_out_tongue:

1 Like