Chaos Portal drop rate displays are incorrect (and the math that shows why, and what to do about it)

The Bug

Before you change this to [Not a Bug] without reading because it is about drop rates, this is math backed and the rates displayed aren’t even close to what is being sampled. So bear with me. And please actually read some of it. At the very least, if you are going to skim the math, read the proposal at the end/in the linked thread: [Poll] How do you feel about the current Treasure drop rate?.

Platform, device version and operating system:
All of them.

Screenshot or image:
image

What you were expecting to happen, and what actually happened:
Expected portal drop rates to display something reasonably close to what they are dropping.

How often does this happen? When did it begin happening?
Always. Well, the drop rates were always what they are now, for at least as far back as I could find data. The rate displays have at least been wrong since this screen has existed.

Steps to make it happen again
Go to open portal on any kingdom and click the question mark. Open any large number of portals to get a good sample size (500 is more than enough, even 100 shows things are off). Observe that the stated rates and what you got aren’t even close.

Now that that out of the way, lets look at how I know that the rates being displayed cannot possibly be the rates being used to push actual drops to players.

The Math

I have to start off by assuming each drop is independent and random. If they aren’t, things are already more screwed up than I thought, because the drop rate display alone doesn’t really mean anything if the rates aren’t independent and random.

For now, we will be focusing on just treasure drops. This is enough to show that the stated drop rate display as a whole is not in line with what is dropping.

We will start with some community gathered samples, mostly taken from here:

I compiled a table with all drops (including two sets from guild members). Results here:

Table
Submitter C R U E L M Total Treasures Shards Spent Portals Opened
TheIdleOne (Thieves) 28 84 56 19 15 1 203 7600 380
TheIdleOne (Amanithrax) 27 81 52 32 7 3 202 7400 370
TheIdleOne (Silver Nec.) 32 106 52 30 10 5 235 8200 410
TheIdleOne (Dark Pits) 31 90 47 26 7 3 204 7600 380
Fleg (Dark Pits) 25 98 60 22 11 4 220 7700 385
Fleg (Wild Court) 42 161 90 32 19 7 351 13,000 650
Dibbs (Wild Court) 23 105 63 11 4 13 219 7800 390
CrzBoarder (Wild Court) 37 133 65 25 9 8 277 10,000 500
TOTALS 245 858 485 197 82 44 1911 69300 3465
% of Treasures 12.82% 44.90% 25.38% 10.31% 4.29% 2.30% 100.00%
% of Total 7.07% 24.76% 14.00% 5.69% 2.37% 1.27% 55.15%
Stated % of Total 7.50% 7.50% 7.50% 4.20% 2.40% 0.90% 30.00%
Confidence Interval (95% Confidence) 6.22%~7.92% 23.32%~26.20% 12.84%~15.15% 4.91%~6.46% 1.86%~2.87% 0.90%~1.64% 53.50%~56.81%
Confidence Interval (99% Confidence) 5.95%~8.19% 22.87%~26.65% 12.48%~15.52% 4.67%~6.70% 1.70%~3.03% 0.78%~1.76% 52.98%~57.33%
Confidence Interval (99.9999% Confidence) 4.94%~9.20% 21.17%~28.35% 11.11%~16.88% 3.76%~7.61% 1.10%~3.63% 0.34%~2.20% 51.02%~59.28%

A bit on the calculations. Each subset is considered to be a binomial normal distribution (all random, all independent, chance of given drop versus total chance of not). Confidence intervals are calculated using a normal approximation interval (Wald interval). Note that while this may skew results with low samples, our smallest number of “successes” in this is 44 on the mythic treasures column out of 3465 total trials. This can be estimated in other ways, but that is getting away from the main issue - we don’t need to know that, for example, mythic treasures rates with a high degree of confidence to know that the overall display table is wrong, we just need to show that some of the things is wrong, and we can actually show more than one.

Our biggest indicator from this data set is total amount of treasures. Stated in-game treasure rate is overall 30%, sampled rate is over 55%. Our 99.9999% confidence level estimate for the samples we have here has the “true rate” being 30% far outside of possibility. Similarly, the cumulative probability to have 1911 successes out of a sample size of 3965, which is what this set shows, is infinitesimally small. Can’t even find a good way to measure it due to the scale of the numbers involved. But if you break this into subsets based on submitter, each of these sets are also so far off the scale as to be basically impossible to describe as “luck”. For example, our first subset shows 203 successes out of 380 trials (total treasures out of portal pulls), and with the stated individual probability of 0.3, the cumulative probability to pull any amount of treasures greater than or equal to 203 is 1.80E-21. As in, the sum of the binomial probabilities to pull 203, 204, 205, 206, etc combined is 1 in 555,555,555,555,555,555,555 (1 in 555 quintillion). This test can be repeated for every sub-set to show that if 0.3 were the correct rate, every recorded individual sub-set would be describing an outlier of similar scale, give or take a few orders of magnitude (and considering “a few order of magnitude” wouldn’t affect our conclusion alone should be a tip off).

Probably the most noticeable indicator, the one that nearly anyone playing doing a lot of portal pulls the game could tell you anecdotally, is the relative drop rates of coin purses versus gold rings versus priests chalices. The in-game drop rate display has all of them at an equal 7.5% drop rate. Our table here has them at consistently more rings than chalices and more chalices than purses, at almost a 4 to 2 to 1 ratio. Also neither Rings nor Chalices overall rates their respective 99.9999% confidence level estimate even close to their stated percentages independently.

From just our treasure subset, we can take also take any number of sub sets and see that things just don’t line up. For example, we can check only the drops of rings and chalices, which are supposed to be equal, so our expected success rate is 0.5 and our total set size is 1343. Doesn’t matter which one we think of as a “success”, 485/1343 and 858/1383 are such outliers for both success and failure for a 50% assumed success rate as to be normally described as “statistically impossible”. If you compare the ratio of gold rings or chalices to any of the drops on the table you see that they are far off even the stated ratios within the treasure set. Nearly everything else is “off” generally in a predictable manner, but these two are the most obvious and measurable.

When compiling this, I considered the following possibilities:

  • The real drops are actually not configured from the same data that does the drop rate display, despite constant claims to the contrary, and thus the display is incorrect.
  • The real drop calculations and displayed rates are configured from the same data, but there is an error in how the drops are calculated by the server. Unlikely since none of the other chests seem to be off with relation to their midline drops (but maybe they are? I have no proof one way or the other for these)
  • The real drops cannot be normally distributed for whatever reason (eg., having prior results skew future ones) If this is the case, then the drop rate displays are useless anyways because the conditions to get said rates are not described. And yes, I know, we are dealing with pseudo RNG, which might not be perfect, but this isn’t even ballpark over a massive sample size
  • Something influences the real drops in a way that is not represented by the displayed drop rate table (which would make the sample set not describable by a binomial normal distribution but would also make the displayed drop rate table, again, incorrect)
  • Sampling error. It is possible these samples contain errors, but I have one screenshot backed set of 500 that alone would describe a statistical impossibility (see below).
  • Sampling Bias. Most samples were taken from the forums, but treasure rates aren’t supposed to be based on anything related to the individual player, except with relation to how many chests they opened (how many samples in a given sub-set) - everybody gets that same rate display, everybody should be pulling things at the same rate. If sampling bias were even possible this would imply different odds for different people or different sets of circumstances, then, again, the display is wrong because the display never changes.
  • Mathematical error. Feel free to check all of my math. I’m fairly confident it is correct barring any initial assumptions being incorrect, which I already covered here (which, again, would make the in-game droprate display pretty much useless). You can use less stringent tests and smaller samples to show that it is highly highly unlikely the sample can be used to describe the stated drop rates

Heres one last one. Even if we assume every other set had some kind of sampling error, this one has screenshots to back it.

These are from a guildmate of mine and are before and after screenshots of treasures, and the screenshot for Wild Court troops, showing a 10,000 shard pull (500 total portals).

Screenshots and Table

Treasures Before

Treasures After

Troops After (none before)

Submitter C R U E L M Total Treasures
(Wild Court) 37 133 65 25 9 8 277
TOTALS 37 133 65 25 9 8 277
% of Treasures 13.36% 48.01% 23.47% 9.03% 3.25% 2.89% 100.00%
% of Total 7.40% 26.60% 13.00% 5.00% 1.80% 1.60% 55.40%
Stated % of Total 7.50% 7.50% 7.50% 4.20% 2.40% 0.90% 30.00%
Confidence Interval (95% Confidence) 5.11%~9.69% 22.73%~30.47% 10.05%~15.95% 3.09%~6.91% 0.63%~2.97% 0.50%~2.70% 51.04%~59.76%
Confidence Interval (99% Confidence) 4.38%~10.42% 21.51%~31.69% 9.13%~16.87% 2.49%~7.51% 0.27%~3.33% 0.15%~3.05% 49.67%~61.13%
Confidence Interval (99.9999% Confidence) 1.67%~13.13% 16.93%~36.27% 5.64%~20.36% 0.23%~9.77% -1.11%~4.71% -1.15%~4.35% 44.53%~66.27%

Even with just this sample set, with a single calculation, we can see just how far off things are. A total of 277 treasures were pulled out of 500 draws. With a supposed 30% drop rate, if we look at the cumulative probability of getting 277 or more drops our total chances are somewhere in the order of 4.00E-32 (the best resolution I could get for the value, not that it matters much with that order of magnitude). Represented in decimal notation and as a percent, thats:

0.000000000000000000000000000004%

Or a 1 in 25 nonillion(1/2.5e31) chance. Hard to even put that into context. There are estimated to be about 1e22 stars in the known universe. This number is 25 billion times that. So… one in that many. That would be the chance of getting at least that many treasures with 500 pulls assuming the 30% rate is correct.

Note that this subset also follows the predictable pattern of drop imbalance on the first three rarities of treasures and can show even these individual ratios are off with a fair degree of confidence.

Just in case it was fixed in the last month or so (like last patch), I did a 100 sample draw (about 2 days ago now by the time this post goes up) and still got these results:

Screenshots

Still 48 treasures out of 100.

The smaller set skews some of the smaller numbers involved, but for total treasures alone this outcome is still several order of magnitude more likely using the sampled estimate droprate of 55% than the stated rate of 30%.

There is absolutely no way to reconcile the display rates coinciding with what is actually being pulled.

Of course, without any complex computations at all, simply granting a dev account, say, 20k shards and opening portals will highlight that the current drop rates and the displayed drop rates describing the same thing is pretty much impossible by looking at them, much in the same way could determine that a coin that hits 70 heads out of 100 is probably not a “fair coin”. In significantly less time. Less, even, if you had a tool to do this automatically. Especially when the trial can be repeated, and larger samples don’t move you any closer to your expected result.

TL;DR (for this section) Displayed portal rates are wrong. And theres the math.

About Treasure Drop rates

The rate of treasures in portals is a big deal, as the entire underworld and several systems extended out of underworld are balanced around it. Please don’t just knee jerk “fix” this by hastily changing whatever table you have on the back end to whatever one is used for display and call it a day because that was the one that was “initially intended”. This system has been live for over a year now at these rates, and feedback has been amassing based on these rates. Changing the current in-game displayed rates to the ones displayed would be tantamount to a massive nerf for those that consistently attempt to push delve progress, specifically, faction teams. It has already been (repeatedly) expressed that certain factions with the faction team were too difficult, and while I don’t believe pushing stats much higher in tiny increments would help them all that much overall, taking away the ability to get these stats by taking away treasures certainly wouldn’t help either. The average xp per portal pull is estimated currently about 31.845, while the stated rates would yield about 21.075 xp per portal pull on average - about a one third reduction. An overall reduction on treasures to this degree is an overall regress on the ability to obtain stats to basically before deeds or medals were added, invalidating any sort of impact those systems have with the intent of making level 500 faction team runs less painful.

Overall sentiment also seems to be split on the issue of getting more troops if it means less treasures will drop. Some would prefer more troops to drop because they don’t push delve progress and just want the troops, and an adjustment to the displayed rates would facilitate that and lower the average shard cost threshold to mythic (1 copy of each) a set of troops by an average of about 20%. Some would prefer the amount of treasures we have now, and some would prefer more treasures.

Proposed win-win-win: Guardian Style faction troop drops.

The numbers and proposal on this are outlined here:

The basic gist is that we can reach a similar level of progress speed for most subsets of players with the displayed rates being the actual rates. We can do this by implementing the Guild Guardian-style faction troop drops - basically, any faction where you have every troop at mythic and four copies, no more troops will drop from portals there. Any potential troop drop will just become a treasure drop instead.

Since the displayed troop rarity ratios are also a bit more punishing that what is currently going on in game (we currently generally mythic ascend our high rarity troops before our low rarity ones, this happen significantly less frequently with a 70% troop drop rate under the stated rates), implementing the stated rates with guardian style drops does have its downsides for players. Any reduction in treasure rates during the beginning of the scale delves harder to “break into” (get your first hoard quality 10 for decent rewards, with each hoard quality milestone meaning more shards per day and each one taking a bit longer to get than now). However, I believe this outcome satisfies the most player subsets, and also the dev mindset of keeping progress to within a specific intended “schedule”. Also, since the average amount of shards needed to get the mythic + 4 threshold (estimated at ~7200 under the best conditions) is still greater than the average amount of shards you’d get from even a full month of daily delves (estimated at 6k~6600 even under best conditions), people on the threshold would be greater incentive to play the delve events if they want treasures. Unless treasure distribution ratio is rescaled for when troops leave chests, this does mean times bump to 3 1/3 times as many treasures after all troops are mythic +4 and you keep pulling, but keep in mind that we suffer a 30% initial reduction in overall treasure xp before mythic, so the total difference from what we are seeing now means we wouldn’t even catch up in total hoard xp until about 8k shards are spent, and our difference in xp at 12k shards spent is roughly hoard level 211 after versus hoard level 190 before. Beyond even this many shards, keep in mind that since hoard levels are not a linear function of hoard xp, you need more xp (and gold) every level, and the amount of chaos shards obtainable per month has a very clearly defined softcaps both before premium currency is converted, and we need much more effort and currency to get more shards, and therefore treasures, which then runs into another softcap converting xp to hoard levels, this ends up being a lot more effort for a little more hoard level progress. Which seems right in line with the overall progress paradigm at endgame.

Since we have similar (only slightly bigger, since current expectations are to mythic the troops in rarity order instead of reverse rarity order like the drop rate display implies) estimated mythic +4 thresholds dealing with the current actual drop rates, Guild Guardian style drops could also be a benefit if the drop rate display gets changed to what the rates are now rather than vice versa. With the treasure rates being so far off stated, assuming they wouldn’t be rescaled for the “treasures only” table, this is significantly less of a bump after all troops are gone, about 1.8 times treasures dropped. We’d get a very similar bump from hoard level 195 to 210 at about 12k shards spent, and fewer than the “guardians style troop drops, 70% initial troop rate” model afterward. I personally think “guardians style drops, 45% initial troop rate” keeps the progress curves a bit smoother for all players, but its also a straight buff, which is less likely to cover the dev “progress schedule” mindset. I will point out though, again, chaos shard gains are fairly clamped to a certain range that requires premium currency to break through, and the level of gains we are talking here is less of a buff as it is making the system feel less “wasteful” - you still need massive amounts of premium currency and effort for any given stat point granted by hoard level at this point.

Closing

I didn’t want to have to force the issue, but we seem to hit a pretty hard wall whenever anybody try to bring up in-game rates being wrong. About anything. And this is a very egregious case. It should be a big deal that these rates aren’t right, and in this case doesn’t even pass an “eyeball test” that everything is as it should be.

Please, take things like this seriously when they reported. I know you can’t chase down every “drop rate is wrong” claim, but there was lots of data floating around already that suggested that things weren’t correct for anybody paying attention. Like anybody pulling 100 portals or more. Its been obvious things are off for a while and have been gathering evidence, but I didn’t want to make a bug report like this until I was absolutely certain by just how much and had an entire free day to go over the calculations back and forward, because it seems like I get stonewalled if I try to mention things like this with anything less. Being told, again, that the displayed rates were configured from the same data used to push actual drops (without any actual checks being done, because this one doesn’t even pass muster with the most basic of tests) in the face of mounting evidence told me I had to force the issue, lest it come back to bite us in another year with “whoops, it was a bug the whole time! heres 50 gems, we cool right?”. I also wanted enough to make a counter proposal to just “nerf everyone” that I expect as the outcome after this does get taken seriously (or get stumbled upon by a coder and “fixed” summarily without any thought on the impact of the “fix” on the game economy as a whole). I hope by taking the time to examine the numbers involved and feedback already given that some thought will be put into which direction to take on correcting the drop rate display. We saw the ripple impact a “fix of a long term bug to originally intended amounts” brought with PvP points scaling, don’t make the same mistake again.

Please, please, please, for everyone’s sake, have some tools where you can quickly sample drops from the live drop pool. This is the umpteenth time drops have been wrong, and its really bad when there is now an in-game official source that reports things differently.

TL/DR - Actionable feedback items:

  • Make an official news post about how this issue will be fixed, how these were wrong in the first place, and why this will never happen again
  • Then fix portal drop rate displays
  • Whether or not this fix involves changing treasure rates from what they are now, implement faction wide “guardian style” troop drops for each faction (once you have all troops from there mythic +4, you won’t see anymore from that faction)
  • If the problem is potentially applicable to any other system in the game, check those too
  • Implement a tool to test live server rates through sampling so you can quickly check when there is a credible report that rates are off.
  • Please let us know how we, as players, are supposed to report things like this and not have them be ignored. Preferably with significantly less effort than I had to put in here.

Sorry in advance everyone. It had to be done. Hopefully some good will come out of it.

23 Likes

Tossing a ping at @Saltypatra to make her aware of this, if she isn’t already.

Surely far too late for an answer before the dev Q&A, but hopefully early enough that an “we’re looking into it” can be said on the the stream.

Looks about right to me. Source: math phd, working as a statistics professor, whatever that’s worth.

Thanks for taking the time to look into this!

3 Likes

Well, the question was asked on stream, and apparently the answer is that they’ll be able to investigate in a couple weeks. Hopefully this feedback gets through in the meantime. Now it looks even more likely we will get a straight nerf (“correction” to the “intended” drop rates). In the meantime, I’ll highlight this, again:

Lets look at this a bit further.

Our 8k shard projections currently gets us to roughly hoard level 159 where the display rate estimate gets us to about about 128, a raw difference of 31 stat points. At 150% hoard stats, thats an additional 16 additional skill points difference (give or take a couple from rounding). So lets say the “average” delver (that is interested in progress, so they get all their troops to mythic) would be about 47 skill points lower for the same effort, and would need about 50% more shards (generally amounting to significantly more than 50% more effort, and also involving spending gems on delve events) to get those same skill points back. Thats a good chunk of the bonus that kingdom 14 (deeds) gives at this specific levels, but more than elite levels and medals combined. For players getting less shards per month (and thus less average xp per month, and less projected hoard level per faction), this stat gap is significantly smaller, but those generally aren’t the players attempting the difficult faction team runs. For players getting more than this, the raw stat gap is slightly bigger, and they cannot put any more effort into delving to correct the stats that are lost.

So while maybe not completely invalidating deeds and medals and elite level for the players at the top of the curve who are complaining about level 500 faction team runs, if the intent is to “chip away” at the whole 500 faction team problem with various systems, a nerf in this direction basically still regresses this issue by at least an update, possibly more.

Since on the dev side it was assumed the rates have always been the displayed rates, I know this can’t have fit into the long term “plan”, but all feedback is based on what they are currently, which is why I’m asking that the “true rates” be assessed before they are changed instead of just “bugfix, it was supposed to be this the whole time”.

Also, I’m glad to hear that this was at least brought up in the office, but it would still be nice to know if something is even planned at getting looked into on the forums, when it is brought up.

3 Likes

So if they adjust the rates to the displayed drop rates without implementing any counter fix to the decrease in treasure value, then what? Based on their response time on issues, players are going to be sitting on this reduction for quite awhile.

It’s interesting that they stated on stream that the rates look correct on the server. A straightforward way to code the droprates from your screenshot would be to call a random number [1,1000] and assign drops as appropriate. That we’re seeing wildly different rates suggests that the code is more convoluted, which there’s certainly precedent for (see: strange explosion rounding).

I wonder if the issue is with how the droprate algorithm interacts with generating batches of 10 troops/treasures. It’s not impossible to imagine code that performs correctly if the devs roll 10,000 drops simultaneously, but has issues (maybe due to truncation) with sets of 10. Hopefully whatever internal investigation is conducted will include an account opening portals as players do.

Anyway, it seems like the best solution (aside from switching to a guild guardian-style system…) would be to leave the drop rates as they are and change the stated rates to match what we’re currently seeing.

On a side-note, it was a bit discouraging to hear that there are no plans to allow conversion of lower rarity treasures. It would solve a huge problem, allowing souls to mitigate the enormous gold cost of using those treasures. If the devs don’t see this point of this, I worry that they don’t understand how useless the lower tier treasures are for the majority of players.

2 Likes

Then they are sending a pretty clear message, I’d say. Hopefully, they actually consider the ramification of fixing a long lasting “bug” and have a long term solution at least planned before they do. This has come back to bite them before. I just know they can’t have a plan at this point because they have already stated they weren’t aware there might be a problem until last week, and they won’t even be investigation into whether or not there is a problem for about two weeks. Guess that means they don’t think it is that important one way or the other?

The lower rarity treasures will be the hardest hit. The fewer of those you use now, the closer your projected “fixed” xp and current xp will be… but you’ll lose some in nearly every category.

Yeah, something I considered might be happening. However, while not done all that recently, there has been extensive sampling of other chest style drop in the game, many of which were done in lots of 50 or 10. There was no indication of this error at that time. Current drop rates for most chest types tend to “feel right” anecdotally, and, if you are on a losing streak, using a binomial probability calculation will generally show you aren’t that far off (I’ve seen some what would be <0.1% cumulative probability occurrences reported dealing with chest drops, but few and far between).

But also on this for every other random chest style drop (I’m not 100% sure on this one since it was never stated specifically, but we have been assuming) drops are calculated by first pulling a random number representing chance of category, then another random number for chance within the category, and so on, until an item is reached. For example, when opening gold keys, to get to an “Orc” card, you need to roll “troop”, then “common” then “(whatever the id of orc is from the list)”). It is possible the same algorithm used for other types of chests drops isn’t the same one as for portals. Meaning portals could use the same methodology but have an error where one of the numbers in the chain isn’t actually a new random number assigned.

I believe there is a simpler explanation here: different server files are actually being used.

I recall a similar issue with a kingdom being left out of tributes a while back: Blighted Lands does not give tribute

The basic probably was when they updated into using a new set of tables and didn’t clear the old ones out. On cursory inspection, everything looked right, but actually two different tables were being used.

This transition might have occurred when they implemented the drop tables in the first place. They used to send out data to the client that could be intercepted that contained sets of drop rates that were allegedly “outdated and incorrect” (even though the parsed data lined up with both what is being reported and what is being experienced on all normal chest types, and was used to catch multiple times when troops were inadvertently left off drop tables). However, if true, this means at some point at least two different versions of the same droprate table existed. The datamined versions of portal drops were posted here when delves were released (!) Spoiler Alert (!) -- [Any Details Provided are Subject to Change], and line up exactly with the current stated drop rates.

Wishful thinking, but it could be that these rates here were the “outdated and incorrect” ones the whole time, and that file is still kicking around and somehow is the one being used to configure the display rates. And nobody noticed because all the other display rates were the same as the actual rates the whole time. Meaning the rates we are experiencing are the correct ones, the display rates are wrong.

I agree this would be the least-damaging and simplest short term fix. I still think long term the way that portals are interacted with (the ‘value’ of a chaos shard dropping sharply once you pull all troops from a faction to mythic) could be done in a way that incentivizes always trying to push forward a bit softcaps, rather than just treating the soft caps as hard caps as I’m sure most people do.

2 Likes

I’m new here so take with a grain of salt. But there’s got to be an economic reason behind all this, the drop rates being broken and no one doing anything about it, even after you (Mithran) embarrass them in front of everyone in the forums and hand-feed them several solutions.

I guess you must love this game to go through all this time and effort for people that created the Diabolist class a few days ago. People who implemented a 40% cleanse token of cedric. I hope your effort does not go to waste, but I’m skeptical.

Drop rates being broken actually happens quite regularly, most players just don’t notice. It’s always treated as a hush job, there isn’t any official acknowledgement unless there is absolutely no way around it, there isn’t any information once it gets fixed, the only viable answer for anyone bothering to inquire amounts to “we didn’t change anything”. The most recent example is [Not a bug] Adventure Board misconfigured for Traitstone tasks, I wouldn’t be surprised if anybody doing a task analysis over the past three weeks would find that distribution rates have changed significantly, despite “everything working correctly”.

I guess there are some legal consequences associated to admitting such mistakes. Like the few times event chests were actually unable to pull the kingdom mythic, as the grand prize in their gasha system. I just wish it wouldn’t make trying get these type of issues fixed such a miserable job, there’s a huge amount of effort involved to report it accurately enough to be considered beyond doubt, and the only official feedback you’ll at best get is that you are just imagining things.

I just wanted to add to that info. I have saved records for my last 8 sessions of opening 400 chaos portals + I went and open 1k chaos portals yesterday.
So, my total treasures from 4200 chaos portals (84k shards):
Common - 326 (7.8%)
Rare - 1036 (24.7%)
Ultra-Rare - 544 (13%)
Epic - 253 (6%)
Legendary - 102 (2.4%)
Mythic - 30 (0.7%)

The only significant difference with your data is the rate of Mythic treasures but I believe it’s a flaw of a method of compiling from different people - people much more inclined to post very lucky/unlucky data (we all can see a lot of that in new mythic threads), and 1 set of 13 Mythic treasures skewed your data too much.

That assumption may be incorrect. As this assumption has to be proved first and foremost, I did not read carefully anything that follows. Since all the drops are calculated server side, there is no clear and fool proof way to test the assumption. However, doing a simple runs test on a sequence of data might be used to prove or disprove the assumption. Since there are quite a few outcomes, I would estimate a sequence of 500 should be sufficient for 95-99% significance.

So, there are 3 possible ramifications of the results:

  1. The numbers are not random or not independent or not random and independent.
  2. There is a coding error server side with the calculations.
  3. Developers have intentionally or non-intentionally input wrong drop rates, so technically speaking, the app can be re-reviewed for compliance with Google play store rules, for example, which require disclosure of drop rate. I presume, the drop rates have to be correct, and not simply some arbitrary random numbers be put there claiming that this is our drop rate.

Since this has been brought up so much recently we are doing a more thorough investigation this week.

5 Likes

@Mithran the screen shot in your original post displaying the official claimed percentages, do you have your own estimate for all of those numbers? Nothing fancy, text is fine :stuck_out_tongue_winking_eye:

Those drops show the treasure pulls. Any chance you also have the troop pulls for those drops and could post them here? I’m curious whether the troops on their own follow the published rarity distribution within the troop pool (413/182/77/28), it might help to home in on the root cause. It feels unlikely that whatever is causing it is isolated to shard chests, token chest drop rates look highly suspicious too.

What are the token chest drop rates “supposed” to be? I have not seen them published anywhere, but I would like to know…

From these two guildmates:
97R/42UR/19E/13L (171 out of 390 total pulls)
123R/56UR/29E/15L (223 out of 500 total pulls, screenshots shown in OP)

Totals 215/98/48/28 (389 out of 890 total)
55.27%/25.19%/12.34%/7.2% (out of just troops)
(legendary rate of total pulls here is about 3.1%, the only one higher than the stated rates, everything else significantly lower)

Versus the displayed ratios of
59%/26%/11%/4% (out of just troops)

So very similar… until we get to legendaries, which would have the lowest degree of confidence. However, this sample here is in line with anecdotal evidence… in general, people finishing legendary ascensions way before the other ones, which is why the mythic +4 cost didn’t change much in my estimates under current rates above.

Avg shard costs to mythic (1 copy) under the stated rates is (roughly):
4400/4500/4200/4200
7140 for mythic legendary on average is far above the others (next one is avg 5200 to get the epic to mythic +4, < 5k avg for the others)
With the current rates, there are multiple reports of already having legendaries at mythic +4(or 6, or 8 as in the above sample set) before pulling the last rare or ultra-rare.

With the caveat that these are estimates - I don’t want people going around quoting these categorically as the “true rates”, I just know they are so far off the stated rates in some instances that the stated rates can’t possibly be correct - and the understanding that closer they are to 50%, the more confidence I have in their relative accuracy, heres all treasures recorded to date, in a simplified table (after adding my sample set and Neritar’s):

C R U E L M Treasures Portals
TOTALS 578 1917 1038 453 189 75 4250 7765
% of Treasures 13.60% 45.11% 24.42% 10.66% 4.45% 1.76% 100.00%
% of Total 7.44% 24.69% 13.37% 5.83% 2.43% 0.97% 54.73%
Displayed % of Treasures 25.00% 25.00% 25.00% 14.00% 8.00% 3.00% 100.00%
Displayed % of Total 7.50% 7.50% 7.50% 4.20% 2.40% 0.90% 30.00%

Bolded row (% of total) are how they would display in game. @UKresistance Other stuff shown for reference.

Recalculating for additional samples, our avg expected hoard xp estimate per portal is now 30.35 (down from 31.85). Our avg hoard xp per portal under stated rates would still be 21.075. At a 400 portal pull (8k shards), this would mean the current hoard level estimates for “now” would be about 3-4 levels lower. Still a projected reduction of 27 hoard skill points before deeds and about 41 after kingdom level 14 for the same amount of shards used if we change from “sampled rates” to “displayed rates”. Still a pretty big gap, so my earlier statements still apply.

Yeah, it very well could be intentional, even though it does seem wonky for a “rarity” distribution to heavily favor the “ultra-rare” item, with the “common” and “rare” being significantly less common. Still waiting on that post they said they’d put on the forums detailing those rates. I promise I wont pick them apart unless they make an assertion that is completely out of whack with current observations, like cedrics token drop rates being stated to be anything lower than 35%, because there are already enough samples floating around to show that would be highly unlikely.

I don’t want to discourage drop rates being shared by the dev team, we need more of this. Well, not more of this this, but more of being able to independently verify stuff, and then hopefully being able to communicate that something is off and get a speedy (hopefully beneficial) resolution to an issue. The climate on all things drop rate seems to be “we don’t want you to know about us because you’ll constantly bother us that it is wrong”, but when it is wrong, with ample evidence to show it is, shouldn’t we be bringing it up?

9 Likes

Any news? The display rates are still the same a week later, a new faction is fast approaching and some of us are wondering if we will be dealing with a new reality for expectation of troops/treasures when opening up the next set of portals. If we are getting nerfed before then (or if we already have been, it won’t be widely noticed until Friday), it is probably better to know about it now.

Edit: So no news?

9 Likes

@Saltypatra @Sirrian
Sorry for the directness, but please provide any updates before the new faction event starts (even a nothing to report is an update or sorts)
Thank you

A nerf should really not even be on the table. It’s hard enough as it is to get enough rare treasures to level delves above 100, a nerf would give players even more reason to not care about high-level delving.

Nobody’s complaining about it being too hard to get faction troops and nobody is steamrolling delves with multiple hoards above level 200. If it ain’t broke, don’t fix it is a very viable response.

Just a quick update for you all.

3 Likes