Yeah, I want to emphasize this in case the people making decisions about how to handle this aren’t aware. Vault keys basically don’t exist as drops outside of vault events. Keeping epic keys at 10% of this rarity during events is a really bad solution, and should not be considered as an option.
I’m level 1401 and I don’t think I’ve ever found a normal Vault key outside of an event. 10% of that rarity is absurdly rare.
I’m currently at 1520. I’ve gotten two outside of gnome weekend. Ever. I don’t remember the exact date when gnomes were introduced, but it sure feels like a long time ago.
I typically spend most of my time fighting Events, Delves and Dungeons, so outside of a few daily AB battles and some PvP to reach rank 1, I don’t encounter many gnomes.
I would say I’ve had more than 2 vk drops in non-Gnome events, tho I realize others spend more time in non-Gnome events farming Explores. I will try to capture data in between these Gnome weekends, tho I make no promises.
Executive Summary at the bottom.
The anecdotal base rate for vault keys is probably at least closer to ~1% than 3.18%, which also aligns better with the data we got from the first weekend. An assumed 3.18% (making the vault event “triple drop rate” for vault keys) would still have everybody posting here firmly on the “poor” luck side to a degree that is still improbable to the degree of being basically impossible to describe the assumed rate with the sample set with a binomial hypothesis test (x3 vault key in event means an expected 3.33% of VKs are EVK during event instead of 10%, where 0.7% is still way too much of an outlier over 1000 samples, whereas a x10 during the event would mean an expected 1% of VK during the event are EVK instead of 10%, which is within the realm of plausibility for the first data set).
At this point, there have been ample opportunities to clarify that the 10% only applies outside of the vault event and that epic vault keys are not proportionally increased inside the vault event, (nor are they intended to? which is the real question) which would probably irritate a lot people but would clear up all confusion. If this was the case, it should have been clarified one of the many times it was brought up, including when the question was asked directly. That we had to into another vault weekend without having an easily verifiable plausible explanation is extremely disheartening.
A second set of samples was mostly to help to eliminate any perceived bias and error more than anything.
But lets look at the recent weekends set, pre commited only. We now have 5 for 218. However, it would be disingenuous to just directly compare this do the original claimed ratio of 10% (10:1, or 9.1%) because of this information:
There are multiple quotes from devs on the forums that what was experienced on the first weekend was not a bug. They they looked into this, and despite it not being a bug in their eyes, slapped a flat increase on this. So an intended 10% going into this wasn’t even accurate anyway, because it should be higher than 10%.
But lets do it anyway. The best possible outcome for the initial claim being “correct”, even if we take liberties giving the benefit of the doubt with what correct means (10:1 ratio rather than 10%), means that we want to perform a binomial test on 5/218 versus a rate of 9.1%. Even if we do this with the precommited samples, at 99.99% confidence that the rate of 9.1%, we still have to reject this.
If you want to repeat the test yourself, heres a short guide:
Or just use an online calculator to get the exact binomial calculation:
http://vassarstats.net/binomialX.html
Based on what we are observing here, the following is now the prevailing theory, and the only one that makes sense at this point.:
So what about other theories?
Why does any of this matter?
The decision making process and how this affects the larger game metrics. EVKs were released with a claim that they would be “about 10%” as rare as vault keys. Vault Keys are nearly non-existent outside the vault event already, nobody “hunts” for keys outside of the event weekend. The claim heavily implies that, “on average, about every 10th vault key I earn from battles will be an epic vault key”. With nearly all vault keys from battles for an average coming from a vault weekend already, one would measure this in number of weekends, ie., number of sessions before it is likely for them to get an EVK, and push accordingly.
So lets throw out a very rough estimate of 120 battles on a vault weekend to get a vault key on average. Doable in roughly 40-45 minutes grinding explore 2s… on average. So your expectation of EVK would come on average after 6.6-7.5 hours. Due the the rarity, this expectation would mean that you could grind explore2s for 8 hours a day during an entire vault weekend and still have a tangible chance to not get an EVK. So already pretty rare if EVK rates are proportionally increased during a vault event and 10% of vault keys are always epic.
But if they arent, we have to assume ten times that long as a baseline. So our expectation of a single EVK comes at roughly 66 to 75 hours of mind-numbing grinding on vault weekend time. So your expectation becomes… I’m not getting one in one sitting, or even one weekend. We can triple that expectation outside of vault weekend due to lower gnome rates overall, but if they aren’t increased during vault events then you have a proportionally massively increase chance to see it outside the event given your “inside event” time would only be about three times as efficient at finding EVKs rather than thirty, and there are (usually) 25 days of non-vault-event time per 3 days of vault event time. This changes the EVK achievement from a “go chase this on vault weekend” to a “you’ll get it when you get it” kind of deal after thousands of battles. This would fundamentally affect how a huge number of players interact with the game, as this claim is artificially inflating interaction numbers on vault weekends (at least the first time, I’m already seeing signs of burnout here after just two vault weekends). And while not directly correlated 1:1, interactions drive monetization. Hard to say whether the fallacious (or at best misleading) claim is going to help or hurt here beyond very short term (especially given the potential scandal), but the fact remains that this number being out there the way it is is causing money to change hands.
And while this is probably less severe of an issue than rates being published wrong on directly monetized stuff. As I have said before, drop rates in this type of game are sacrosanct. There is a responsibility to get it right, so we can trust that it is right, even if “getting it right” results in no buff to EVKs and just a clarification on how they are intended to work based on an off-the-cuff claim given in the patch notes. Once it is correct, then we can move on to whether or not they are “too rare”. But we need to start by getting on the same page. Like, if you are intended to get it randomly as a fluke and barely have any influence over when other than being three times as efficient during vault weekend, it might shift perception a bit on the role these items are intended to play.
@Saltypatra -
tl;dr: Epic Vault Keys are not about 10% as rare as Vault Keys, at least during vault weekends, they weren’t last weekend, and they still aren’t now. The simplest explanation (and the only one that even makes sense anymore given claims on both sides and data gathered) is that they do not proportionally increase (or more likely don’t increase at all) during event weekends, and we (the playerbase) still don’t know whether or not they are even intended to. If EVK drops are not intended to increase proportionally to Vault Key drops during vault event weekend, please clarify this so we can move on from there about discussing how rare they are even at intended rates. If they are intended to increase proportionally to Vault Key drops during vault event weekend, well, there is ample evidence to suggest that they currently aren’t, and need to be fixed.
From a guild mate who lurks the forum but doesn’t have an account:
172 gnomes found total
20 vault keys total
7 pity keys
13 found keys
0 epic vault keys
From my mother:
287 gnome-eligible Battles,
34 gnomes found (33 killed successfully),
4 Vault Keys,
0 Epic Vault Keys
And for myself I was going to do some nifty stats like some people above, but I feel it’s redundant at this point. Regardless, for those curious, here’s the full list of what I ran through this weekend:
8
G - Vault Key
23
T - 4000 Gold
5
T - 5 Gems
6
M - 20 Ultra-Rare Ingots
2
T - 2 Diamonds
10
T - 4 Gem Keys
21
J - 200 Shards
6
S - 1000 Souls
1
M - 9 Legendary Ingots
14
T - 1 Event Key
19
T - 3 Event Keys
19
T - 2500 Gold
1
G - 200 Glory
14
T - 100 Souls
1
J - 20 Rubies
10
T - 2 Diamonds
17
T - 5 Gems
19
M - 4 Mythic Ingots
7
S - 5000 Souls
4
T - 100 Souls
15
T - 100 Souls
3
T - 500 Souls
3
M - 20 Ultra-Rare Ingots
8
G - 200 Glory
T - 2 Diamonds First back-to-back!
2
J - 20 Emeralds
6
G - 200 Glory
11
M - 12 Epic Ingots
3
M - 12 Epic Ingots
15
T - 100 Souls
10
M - 20 Ultra-Rare Ingots
12
S - 2500 Souls
7
D - 20 Chaos Shards
13
S - Vault Key
7
T - 5 Gems
34
T - 2 Diamonds
1
D - 20 Chaos Shards
24
J - 20 Emeralds
20
T - 2500 Gold
10
M - Vault Key
5
M - 4 Mythic Ingots
2
J - 20 Sapphire
2
T - 100 Souls
6
S - 1000 Souls
2
T - 20,000 Gold
6
T - 3 Event Keys
6
T - 100 Souls
37
T - 100 Souls
19
M - 12 Epic Ingots
4
S - 2500 Souls
11
T - 5 Gems
20
J - Vault Key
12
M - 4 Mythic Ingot
M - 20 Ultra-Rare Ingots
8
T - 500 Souls
7
D - Vault Key
10
T - 100 Souls
1
T - Vault Key
29
S - 5000 Souls
12
T - 5 Gems
5
T - Vault Key
5
J - Vault Key
S - 1000 Souls
13
T - 2 Minor Earth Traitstones
13
T - 500 Souls
15
T - 2500 Gold
4
M - 20 Ultra-Rare Ingots
6
S - 5000 Souls
16
T - 5 Gems
9
T - Vault Key
10
S - 2500 Souls
23
M - Vault Key
6
T - 1 Gem Key
11
S - Vault Key
8
M - 12 Epic Ingots
I’ll try to summarize the options:
1). Developers are openly lying trying to cheat the players or simply messing up. This is a pretty silly supposition imho, as I personally see no reason for that happening never ever on the side of the developers. There is a certain degree of trust required to take people’s money, even if it is a casual game, so giving up this reasonable degree of trust for something like that is simply not reasonable.
2.) There is an honest mistake somewhere in the code which has to be corrected to match the description. This is a very likely turn of events, and hopefully eventually it will be acknowledged and corrected by the developers provided that they can actually find what went wrong.
3.) There is some confusion with the decimal placement somewhere. The most likely hypothesis would be that 0.1 was assumed to be 10% by mistakes when it is indeed 0.1%, which is 0.001, which is probably the case meaning that everything is indeed working as intended by the developers and the description is simply wrong.
So, I personally think that option 2 and option 3 are both really likely.
IMHO, this whole issue has been blown way out of proportion. And, yes, it does not matter IMO.
Sure, I’m also willing to shrug it off either way. What’s missing though is an explanation. Also, it doesn’t sit right with me that the onus of proof is on the player base - where is the flip coin evidence that everything is working as intended? The silence is deafening.
I obviously dont know if its option 1, 2 or 3 but I wont disqualify option 1 as easily as you. Mainly because the F2P games with lootbox involving real money actually have ZERO control from a superior authority! Unlike casino gambling machines who are extremely severely controlled.
This important fact must always be remembered.
My money would be on Option 3. I doubt we’ll ever know for certain.
Everybody see The Rock? Sean Connery, Nic Cage? That one mercenary had a great line. Not the ‘We ain’t soldiers, we mercenaries, and mercenaries get paid’; the other one. ‘Great, so we’re not incompetent. We’re just cowards’. 1, 2, or 3, devs don’t come out of this looking good. Hence, the silence.
And I agree, in the end, it does not matter. Maybe I’ll get the EVK achievement before the Geof one, maybe after. Maybe I’ll never get either. Doesn’t matter.
Here’s my rather disappointing stats for the weekend:
Eligible Battles: 172
Total Gnomes: 15
Vault Key Drops: 0
Pity Vault Keys: 2
Hey everyone, unfortunately, I made a mistake when answering some players’ questions about the Epic Vault Key drop rate on Vault event weekends. I answered that we had taken the feedback on and had increased the chance for Epic Vault Keys during the last weekend Vault event.
The correct information is that this change won’t go live until version 5.2 (which is coming soon).
To be clearer, while the normal Epic Vault Key drop rate is roughly 10% of a normal Vault Key when no Vault event is running, the Epic Vault Key drop rate will NOT be 10% of a Vault Key during a Vault Event, but the Epic Vault Key drop rate for the event has been increased independently for those events.
Also just want to reiterate that we did run tests and check player data for the Epic Vault Keys when the community started reporting issues with the drop rate and the data and tests came back that this is correct.
So these changes being made in 5.2 for Vault events is our response to community feedback only, not a “fix”. We will review the drop rate changes for the Vault weekends again after the first Vault event in 5.2.
I’m very sorry I gave the wrong answer about the Epic Vault Key drop rate for last Vault weekend. This is my fault and I feel like trash for it. I know a lot of people not only trusted me to give the correct information but also spent a lot of time tracking their Vault Key data, so I let you down.
Oops. [spoiler][/spoiler]
Edited post to add this to the end:
I know a lot of people not only trusted me to give the correct information but also spent a lot of time tracking their Vault Key data, so I let you down.
Hey, some communication. Awesome! Question:
So without feeling like you’re backing yourself into a corner with a hard statement… what kind of increase can we expect? Hopefully this isn’t some kind of “we increased the rate 0.01% more to say we did something” rate of increase, right?
Brave answer and very much appreciated. 
I did have an inkling that 10% outside of vault events is about right, because we had guild members reporting EVKs between the only two events so far that could drop EVKs. VKs outside of events is already pretty rare, so it is not possible to receive that many EVKs outside events unless the rates were different in and out of events.
Looking forward to more EVKs in vault events after 5.2!
Personally, I got an EVK in each event, so wasn’t too fussed. 
This is so not clear 
Am I reading correctly? The drop rate of Epic Vault Keys is roughly 10% that of the drop rate of Vault Keys outside of vault events. Tested and confirmed, working as intended, won’t be changing. I follow that much.
I don’t understand the part about vault weekends though. You said there’s no fix, so are you saying the drop rate for EVKs was previously intentionally left unaffected by vault weekends? Or was increased for the event, but to less than ~10% of VKs? And now, after listening to feedback, with the 5.2 update, the drop rate of EVKs will be affected by vault weekends and/or will increase to … something more than it is now, an undisclosed rate?
Please don’t feel like trash. Mistakes happen. Thank you for coming back and telling us about it 
@mitamata I know I said “to be clear” then I was anything but clear haha sorry.
Yes so previously the EVKs were meant to be unaffected by Vault Weekends, but based on player feedback we decided to change that. The increase for EVKs on Vault weekends isn’t 10% of the Vault Keys on Vault weekends.
The reason I’m being cagey about giving a % is because the programmers have left for the day and I’m not sure how to give a correct % for the EVKs on Vault weekends as it’s not a field in the code or something where it says very straight forwardly “this is the EVK drop rate on Vault weekends” I’ll see if I can get a % and if I can disclose it later on.
That might have been very important to have mention on release (for future references). It kinda changes expectations from the get-go
