data to be posted when im done, itll take a while probably
Sat, 02/27/2016 - 12:25
im not sure if this has been done before, but I'd like my own evidence
so heres the explanation in the form of questions that have a tone similar to how people talk to me when they recognize me from the forums in game
Oh great, this guy, what does he want now?
the myth im trying to test is if a larger party yields more rarities per player
so far ive seen that the general consensus among people is "no" but ive had enough annoying people who think their anecdotal evidence is valid, im here to prove those annoying people right (even though their reasoning is wrong) or wrong (because they very well might be)
How will this be tested?
the goal is to test different party sizes in DaN (dreams and nightmares) and see the average radiant fire crystal count
Why would I want to help you find some boring statistic?
you probably have some unheated 5* items, heres another excuse to get rads for them
this will also help you to find if its better to do things solo to get runs done faster or to carry your noob guildies for more loot
What are the specifications?
-we are to run DaN, and only DaN
-elite difficulty (who in their right mind farms on normal?)
-for it to count all party members have to be there from lobby to refuge (if someone leaves at refuge just say how many people were in the party and give numbers for the remaining guys
-when in a party of more than one, give numbers for all party members (try not to post about the same run that one of the party members already posted)
-if someone leaves on the elevators, do not count the run (youre still allowed to finish it though, of course)
-do not get on the party button at refuge until all dead party members are loaded in, there is a bug where they will get stuck and only kicking them can fix it (this isnt for the testing, its just to make sure no one gets cheated out of rads)
Get to the point, what are we supposed to do?
easy, just do DaN runs
keep the same party size during the whole mission
look at the mission payout report at the end screen
post the numbers here with party size
you can add comments too
example post:
title:
ok I'll help
field:
2 person party
18 and 27 rads
hey voh I thought you were finally gone, stay inactive for once
if you dont have a forum account/dont want to take time to log on just send a mail or PM Vohtarak with party size, data, and optional comments
You have my attention, now wheres the data?
right here
solo rad counts: 18, 9, 15, 6, 45, 24, 24, 27, 12, 21, 27, 18, 24, 27, 15, 24, 18, 24, 21 (19 data pieces)
duo rad counts: 18, 27 (2 data pieces)
trio rad counts: (no data yet)
full party rad counts: 36, 21, 18, 18, 18, 12, 30, 18, 15, 24, 30, 18, 9, 21, 21, 18, 30, 18, 21, 36 (20 data pieces)
averages
solo:
duo:
trio:
full:
to be caculated after a sample size of at least 50 numbers for each
honorable mentions (otherwise nameless non forum contributors who helped a lot):
Skycapp
Super-Cheesy
Cobalt-Luster
Ariso
Mertorius
Will you ever shut up?
no, in fact I have more to say
according to fehzors (very well done) overall study of DaN, the average rad count for all party sizes was found to be 22.7
using the numbers that it takes 453 rads to heat a 5* item (the percentage chances really arent worth it if you have bad luck) it takes 19 point something runs for a single 5* item, I rounded it to 20 because im pessimistic
so I chose a 50 sample size to both get a large data set and to get people more stuff heated
once everything is calculated I'll take percentages for how much more of a payout you get for party sizes, but if its a very small difference I wont bother because that means party size has no effect
Sat, 02/27/2016 - 12:47
#2
Hmm.
I mean, it's great to have data and all that...
But on the release of the new loot system, the update post literally has the following said in it:
- Loot now drops for each member of a party and each party member must pick up their own loot. You now will earn the exact same amount of loot in a party as you do solo, so party up!
So uh, good to get statistics to back you up. The other day I did two back to back solo runs and first run I got 6 radiants, second run I got 45. Going to go out on a limb and say it's just RNG, nothing affecting it.
Sat, 02/27/2016 - 14:09
#3
now we've posted it
Our missions rarities page now has an analysis of Radiant Fire Crystals vs. party size, based on 51 runs of 18 distinct missions. As you can read there, we see no evidence that party size affects Radiant Fire Crystals.
That said, we don't mind sharing our data. Here is the subset of our data that fits your experimental design: 13 solo runs of DaN, with the following numbers of Radiant Fire Crystals.
24, 24, 27, 12, 21, 27, 18, 24, 27, 15, 24, 18, 24.
"when in a party of more than one, give numbers for all party members (try not to post about the same run that one of the party members already posted)"
By the way, I recommend against this policy. Within a single run, the payouts to the players are correlated, which is going to super-complicate your analysis. Unless you plan on handling that, it's better to have each data collector report only her individual payout. On the other hand, this correlation I'm worrying about probably matters very little.
Sat, 02/27/2016 - 17:07
#4
at least i got a response
@fangel
im making this as proof of that, i made this as a response to a guy who made the party wait 10 minutes to get a 4th member
@donkey
im making a larger sample size than that, and i dont quite trust lancer knightz data on exact numbers (pointing to the 42% chance of elites in sewer stash)
"By the way, I recommend against this policy. Within a single run, the payouts to the players are correlated, which is going to super-complicate your analysis. Unless you plan on handling that, it's better to have each data collector report only her individual payout. On the other hand, this correlation I'm worrying about probably matters very little."
i think youre misunderstanding me, if a party finishes a run the screen will say "w got 9, x got 12, y got 27, z got 30", w posts all four payouts, im trying to avoid x, y, or z from posting payouts from the same run, that would add numbers that had already been added and skew the data to what that run has
(updated OP up to here from comments and my own data)
Sat, 02/27/2016 - 17:16
#5
no, he doesn't misunderstand you
i think youre misunderstanding me, if a party finishes a run the screen will say "w got 9, x got 12, y got 27, z got 30", w posts all four payouts, im trying to avoid x, y, or z from posting payouts from the same run, that would add numbers that had already been added and skew the data to what that run has
Yes, it would be disastrous for your data set if multiple players posted the same observations as if they were different. But Donkeyhaute is talking about another problem. He's talking about how, if you take multiple observations within a given run, then they might be correlated --- inter-dependent on one another, because of how that particular run went for all players on that run. And if you were to ignore that, then your analysis would be damaged. But I agree with his final statement, that it probably doesn't matter much.
Sat, 02/27/2016 - 17:43
#6
if thats what he meant
i wouldnt consider it to be possible, its very common where one player can get wildly different numbers to another, such as a run i had a few minutes ago with a 36 and an 18, player earnings seem to be completely independent of other factors, i actually did test a few things (about 3 months after DaN came out) such as who does the most damage, whos applying status, whos moving the most, whos dying the most, compared to rads, ive since lost the data to a corrupt file but it found those to not affect anything
Sat, 02/27/2016 - 17:52
#7
quite possible
i wouldnt consider it to be possible
I consider it to be not only possible but probable. But again I wouldn't worry about it, because the correlation will not be strong and its effect on your conclusions will be negligible. :)
its very common where one player can get wildly different numbers to another, such as a run i had a few minutes ago with a 36 and an 18
That has little to do with whether player payouts are correlated. "Correlated" does not mean "identical" or "equal". It means that a relationship exists after uncertainty is taken into account. None of your anecdotes take uncertainty into account. Which is why you're collecting data, right? To get a handle on the uncertainty.
I look forward to seeing what your study produces.
Sat, 02/27/2016 - 17:56
#8
what uncertainty?
i did just say i took almost every aspect into account, are you saying that the game itself will cause people to get x rads if someone else got y rads?
Sat, 02/27/2016 - 18:22
#9
yes; it's quite simple
i did just say i took almost every aspect into account, are you saying that the game itself will cause people to get x rads if someone else got y rads?
Yes. For example, did you account for how many treasure boxes those knights were seeing? Anything that drops from treasure boxes, such as fire crystals, is (almost certainly) correlated with the number of treasure boxes that have been opened. As far as I'm aware, every knight in the party sees the same number of treasure boxes (although they might not be the same colors). Therefore the fire crystals that one knight gets is correlated with the fire crystals that another knight gets. Make sense?
Radiants in DaN is something of a special case, because I've never seen Radiants drop before the treasure room, which always has 41 treasure boxes (although the colors vary slightly). And that's why I tell you not to worry too much about this issue.
Sat, 02/27/2016 - 18:39
#10
it still doesnt seem co-related for knights
DaN has the same number of boxes every time in the treasure room, and from what ive gathered every knight sees the same color boxes that are there (even though the red and green ones can change per run), even with that it doesnt seem related between a knight getting one thing causing another to get something else
(side note, the maskeraith's cloak does not last long enough to fix a 1 word typo)
Sat, 02/27/2016 - 21:18
#11
yes
DaN has the same number of boxes every time in the treasure room
Yes. I said that already.
and from what ive gathered every knight sees the same color boxes that are there
In my experience, that is not the case. But admittedly I don't have video proof, and it's possible that I'm wrong.
even with that it doesnt seem related between a knight getting one thing causing another to get something else
You don't think that the number of boxes affects the amount of box-only stuff that knights get? How wouldn't it?
Sat, 02/27/2016 - 21:38
#12
um
from what youve said before it looked as if you meant to say that "if knight a gets x, then knight b gets y" as the uncertainty for correlation between knights, now youre saying that youve been talking about boxes broken (which has nothing to do with connecting knights), can you try to clear up what youve been saying because that makes no sense at all
Sat, 02/27/2016 - 21:55
#13
knights are connected through boxes
Knights are "connected" (meaning, correlated) through boxes, yes. This is all in my post #9.
Maybe it would be easier to understand without uncertainty. So let's suppose, for the sake of argument, that each treasure box is guaranteed to produce a certain number N of Radiants. This isn't true, but there is some long-term average, and you can set that to be N, if you're willing to average over many runs or ignore the variability from run to run.
* You do a run of DaN in which the first level has 0 boxes, the second level has 0 boxes, and the treasure level has 41 boxes. On that run, every knight in your party gets 41 * N Radiants, right?
* Then you do a run of DaN in which the first level has 12 boxes, the second level has 12 boxes, and the treasure level has 41 boxes. On that run, every knight in your party gets 65 * N Radiants, right?
The key word in the preceding scenarios was "every". The knights' Radiants aren't independent of each other. They depend on the boxes, and every knight sees the same number of boxes. In this way the knights are connected. Do you see what I mean?
I'm not trying to shoot down your study. I'm just trying to get you to understand some basic concepts that Donkeyhaute was pointing out.
Sun, 02/28/2016 - 05:55
#14
clarification
im making a larger sample size than that
I have edited the page to make clear that our results are based on 208 solo runs and 51 non-solo runs of 18 distinct missions. We count in this way because we are analyzing many missions which have different average payouts, so we normalize each mission's non-solo payout based on its solo payout.
So your sample size will be larger for non-solo runs but smaller for solo runs. Anyway, getting more non-solo data would be great.
Sun, 02/28/2016 - 06:09
#15
"trust" and "exact numbers"
and i dont quite trust lancer knightz data on exact numbers (pointing to the 42% chance of elites in sewer stash)
Do you think that we've faked our Sewer Stash data? No, we haven't. Here they are (with personally identifying information stripped out).
[["Sewer Stash", "6-2", 3],
[1, 3479, 1],
[1, 2922, 1],
[1, 2985, 2],
[1, 3298, 1, "", "2014/01/16"],
[1, 3269, 0, "", "2014/01/19"],
[1, 2962, 0, "eliorb", "2014/01/22"],
[1, 2809, 0, "eliorb", "2014/05/28"],
[1, 2654, 0, "", "2014/09/05"],
[1, 2865, 1, "eliorb", "2014/09/10"],
[1, 3302, 2, "", "2014/09/15"],
[1, 2625, 0, "eliorb, eliorb", "2014/11/15", 31, 1],
[1, 3303, 0, "", "2014/12/13", 35, 1],
[1, 3232, 0, "", "2015/01/04", 31, 0],
[1, 3108, 0, "", "2015/01/05", 28, 1],
[1, 2910, 0, "", "2015/01/12", 34, 1],
[1, 2475, 1, "", "2014/01/14", 28, 2],
[1.4, 2821, 3, "eliorb", "2015/04/13", 7+8+8, 2+1+0],
[1, 2941, 0, "eliorb", "2015/08/16", 11+14+8, 0+0+0],
[1, 3096, 1, "", "2015/12/07", 8+10+8, 0+1+0]]
The first three data are from before we starting recording rarities. Ignoring them, you see that we have found 7 Elite Orbs of Alchemy on 16 runs. That's 0.4375 orbs per run on average.
Now, someone else running the same mission might not get 0.4375. She might get 0.2 or 0.3 or 0.5. That's the nature of measuring a random phenomenon. There's always some uncertainty. There are various ways of calculating the uncertainty, but they are invalid when you have as few observations as we do. That's why we're always collecting more data. To understand the uncertainty.
You're going to have uncertainty in your study too. You will never get "exact numbers", but I will not fail to "trust" your data because of it.
Sun, 02/28/2016 - 11:49
#16
so much for forum data
@donkey
I never said your data is faked, but ive never seen someone find elites at even half that rate, and I know a lot of people who seem to think your data suffers from people posting runs where they do get elites and leaving out runs where they dont, im indifferent to these theories but not finding elites for months of nothing but sewer stash (multiple people other than me) seems to hint that its not quite right
it seems this thread is done with any information that could be gotten off the forums, no ones interested so once I get to my computer ill offload the information ive gathered, graveyard this derailed train, and continue working on it exclusively in-game
Sun, 02/28/2016 - 13:52
#17
reasonable, but wrong
"people who seem to think your data suffers from people posting runs where they do get elites and leaving out runs where they dont"
That is a reasonable suspicion. It would explain why our Sewer Stash orb rate is so high. But it's wrong.
Every one of those Sewer Stash data was collected by either Bopp or me. As you can tell from this thread, we are both much more careful than the average player. For example, we decide before the run, not after, whether we are going to collect data. So the source of bias that you mentioned is not happening. Those data are rock-solid. (By the way, Bopp and I together account for at least 95% of the Lancer Knightz mission data set.)
It's possible that something else in our testing procedure is skewing our results. For example, we're clearing every monster in the mission. Maybe the rates are different if you speed-run? I have no reason to believe so, but it would be fascinating to know, wouldn't it? :-)
Sun, 02/28/2016 - 15:48
#18
not likely
It's possible that something else in our testing procedure is skewing our results.
That's possible, but you're being too modest. It's more probable that you and Vohtarak-Forum are both right.
Suppose we're interested in the question, "In what fraction of Sewer Stash runs do I find at least one Elite Orb?" Now, there is a true answer, which we'll call p. But we don't know it, without reading the game code. So instead we collect data, finding at least one Elite Orb in 6 out of 16 runs. So we estimate p to be 6 / 16 = 0.375. (This is different from the 0.4375 mentioned above, because I'm asking a slightly different question.)
But there is a lot of uncertainty in that estimate. Using a normal approximation to the sampling distribution for p, we can build a 95% confidence interval. We find that p could be anywhere from 0.1329385 to 0.6170615.
So, Vohtarak-Forum, if you're finding an Elite Orb only 14% of the time, then your experience is consistent with ours, even though it doesn't look like it.
So why doesn't Donkeyhaute report this uncertainty in the Lancer Knightz results? Because the approximation that I just used is not actually valid for data sets this small. We need more data, before we can really quantify the uncertainty in p like this.
Vohtarak-Forum, we've been talking about this kind of thing for years. We know that the only is to patiently, carefully gather more data. So that's what we've been doing. And that's probably why Donkeyhaute bristles a little when you question this data set, on which he's worked so hard.
You might want to contact Donkeyhaute, who runs the Lancer Knightz data project. They have a page of data on mission rarities. They don't test party size vs. fire crystals, but they might, if you ask them. And separately they have tested party size vs. crowns, with no correlation found.