Spiral Knights - The problem behind scientific approach to Black Kat hunting

The problem behind scientific approach to Black Kat hunting

25 replies [Last post]

Thu, 09/25/2014 - 13:46

Round-Shinigami

Hello, since I know quite a bit about statistics and how random number generators work in PCs, I wanted to share a little opinion with you all. I have a specific opinion and you won't like it.

Thing is, the chance of getting a Book of Dark Rituals (same as Black Kat spawn rate) is random, not incremental.
Problem with random number generator versus statistics is the fact its random.

Normally, you'd have a higher chance for a non-usual event to happen if you tried and tried and tried. As long as there was a non-zero chance of something unusual happening in the first place. Simply speaking, statistic says, that if you have 99% to hit someone in Spiral Knights, you'll have slightly smaller chance to hit them twice in a row, and a rather low chance to hit them 100.000 times in a row without missing once.

The sad part is: this doesn't apply to Computers. Computer random number generators reset chances EACH TIME they roll a random number. This means, effectively:

Every time you kill a black kat, you have a fixed chance of a book appearing. And chance is so small, you have a better chance of winning a lottery. If lottery was played once a day, and you played for 10 years straight, you'd certainly win at some point. Problem with computer-based random number generator: it can simply roll you for 100s of years without letting you win, statistics don't apply to random number generators (or you could win instantly).

This is why I keep saying that "low chance grind" is the worst thing in MMO computer games. Collecting 1.000-10.000 items of some specific sort, or killing 1.000 Spookats would be simpler, as it'd give you a goal at which you'd obtain your desired item. As it stands now, you have a chance to get no books of dark ritual even after 10.000 black kat spawns. The chance is low, but it exists.

So, if you're going to keep on playing this even, please do so for your own pleasure and enjoyment, but don't expect to get a book of dark rituals. If you win a lottery, great! Except that you won't. Some win it by pure coincidence. But you won't. And I won't. And that guy there won't. And the guy left of him won't. If you're going to be sad about not winning the lottery - don't. Its not made to be won.

My conclusion:
Enjoy the game! :)

Thu, 09/25/2014 - 14:24

Golfdinger

"the chance of getting a Book

"the chance of getting a Book of Dark Rituals (same as Black Kat spawn rate) is random, not incremental."
Probably, but we don't know for sure how drops work. It's reasonable to test otherwise.

"if you have 99% to hit someone in Spiral Knights, you'll have slightly smaller chance to hit them twice in a row, and a rather low chance to hit them 100.000 times in a row without missing once[...] this doesn't apply to Computers"
What doesn't? What you just described is typical RNG.

"And chance [of finding a book] is so small, you have a better chance of winning a lottery."
no

"Some win it by pure coincidence. But you won't."
Thanks.

Thu, 09/25/2014 - 14:32

Bopp

not a great post

I agree with your ultimate point, that players should stop worrying about Black Kat Raiment and just enjoy the game. And yes the designers could have made this event more player-friendly and less frustrating.

Sorry, but I'm not convinced that you "know quite a bit about statistics". For example, the concept of independence is crucial to your post, but you never mention it. (Or maybe your presentation is obscured because you're not speaking in your native language?) Statistics offers tools for dealing with every problem that you bring up. Pseudo-random number generators are good enough that a player of a game will never statistically detect that they are not truly random.

Thu, 09/25/2014 - 14:36

Skepticraven

↓

"If lottery was played once a day, and you played for 10 years straight, you'd certainly win at some point."

Statistics disagrees with that statement. In fact, the Bernoulli distribution describes how the probability changes when you test something multiple times.

Thu, 09/25/2014 - 15:12

Fehzor

You have an interesting view of statistics, and by interesting I mean wrong.

The crux of your argument is that-
Normally, you'd have a higher chance for a non-usual event to happen if you tried and tried and tried.
[But that]
this doesn't apply to Computers
[Because]
Computer random number generators reset chances EACH TIME they roll a random number.

Yes, computer RNGs have a new set of odds each time. But what about dice? Do they take into account the last 10 rolls?

"Well we gave this guy snake eyes 3 rolls in a row, better pay out with some 6s now to make up for it"

lol no.

Let me prove this to you really quickly with an experiment that you can do at your computer. You will need:

A coin

Access to either Java's Random class or any other prng

Lets call heads 1 and tails 2, and set our prng to 1 and 2. Generate random numbers on each until you get a '2', recording how many tries it takes you to get a '2'. Do this 20 times for the coin, and 20 times for the prng, then calculate the averages. Did it take you significantly less or more tries on either? You could do the same for a D20, but that would require more trials. The variance of the event is the real issue. It makes it feel far less "fair" than it actually is, statistically.

I generated 20 random numbers for you with java as well:

0, 1, 0, 1, 0, 1, 1, 0, 0,1, 1, 1, 1, 0, 1, 1, 0, 1, 0, 0

Using the following code:

class prng{

public static void main(String[] args){
Random random = new Random();

for(int i = 0; i<20; ++i){
System.out.print(random.nextInt(2)+", ");
}
}

}

As you might not have easy access to java's RNG.

Thu, 09/25/2014 - 15:48

Dibsville

I like how innocent your view on statistics is, but you're looking at it the wrong way. Yes, RNG is random and it almost certainly is not additive. That much is obvious. The mistake here is that people already know that. When testing, it's always in the back of someone's mind that the statistics are in fact not additive. But that doesn't make the statistics any less approximate. People already know that it doesn't work out like we've killed X amount of Black Kats, so we'd get a Book in Y amount of more Kat kills. Instead, the point of the scientific approach is to give people a general idea of how rare an item is, and when it would make sense to actually expect the item to drop. It does not mean that it's telling people when the Book is going to drop, and it certainly doesn't mean that the Book is any more common just because we've killed a set number of Black Kats. That's not the point of the scientific approach at all, so I fail to understand what the point of this even was. You're just explaining what people already know, as I currently am now.

Also, I like how you compared the "always-the-same-percent chance" (except it's not actually always the same because of pseudo-generation) of a Book dropping to winning a lottery.

Thu, 09/25/2014 - 16:08

Turn-Me-On

@OPSo, youre saying that the

@OP

So, youre saying that the chance that 1 person who kills 100 BKs has is equivalent to the chance that 10 people killing 100 BKs each has?

As someone stated before, the percent is obviously not additive. If it were, the Bodr wouod be available for tokens, much like the grasping aura was (for a seemingly high amount).

Statistics dont guarantee anything. In this case, Statistically, 10 people grinding for 1 book (not each obviously) gives, in an absolute sense, a higher percent of finding the Bodr. (Ex: the 10 group would have a 5% chance vs a .5 of just one person).

Now, I dont really give af about BKR, and would like to leave with a short clip that describes everyone (including me) in this thread:

www.youtube.com/watch?v=IRsPheErBj8

Enjoy the game ;)

Thu, 09/25/2014 - 16:31

Avihr

Hmm, i've fought worse

And then you also have people saying that there's a 99% chance of you getting a book after killing 100 black kats or something like that...

shows how many players are out there having no idea about the real chances of getting one book despite there being so many posts adressing the topic

Thu, 09/25/2014 - 17:14

Spiral-Spy

The sad part is: this doesn't

The sad part is: this doesn't apply to Computers. Computer random number generators reset chances EACH TIME they roll a random number. This means, effectively:

I am afraid, this is not only wrong, but in fact exactly the other way around.

Dice don't have a memory, so they "reset" on every roll, giving you a fresh 1/6 chance on every attempt.

TTL Logic (upon which computers are build) doesn't allow for true randomness (you simply can't do it with 0, 1, AND, OR, NOT). Computers instead use pseudo RNGs. Put it in simple terms, they compute an irrational number and for each "roll" return the the next digit after the nth digit (with n typically being the system time when the RNG was initialized).

Nerd note: You can force your next kat to be a black kat by throwing trick or treat confetti.

Fri, 09/26/2014 - 07:10

Enfeebler

@Aviri

@Aviri These people are right, not after 100 Black Kats, but still statistically they should get (aren't guaranteed, but should get) a book after X amount of BKs with those 99%. Let me explain it to you since I think you misunderstood what they mean and you probably ignored Skepticraven's link with Bernoulli distribution as well. It's a 'tool' that allows you to tell how big is your chance to get X amount of successes in Y amount of tries, with fixed (stabile) percentages. And just like it was said in OP, all events are independent. If you have basic knowledge about probabilities then you can get take it a little bit further and calculate a chance for X and less/more successes in Y tries by using one of first laws that you'll learn in this subject.

For example you killed 200 Black Kats and the chance to get the book each time is 0.5%. Let's see what's the chance to get 1 or more BoDRs. First you have to calculate the chance of getting no book and if you use the formula correctly you'll get 36.7%. After that all you have to know is that chance of getting no books and chance of getting 1 or more books combined give 100% (there are simply no other options than 0 or 1<). So you just substract, 100% - 36.7% = 63.3%. That's your result, no one is saying it'll happen for sure or that 200th BK will drop the book with 63.3% rate and each next one with higher, but you have 63.3% to get at least 1 book after killing 200 BKs.

If you're too lazy you can just use Wolfram Alpha's free tool and it'll do it all for you (in our case: CLICK, just add up "x=1" and "x>1").

Fri, 09/26/2014 - 00:58

#10

Ironclaw-Mender

Why are you guys doing complex calculations over a game again?

Fri, 09/26/2014 - 05:13

#11

Round-Shinigami-Jr

Don't mind me using my FB

Don't mind me using my FB account, I recently changed my Steam password and am still memorizing it.

Fehzor
Thanks for the code but I wrote this post after doing a number of Java tests (learning Java at this moment) to calculate Pi using random number generator. The fact is, you can prove Pi is really 3.14 statistically using a random number generator. Problem is: you need some 10 million repeats to reach a number close to Pi we know to be a constant. Do it 100 times and it can be 4.

Bopp
Yes, its not my native language (Ukrainian is), so I may have some trouble explaining my point of view on science-related stuff to natives.

Skepticraven
I'm sorry, my knowledge of statistics as a science is rather limited, and I do not wish to engage in science battles, as I'm a complete newbie through and through in the field.

@all

The point of this topic is not to discuss scientific approach to Book hunting (topic may be a bit misleading), its just a small (seemingly large) message:
You're very unlikely to get a book, no matter how many Black Kats you kill, as your chance doesn't change from Kat to Kat. So instead of hunting for the book, just enjoy the game! If you get a book, you'll be many times more happy this way! If you don't, you won't be sad! You knew this was gonna happen!

Thanks :)

Fri, 09/26/2014 - 06:11

#12

Round-Shinigami-Jr

What I'm trying to say about

What I'm trying to say about statistics and "kill 200 kats to get a book": No.
Thing is: I just performed 10 millions of tests. 1 test = killing 200 black kats, each having a 0.5% Book of Dark Rituals drop rate (it can be anywhere from 0.5% to 2% by calculations of other players, but I took the pessimistic approach).

By my calculations,
If you kill 100 Black Kats, there's a 39% chance you'll get a Book of Dark Rituals. Test performed 10.000.000 times.

If you kill 200 Black Kats, there's a 63% chance you'll get at least a single Book of Dark Rituals. Test performed 10.000.000 times.

If you kill 1.000 Black Kats, there's a 99.3425% chance you'll get at least a single Book of Dark Rituals. Test performed 1.000.000 times.
Note: 99% chance doesn't guarantee success, only 100% does.

If you kill 2.000 Black Kats, there's 99.9959% chance you'll get at least one Book of Dark Rituals. Test performed 1.000.000 times.
Note: You didn't get a book after 2000 Black Kats killed? Well, tough luck!

10.000 Black Kats gives me 100% success rate. Except killing 10.000 Black Kats would require around 80 years at my current pace, considering there are 2-3 events per year...

I need 80 years to guarantee myself a Book of Dark Rituals O_^

Here's program code that allows you to calculate my values (written in Java, compile it in BlueJ):

// BlueJ Project
//
// Estimates chances to get a Book of Dark rituals in a drop from a Black Kat
// Takes user-specified amount of tries and user-specified amount of kats killed per try
// Chance is calculated as amount of hits divided by amount of tries, multiplied by 100

import java.util.Random;
import java.util.Scanner;

public class BlackKatHuntChanceTester
{
public static void main(String[] args)
{
System.out.println("Book finding chance after 100 Black Kats killed");
System.out.println("Enter the number of tries: ");
Scanner in = new Scanner(System.in);
double tries = in.nextInt();
int books=0, hits=0;
System.out.println("Enter the number of Black Kats killed per try: ");
double kats = in.nextInt();
double success=0.0;
double chance = 0.5; // chance a Book is dropped when you kill a Black Kat

int n=1;
Random generator = new Random(42); // because 42 is fair
for(int i=0;i0) hits++;
books=0;
if((i/tries*100)==10*n) // every 10%
{
n++;
System.out.println("Calculating... "+(i/tries*100)+"%");
}
}
double successRate = hits * 1.0 / tries * 100;

System.out.println();
System.out.println("Success rate out of "+tries+" tries is: "+successRate+"%");
}
}

So just enjoy the game and relax :)

Fri, 09/26/2014 - 06:44

#13

Skepticraven

↓

"10.000 Black Kats gives me 100% success rate."

No, it doesn't. In fact, my link in my previous post tells you that statistically it will never hit 100% but instead will asymptotically approach it. It is likely that your 1 million test cases are not enough to detect the tiny probability of not obtaining the book with that quantity of tests. In fact, using this guy's link to wolfram alpha's solution, you would on average need 1.7x10^22 trials of 10,000 random numbers to detect a single failed instance (with 0.5% chance).

Is it practically impossible to not obtain at least one book in 10k black kats? Yes, but I would also argue that is is practically impossible in the 2k instance as well.

Fri, 09/26/2014 - 06:47

#14

Bopp

right and wrong

The problem isn't that statistics is "wrong" or "irrelevant" or "unhelpful", as you suggest in your original post. The problem is that people misuse statistics and probability theory, to arrive at incorrect conclusions dressed in a costume of quantitative correctness.

What I'm trying to say about statistics and "kill 200 kats to get a book": No.

Suppose that an event has probability P of occurring on each trial, and the trials are independent. The probability that it occurs at least once, when you perform N trials, 1 - (1 - P)^N. If you set P = 0.5% = 0.005 = 1/200, then yes, you arrive at 39% for 100 trials, 63% for 200 trials, etc. Good.

Also, the expected number of trials, to get your first success, is 1 / P. So if P = 0.5% = 0.005 = 1/200, then you expect to try 200 times, to get one success. It is crucial to understand that "expected" means something like "average". Some people will get lucky and succeed in fewer than 200 trials, while others will get unlucky and succeed in more than 200 trials, and some players may never succeed. "Expected" does not mean anything like "guaranteed".

I need 80 years to guarantee myself a Book of Dark Rituals O_^

No. Even then, you are not guaranteed to get a book. There is always a small chance that you will fail. Your calculator is just rounding small numbers down to zero.

Fri, 09/26/2014 - 07:41

#15

Round-Shinigami-Jr

SkepticravenA true 100% is

Skepticraven
A true 100% is never possible, that much is true. I'm simplifying things (or my program simply rounded things into 100%).
No books after 2000 kats chance is also very unlikely. But the chance is big enough to be shown as 99.99%.

Bopp
What I meant is:
Those people who believe you will get a book after 200 tries are wrong. 200 tries gives you a 63% chance, good but not great. To get a great chance, you'll need to kill 1000 black kats.
As I said earlier, there's no true 100% chance, its rounded up. But its as close to 100% chance as I can get in my life - if I spend the entire life trying :D

Mystigryx
Can't say for others but I do complex calculations for a game because complex calculations are fun!
Once I was keeping a log of many material prices on Auction House (back in the day it was more popular than now), and I earned quite a bit of money using that info - I knew when one item got too cheap or too expensive and bought/sold respectively. It was also fun to see how certain materials drop in price when Arcade gates change (you can predict those changes by watching the next gate form). Back in the day we didn't have mission system, so we could only play via Arcade.
T'was fun! More fun than current system, at the very least. It had its inconveniences, but generally it made you feel like playing the game, not just point-and-click browser-based text game.

Fri, 09/26/2014 - 07:54

#16

Bopp

right

Those people who believe you will get a book after 200 tries are wrong. 200 tries gives you a 63% chance, good but not great. To get a great chance, you'll need to kill 1000 black kats.

Right, although it depends on what you regard as a "great" chance. Also, I suspect that people quoting the 200 number are using the *expected number until first success*, as I explained in post #14. When studying just about any random phenomenon, expectation is one of the first quantities that one computes.

As I said earlier, there's no true 100% chance, its rounded up. But its as close to 100% chance as I can get in my life - if I spend the entire life trying :D

Great. Talk to you later. :)

Fri, 09/26/2014 - 08:03

#17

Fehzor

@OP

You're absolutely correct that there is no guarantee of you finding one, but consider a simpler case-- finding an ecto drop. You're never guaranteed that either, but you're very likely to find one. I'd really care mostly about how long it takes for 50%, 75%, 95% and 99% of people farming to receive a book. At 50%, you're more likely to have found a book than not. You can then consider yourself unlucky. At 75%, you're very likely to have found a book and at 95%, you're almost certain tohave found a book. At 99%, you're almost guaranteed to find one. You don't have to be at 100% for a chance to pan out in your favor.

@Aviri? I think it was Aviri that said this?

And then you also have people saying that there's a 99% chance of you getting a book after killing 100 black kats or something like that...

Actually, we can make statements like that if we make assumptions about the drop rate of the book, which we can also make using Cheshire's data-- even if more data would really help us make those statements. Google binomial distributions and binomial distribution calculator.

Fri, 09/26/2014 - 08:46

#18

Turn-Me-On

@Mystigry Cause errybody

@Mystigry

Cause errybody here is a: http://www.youtube.com/watch?v=IRsPheErBj8

Mon, 09/29/2014 - 09:43

#19

Cheshireccat

Say it with Pictures!

Because pictures are fun! :)
http://i.imgur.com/Z4RuiHc.png

Red dots are 5% increments
Green dots are where I personally have found books
99% is around 917 (as discussed). That's very high but doable over two events.
90% is around 460. That is high but doable. I myself earned 403 this event and 199 last event.
Formula was p=(1-0.995)^x

If I am completely wrong though, someone please correct me.
--Cheshireccat, the cat that loves pudding.

PS: Look for my next Cheshireccat's Classes to appear in the next few days with my finalized event data (and data combined with the previous event). I've got over four times the number of data points!

Tue, 09/30/2014 - 12:50

#20

Klipik-Forum

This entire discussion is actually pointless; there's nothing you can do to influence your chances of finding a Book other than killing more and more BKs. The important thing is how much you care about how many other people have found it.

Tue, 09/30/2014 - 13:36

#21

Bopp

probably, but don't know

This entire discussion is actually pointless; there's nothing you can do to influence your chances of finding a Book other than killing more and more BKs.

You're probably right, but we're not certain that you're right, in the absence of detailed data.

Knowing the spawn rates with more precision would help players estimate how much grinding they had to do, and hence whether they wanted to do it.

Tue, 09/30/2014 - 14:38

#22

Flowchart

Not really. It would just satisfy the people who are obsessed with detailed data to the point of insisting on it.

Tue, 09/30/2014 - 16:55

#23

Bopp

2.7

When crafting costs were raised in May 2011 --- for example, 5-star costs went from 300 energy to 800 energy --- players exploded with rage. So apparently a factor of 2.7 makes a big deal to players, when it comes to their costs.

It is easily possible that our current Black Kat and Book rate estimates differ from their true values by a factor of 2.7. Thus gathering data on rates may benefit players.

Edit: To clarify, I am talking about gathering data to determine Black Kat and Book rates. I am not talking about gathering data to determine whether speed running (or wearing green, or praying to a deity) affects rates. That seems unlikely enough, that I wouldn't spend my time on it. But nor do I object, if other people want to spend their time gathering such data.

Thu, 10/02/2014 - 01:42

#24

Dawnlife

Normal Distribution

I agree with Shinigami that every BK killed give a fixed ratio for book, and previous kills does not effect the rate. This will be easier in coding (the simplest code has the least bug)

So we says you will get one book per 100 BKs assuming the rate is 1%. But does not necessary means everybody surely got a book every 100BKs

It is downed to normal distribution. 60% of the player should get their book around 80-120Bks. And there will be lucky one and unlucky one but the percentage is lower than medians.

Well, I got mine in 600+ pages, i.e. 200+ Bks or 1000+ runs, I think I am far away from median too :(

Thu, 10/02/2014 - 06:15

#25

Bopp

apparently not

Well, I got mine in 600+ pages, i.e. 200+ Bks or 1000+ runs, I think I am far away from median too :(

The best estimates we have say that Books drop in 0.5% of Black Kats. That's 1 in 200, which is roughly your experience. So it appears that you've had a typical experience. We can't be sure without nailing down the rate better, though.

Search

The problem behind scientific approach to Black Kat hunting