Homing in closer on damage numbers

Sorry for threading on your toes here, Skepticraven. I know you're doing something similar to this.
Here's Skepticraven's thread on the subject.
Doing a new thing. I've updated my lost souls chart. For the moment, I've only gone for even-numbered bonuses, because I plan to later cross-reference with Lancer Knightz data, which is also primarily found at even-numbered bonuses.
The calculation, as she goes
For Valiance, these are some of the numbers I have:
Shot: 245, 304 (no bonus, max)
Charge: 417, 517
If Skepticraven is correct, 245 actually means 244 < x <= 245, where x is the true damage number. It could be 244.0001, or it could be 245.
So at no bonus, it's in a range of 244-245, and at max it's in the range of 303-304.
We know (assume, actually. I'll be relying on a lot of assumptions that haven't been 100% proven for this thread) that max is 124% of no bonus, so we can cross-reference.
303-304 reverts to 244.3548... - 245.1613. Some of that falls outside of range, so we cut it off to 244.3548... - 245. We can apply this backwards again and find that max bonus has a viable range of 303 - 303.8.
Every damage bonus can be cross reference back with base, and then the base referenced back with every damage bonus to further narrow it. The more bonuses we can get, the closer we get. Simple, right?
And we can go further. We also know (assume) that charge attacks for most guns is 170% of the base attack. We narrow down the charge attacks amongst itself:
416.1290 - 416.9355 and 516 - 517
Then cross-reference shot vs charge.
Shot: 244.7818 - 245 and 303.5294 - 303.8
Charge: 416.1290 - 416.5 and 516 - 516.46
Why exactly do we care about this?
I dunno.

That sounds about right. While I was writing the program to verify Gear-Storm's formula, it reported it to be wrong for some numbers. This was because I hadn't checked it against the correct spectrum.
It got me thinking about this method of narrowment. I'm writing a check for all numbers related to Flourish, Blast Bombs, Sudaruska, CIV, and Rocket Hammer right now. I'm fairly confident all of those numbers can be cross-referenced since I know their relations. The results should be interesting.
After this, I can run a similar method to get a good estimate on monster defence in D24, which, using Gear-Storm's formula, can be checked for every number, not merely those above the treshold.

I've stepped on your toes already by going outside the D24/LD. I'm in full support of figuring this out. I'm just slow with getting all my data myself (especially the lower depths due to limited missions with howlitzer heads).
I know a couple reasons to care about it:
1. Gets accurate measurements for armor values (D1 = 10 exact, I'm getting close to finding D2 soon).
2. Get accurate measurements for armor modifiers (poison, sera, "boosted" enemy).
3. Allows heating curve to be measured (proto sword suggests this is hardcoded ratios).

I'm ending up with a minimum value below the maximum value. One of my assumptions are wrong. Here are some assumptions I can think of that I'm making:
- All of my black values on my chart are correct.
- I have accurately defined the relationship between damage numbers for all attacks on leviathan blade, final flourish, and sudaruska.
- Damage numbers are only rounded a single time. So when the charge damage for Sudaruska is 150% of the basic attack, and the basic attack of Sudaruska is 115% of that again, no rounding occurs to arrive at that 150% number.
- Final damage number is rounded up.
- Ruby's BigDecimal class is accurate enough to deal with these calculations.
I'm operating purely on vs souls value for noe, so no assumptions are made in regards to defence. That is, other than:
- Lost Souls and Howlitzer heads have 0 defence and there are no factors operating on damage numbers I'm not at some level aware of.
Upon closer checking, I can't find a relation between Flourish and status Flourish damage, so I left rocket hammer dash out of the equation. When I removed Rocket Hammer dash from the equation things looked a bit righter, and when I removed Suda charge attack/Flourish finisher (115% * 140%), I ended up with the exact same number for both min and max values for Leviathan blade's first attack:
323.21428571428567
I take it with a grain of salt, personally. If the suda charge value was off (despite looking right when only compared to individual damage values as opposed to all of them taken together), then a lot of others could be as well.

I have noticed that the 0* variants of suda/levi (hatchet/proto sword) are a 115% multiplier. I just tested the D4 values for hatchet charge, and they appear to be 150%.
My initial guess is it is likely bullet #1. +1 and +2 are the limiting datapoints, so I just tested +1/+2 and they are correct. That discounts that.
Edit:
All numbers appear correct.
Perhaps the % number is off?
I did notice that comparing levi and suda charge directly gives me a multiplier really close to 1.6125 instead of 1.61 (150% and 107.5% instead of 140% and 115%). Does this have something to do with Suda charge being able to apply stun?
Edit#2:
This makes me want to gather all the values from T1 GHT because I'm fairly certain the armor value there is 10 vs normal.

"(150% and 107.5% instead of 140% and 115%)"
Good find! Using 150% * 107.5% I get only one error that may or may not be precision-related. Levi's first hit with no bonus still comes out as
323.21428571428567 for both min and max. Levi with damage high seems to be exactly 362.0.
I'm not sure I trust it. Here's the code used with results.

Edit: Looking more at the output shows the "problem" is comparing 362.0000000000000000000000000008 to 362.0. (Previous comment deleted because it was wrong).
It appears that the number is calculated correctly as well. Base times the multiplier yields that value correctly.
After doing a short lookup of how bigdecimal functions - its just a float with an exponent. It is a float-related precision error.
I've been spoiled with using mathematica which automatically handles precision and propagation of error down to machine precision (default 16). Is there interest for me to rewrite this for mathematica for additional precision? :P
Edit#2:
I've been meaning to brush up on mathematica syntax, so I went ahead and did it anyway.
Using 100 digits of precision, I resulted in an answer with just under 99.5 digits of precision (Accuracy of just under 97 digits). It produced the same repeating decimal that you found and rationalized it to 4525/14.
As overkill, I added another order of magnitude to the precision and noticed that the repeating decimal repeated the same for another 10x more digits.
If you really want to look at the short mathematica script in plaintext, you can see it here.

What do you think the relations between shard bomb damages are? I was assuming something like this:
- Core: 100%
- Status shard: 110%
- Damage shard: 120%
However, this doesn't end up adding up.

Status shards have a pretty wide range still to get a good estimate.
Given worst case scenario with the current numbers, the multiplier ranges normalized to the core are:
Status: 110.07639386 - 109.819121339%
Damage: 119.82190914 - 119.66569038%
I can tell you that the shard is not 120%.
I do not like guessing this number to add another assumption as it appears different than a nice even value.
Edit:
Started gathering some super quick values for D1 (since I know armor). I dont have irontech destroyer or BaB just yet, so the range is currently at 48.9922481-49 for levi.

Did you mean to write 118.82-119.66?
That looks like it could be 110% × 107.5%, which is in line with other weapons.
Edit: Nope, definitely not it.
And I think I might have figured out levi damage for depth 1 a while ago. Take calibur damage (was it exactly 35 or something of the sort?), then multiply it by 107.5% for each star ranking it's upgraded.
Edit: Calibur was 40, but if I use that I get 49.691874999999996, which should be too high. Perhaps Calibur's damage is not as round as I thought it was? If I regress 48.9922481, I get 39.43682793213176 for Calibur.

http://wiki.spiralknights.com/Calibur
Maybe there's just a number written down wrong, but, the values for the first depth contradict at least one of our assumptions.
30, 36, 51 for 1st, 3rd and charge.
Assuming exactly 10 defence, this makes
39 > x >= 40, 45 > x >= 46, 60 > x >= 61.
The first and third hit check out against other, but the closest you get for the charge damage is exactly 60, assuming the first hit damage is exactly 40, which is exactly too low. I'm not sure how to explain this discrepancy, other than 'floating point shenanigans'.

Sometimes the minimization only requires 2 great numbers. The individual numbers that get as close as possible to that theoretical 40 exact are:
Untested values:
Cali 1st +4 +firestorm = 52
Cali 3rd +6 = 58
Cali charge +3 +firestorm = 74
I dont currently have a caliber to test them. Each of these values will minimize the damage individual damage range to +/- 0.08 or less. If the above values are correct, I think it would be safe to say both cali has an exact of 40 and the multipliers for 3rd/charge on the levi-line are accurate. Again, 40*150% = 60 exact. (50% of 40 is 20) Anything that says otherwise is having trouble using floating points in multiplication.
I do have a tempered caliber (3*) currently.
Its range using the 1st/3rd/charge is minimized to 42.94935897-43. Limiting datapoints are ACali +0 1st and ACali +1 charge. The range could be narrowed a bit more using ACali 1st +1 +pickup +firestorm.
We can also test 0* versions if the relationships between them are correct. Hatchet is a 0* suda. Heavy hatchet is a 1* suda.
I've estimated 34.5 damage for hatchet lvl 10 1st. This means 30 for 0* cali - which is interestingly the proto sword 1st swing value as well.
The question arises now - what is the relationship between damage increase by star count?
Numbers appear very close to:
0* 1* 2* 3* 4* 5*
30 ?? 40 43 ?? 49
It doesn't appear linear. In fact, I've noticed multiple times that there isn't a linear scale. Shield HP is really strange in stratum 5/6. Proto sword heating curve is non-linear. The interesting part is that this non-linear scale is used multiple times, suggesting the existence of the multipliers that you have been working to find. Perhaps I should heat and finish up a heating curve for 1* weapons?
You're teasing out the rounding effects. That's good. Is this because you're pretty confident that Gear-Storm's calculations explain damage up to rounding, and you're trying to get things exactly verifiable?