https://bugs.documentfoundation.org/show_bug.cgi?id=101837
--- Comment #2 from Alex Kempshall <[email protected]> ---
Had another look at this one and it suspiciously looks looks a rounding error.
Pushed the numbers through the medcalc website and compared it with Calc. The
precision on medcalc is 9 digits after the decimal point whereas for calc it's
15 digits after the decimal point. I've increased the precision each time from
0.9 to 0.999999999999999. Everything seems in order until we get to the last
iteration.
Calc MedCalc
4.86518205192533 4.865182052 FALSE =CHIINV(F25,G25) 1
0.9 10
2.55821216018721 2.55821216 FALSE =CHIINV(F26,G26) 2
0.99 10
1.47874346383567 1.478743464 FALSE =CHIINV(F27,G27) 3
0.999 10
0.888920357912875 0.888920358 FALSE =CHIINV(F28,G28) 4
0.9999 10
0.545169540878432 0.545169541 FALSE =CHIINV(F29,G29) 5
0.99999 10
0.338126003244954 0.338126003 FALSE =CHIINV(F30,G30) 6
0.999999 10
0.211104115873973 0.211104116 FALSE =CHIINV(F31,G31) 7
0.9999999 10
0.132327944421752 0.132327944 FALSE =CHIINV(F32,G32) 8
0.99999999 10
0.083152274011688 0.083152274 FALSE =CHIINV(F33,G33) 9
0.999999999 10
0.05233106650163 0.052331067 FALSE =CHIINV(F34,G34) 10
0.9999999999 10
0.032965528624475 0.032965455 FALSE =CHIINV(F35,G35) 11
0.99999999999 10
0.020778597606181 0.020778598 FALSE =CHIINV(F36,G36) 12
0.999999999999 10
0.013099986480001 0.0131029 FALSE =CHIINV(F37,G37) 13
0.9999999999999 10
0.008262202343253 0.008262202 FALSE =CHIINV(F38,G38) 14
0.99999999999999 10
0.005090405273006 0.005211772 FALSE =CHIINV(F39,G39) 15
0.999999999999999 10
--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Libreoffice-bugs mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
