https://bugs.documentfoundation.org/show_bug.cgi?id=162330

Mike Kaganski <[email protected]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |NOTABUG

--- Comment #3 from Mike Kaganski <[email protected]> ---
This is not a bug; see FAQ:
https://wiki.documentfoundation.org/Faq/Calc/Accuracy

Specifically, you compare these calculations:

1. -22.67
2. 176 + 340 - 16.6 - 3.5 + (-60 - 50.02 - 8.55 - 4)
3. -244.1 + 221.43

First, let's look at #1. The number -22.67 is already not representable in
Calc; an approximation is used, which is exactly

> -22.6700000000000017053025658242404460906982421875

In #2, you perform the calculation in parentheses separately (just noting,
because that affects the accuracy, because it excludes this part of calculation
from Kahan summation algorithm). Its result is, exactly,

> -518.5700000000000500222085975110530853271484375

Adding it to the rest of the sum, the result is, exactly,

> -22.670000000000072759576141834259033203125

which differs from #1 by ~-7.1*10^-14.

In #3, the result is, exactly,

> -22.669999999999987494447850622236728668212890625

which differs from #1 by ~+1.4*10^-14. At the same time, it differs from #2 by
~+8.5*10^-14. Note that this last difference is the biggest.

When you compare results of #1 and #2, or #1 and #3, they are considered equal,
because their difference is smaller than a given relative threshold. But when
you compare #2 and #3, the difference is larger than considered equal.

Yes, the pure math isn't applicable to spreadsheets. Users must be prepared to
that, and use relevant techniques (rounding, or approximate comparison).

-- 
You are receiving this mail because:
You are the assignee for the bug.

Reply via email to