https://bugs.documentfoundation.org/show_bug.cgi?id=80017

Matthew Francis <[email protected]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
                 CC|                            |[email protected]
         Resolution|---                         |DUPLICATE

--- Comment #5 from Matthew Francis <[email protected]> ---
This is an unavoidable consequence of the use of floating point numbers to
store numbers. The numbers which are being subtracted can't be precisely
represented, and so the result of the subtraction is marginally different from
what is compared to.

While there are other ways to represent numbers for computation, floating point
is used because arbitrary precision costs much more in processing time, which
would severely limit the size of spreadsheet calculations which could be
performed in real time.

The general solution to this issue when working with floating point numbers is
to test whether a number (such as a calculation result) is within a very small
amount of what it is compared to, rather than for precise equality.


For a related discussion of the same topic in Microsoft Excel, see:

http://en.wikipedia.org/wiki/Numeric_precision_in_Microsoft_Excel
specifically the section
"Subtraction of Subtraction Results"

*** This bug has been marked as a duplicate of bug 67026 ***

-- 
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Libreoffice-bugs mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs

Reply via email to