https://bugs.documentfoundation.org/show_bug.cgi?id=150737
Bug ID: 150737
Summary: Bad float number calculation: 243,362.83 - 243,362.84
should equal -0.01 rather than -0.01000000000931
Product: LibreOffice
Version: 7.4.0.3 release
Hardware: x86-64 (AMD64)
OS: Linux (All)
Status: UNCONFIRMED
Severity: enhancement
Priority: medium
Component: Calc
Assignee: [email protected]
Reporter: [email protected]
Steps to Reproduce:
1. Input the following numbers in cells A1 and A2:
A1: 243,362.83
A2: -243,362.84
2. Input the following formula in cell A4:
= A1 + A2
Current Result:
-0.01000000000931
Expected Result:
-0.01
I understand that there is loss of precision in float numbers calculation due
to the limitation of computer CPUs, but we simply should find a way to avoid
such simple errors. This should not be allowed in Calc especially when used in
advanced calculation.
For instance, in Python, the same problem exists:
'''
In [1]: a = 243362.83
In [2]: b = -243362.84
In [3]: a + b
Out[3]: -0.010000000009313226
'''
But this problem can be avoided using the decimal module:
'''
In [4]: import decimal
In [5]: a = decimal.Decimal('243362.83')
In [6]: b = decimal.Decimal('-243362.84')
In [7]: a + b
Out[8]: Decimal('-0.01')
'''
The default Calc functions may not be changed to use the accurate calculation
(to be compatible with MS Excel or whatsoever), but at least Calc should have
its own function (maybe called DECIMAL?)
--
You are receiving this mail because:
You are the assignee for the bug.
