https://bugs.documentfoundation.org/show_bug.cgi?id=130202
Bug ID: 130202
Summary: Floating-point number round-trip lossage
Product: LibreOffice
Version: 6.3.4.2 release
Hardware: x86-64 (AMD64)
OS: Linux (All)
Status: UNCONFIRMED
Severity: normal
Priority: medium
Component: Calc
Assignee: [email protected]
Reporter: [email protected]
Enter the following numbers and formula:
A1: 31415926535897932384
A2: 31415926535897900000
A3: =A1-A2
Expected number in A3: 32768 (not 32384 because of floating-point roundoff).
Actual number in A3: 0.
Both cells display 3.14159265358979E+019 when edited. If you try to subtract
31415926535897932384 from 31415926535897967616, the latter number is turned
into 3.1415926535898E+019, but the difference is still 0. The ulp of those
numbers is 4096; it should be possible to enter two numbers that size whose
difference is 4096.
When a floating-point binary number is converted to decimal for later
conversion back to floating-point binary, it should be converted with enough
digits that conversion to floating-point binary results in the same number. In
this case, the number should be displayed in the edit box as
3.1415926535897932E+019.
--
You are receiving this mail because:
You are the assignee for the bug._______________________________________________
Libreoffice-bugs mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
