https://bugs.freedesktop.org/show_bug.cgi?id=86005
Robinson Tryon (qubit) <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |[email protected] --- Comment #1 from Robinson Tryon (qubit) <[email protected]> --- (In reply to Jindrich Svorc from comment #0) > Created attachment 109088 [details] > Spreadsheet sample and a screenshot of my laptop just in case it works on > your computer > > Problem description: > In some cases value 1 is not correctly calculated. > > Please see attached files for explanation. I am attaching ods file as well > as a picture. The reason is that I tried it in MS Excel and it shows the > same problem as well. So I am not sure whether the problem is not somewhere > in windows or in my machine rather than in Calc. > I tried it in OpenOffice 3.1.1 on RedHat 5 and it shows the same issue. I think this is just a matter of floating-point arithmetic. > Current behavior: > equation > =INT(8.45)+(8.45-INT(8.45))/0.6-7.75 gives 0.99999999999999800000 > the same for Floor function > =FLOOR(8.45,1)+(8.45-FLOOR(8.45,1))/0.6-7.75 gives 0.99999999999999800000 Note that there are 14 9's and 15 non-zero places in the number: 0.999 999 999 999 998 000 00 If we compare that result with a simpler calculation such as =1/3 we find a similar situation with only 15 decimal places displaying the expected result values: 0.333 333 333 333 333 000 00 So it appears that while LibreOffice may format a number to 20 places, the internal calculations are only valid up to 15 places. -- You are receiving this mail because: You are the assignee for the bug.
_______________________________________________ Libreoffice-bugs mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
