https://bugs.freedesktop.org/show_bug.cgi?id=67026
Tomaz Vajngerl <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |NOTABUG --- Comment #1 from Tomaz Vajngerl <[email protected]> --- (In reply to comment #0) > Problem description: > > If you multiply 0,000005 by 1000000 the result is 5 > > Playing with GPS coordinates, at one cell I did 43,271815-43,27181 and it > gives 0,000005 exact but if you take that result and multiply it by 1000000 > the result is 4,9999999945 > > LibreOffice 3.5.7.2 > Build ID: 350m1(Build:2) > Ubuntu 12.04 x86_64 > > Current behavior: > 4,9999999945 > > Expected behavior: > 5 > > Operating System: Ubuntu > Version: 3.5 all versions Hi, Unfortunately such errors are caused by the real number representation (called floating-point) in computers and happen because of limited capacity to store the numbers and because you can not represent some decimal fractions (like 1/10 -> 0.1 for example) exactly in binary. Read more about this in [1]. I think that Calc sometimes works around the issue by using fraction representation (represents numbers in numerator / denominator form) if it is possible but sometimes (like in your case) it can not. Usually in such cases you need to round the number to a certain decimal places you think it makes sense with ROUND operator. For example (=ROUND((43.271815-43.27181) * 1000000, 5)) if you want to round to 5 decimal places. Regards, Tomaž [1]: http://docs.python.org/2/tutorial/floatingpoint.html chapter Floating Point Arithmetic: Issues and Limitations -- You are receiving this mail because: You are the assignee for the bug.
_______________________________________________ Libreoffice-bugs mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
