https://bugs.documentfoundation.org/show_bug.cgi?id=152039
Mike Kaganski <[email protected]> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|UNCONFIRMED |RESOLVED Resolution|--- |NOTABUG --- Comment #1 from Mike Kaganski <[email protected]> --- (In reply to Jonny Grant from comment #0) > I understand all of this, I don't need an explanation of floating point... I > always recall "double" precision floating point has 15 decimal places. of > course "float" 32 bit only has 7 digits of precision. Is Calc not using > double precision? Are you sure you "understand all of this" and "don't need an explanation", given that you ask this? Your numbers don't use "12" (as in title) or "13" decimals. Number 287.950000000001 has 15 decimals shown: 2,8,7,9,5,0,0,0,0,0,0,0,0,0,1 And the last digit of that number (1*10^-12) comes from a negation involving two numbers, each of which have *four* decimals to the right of the dot; so for those numbers, the 1*10^-12 constitutes the 16th decimal. The negation makes that *normal* error visible because of the loss of significance. All that is explained in FAQ [1] and the linked Wikipedia article [2]. No, nothing can be done. [1] https://wiki.documentfoundation.org/Faq/Calc/Accuracy [2] https://en.wikipedia.org/wiki/Floating-point_arithmetic#Addition_and_subtraction -- You are receiving this mail because: You are the assignee for the bug.
