On Mon, 13 Jul 2009 19:23:45 +0300 Came this utterance formulated by Dotan Cohen to my mailbox:
> I have found another case of Calc rounding numbers in spreadsheets, > this time on my own machine. That means that in two different Calc > spreadsheets, on two different operating systems, I have seen Calc > round numbers in spreadsheets. Has anyone else seen this happen? > Is it rounding the stored number or just the visible number? Search "precision as shown". Has the number exceded the OS's significant number capability (machine epsilon)? Floating point numbers get stored as log numbers and some (generally insignificant) rounding can occur because of this. http://www.akiti.ca/MachEps.html I created a quick sheet (column d formatted as boolean a1 = 1 b1 = 1 c1 = a1+b1 d1 = if(c1>a1;1;0) a2 = a1 b2 = b1/2 c2 = a1+b1 d2 = if(c2>a2;1;0) drag all four columns from row 2 down to row 60. You can see both where the formatting stops being seen as the full number (column b becomes 0) and the computer stops seeing the fraction as it is has become machine insignificant (column d goes FALSE). -- Michael All shall be well, and all shall be well, and all manner of things shall be well - Julian of Norwich 1342 - 1416 --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
