On 13/07/09 17:23, Dotan Cohen wrote:
I have found another case of Calc rounding numbers in spreadsheets,
this time on my own machine. That means that in two different Calc
spreadsheets, on two different operating systems, I have seen Calc
round numbers in spreadsheets. Has anyone else seen this happen?
I don't know how big your numbers are but ...
It seems that Calc on a 32 bit computer stores numbers as 64-bit
floating point values. Of these 64 bits (assuming IEEE format), 53 bits
are used to represent the actual digits, the remaining bits being used
to store the sign and exponent. This gives an accuracy of about 15
decimal digits. You can see this by entering increasingly large numbers
into a column formatted as integer (format as "number" with format code
"0"). I got the following results:
Entered Displayed
1234567898765 1234567898765
12345678987654 12345678987654
123456789876543 123456789876543
1234567898765432 1234567898765430
So a lowish 15-digit number is OK but a 16-digit number isn't. In fact
accuracy breaks down at some point in the 15-digit number range.
If you want accurately to store codes (phone numbers, product codes, ID
numbers etc.) of 15 digits or more you *must* store them as *text*. If
you actually want to handle (as in "do arithmetic on") *integer values*
of 15 or more digits then I don't think Calc will do it unless
(possibly) you move to a 64-bit computer - I don't know if Calc will use
128-bit floats which would give you 112 bits of integer accuracy or
about 33 decimal digits.
Does this help?
--
Harold Fuchs
London, England
Please reply *only* to [email protected]
