JOE Conner wrote:
Ely Schoenfeld wrote:Hello all.I discovered something odd with OOo 3.0.1 Calc. Hope someone could verify this for me. a) Put in two different cells the numbers: 18282.34 (For example A1) -18282.73 (For example A2) b) Add them in another cell. (For example A3)Doesn't matter if you use the "SUM" formula or just use a simple "+" sign.If I add more decimals to the cell format, I get something like -0.390000, that's ok.But If I copy the last cell and then paste special with only text, numbers and dates checked, I obtain:-0.389999999999418You can test that the problem IS affecting further calculations like this.a) Add 0.005 to the result (A3 in my example) You will obtain -0.38 if you are showing 2 decimals. b) Write exactly -0.39 in a new cell, and then add the same 0.005 to it. You will obtain -0.39 if you are showing 2 decimals.In both cases, if you add more decimals to the cells, you will obtain the same: -0.385 But if you copy and paste special with only text, numbers and dates checked, you would obtain in the first case -0.384999999999418 and -0.385 in the second one.I'm attaching an odt file here, hope it passes through. In case it's relevant, this happened on a: Windows XP Version 2002, sp3 32Bit Centrino Duo processor Should I open an issue about this? (Couldn't find one) Thank you all. Ely.<<SNIP>>You are using base ten numbers that are internally represented in base 16 hexadecimal. Rounding errors are inevitable unless specialized precautions are used.For example if you want two places of accuracy, think currency calculations for example, you can use =ROUND(X;2).In your example cell a10 displays the correct two digit display. If youincrease the digit display you will see that the actual number calculated is really =0.38500000 and with the common mathematical rule of roundingof which anything that is half or greater is rounded to the next greater number you can see that -0.38 is indeed correct.If you need multiprecision arithmetic, then I suggest you open the file X-NUMBERS which I have attached which has macros to deal with multipresision.Joe Conner, Poulsbo, WA USA
The server stripped off my attachment which was an SXC file. I saved it as an ODS and attached it here. Perhaps it will pass muster this time.
Joe
XNumbers_01.ods
Description: application/vnd.oasis.opendocument.spreadsheet
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
