2009/2/5 Brian Barker <[email protected]> > At 13:28 04/02/2009 -0600, Ely Schoenfeld wrote: > >> I discovered something odd with OOo 3.0.1 Calc. >> [...] >> If I add more decimals to the cell format, I get something like -0.390000, >> that's ok. >> But If I copy the last cell and then paste special with only text, numbers >> and dates checked, I obtain: >> -0.389999999999418 >> > > It's worth looking at exactly what is happening here. The three fractions > you quote, 0.34, 0.73, and 0.39 all terminate as decimal fractions, of > course. They are 34/100, 73/100, and 39/100, of course - and they terminate > because the denominator, 100, contains only prime factors that appear in the > number base, 10. But, expressed as binary fractions, these do not > terminate. In fact, they all finish with a twenty-digit recurring pattern. > (I think this is because of the two factors of 5 buried in that 100 > denominator: the twentieth power of two is the lowest one which is one more > than a multiple of 25.) So any representation of such fractions in the > binary workings of a computer are almost bound to be rounded and not > precise, no matter how many binary digits are preserved. >
Well... yes and no. (If I understood correctly) The precision error didn't happen when I added 2.34 plus -2.73, it only happened when adding 18282.34 plus -18282.73 so it must have to be related with the "size" of the operated numbers. Am I right? > > There is, I think, a direct solution to your needs. Go to Tools | > Options... | OpenOffice.org Calc | Calculate, and tick the "Precision as > shown" option. If you have this in place when you perform the > copy-and-paste, you should see a difference. (It's the copy that matters, > in fact, not the paste.) > > I trust this helps. > > Brian Barker > Joe Conner, Joe Smith and Brian: Wow... didn't know that option existed. That makes the whole difference. That was what I needed Thank you sooo much. Ely > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
