"Matt Wlazlo" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > On 6/3/06, Bob Long <[EMAIL PROTECTED]> wrote:
>> >> > I'm noticing some strange behavior while calculating prices on my >> >> > invoice. >> >> > >> >> > A1=39:50 (hours worked) >> >> > A2=A1*24 (==39.83) >> >> > A3=65 (rate) >> >> > A4=A3*A2 >> >> > >> >> > Now A4 will equal 2589.17. But, somewhere else in the spreadsheet, >> >> > typing =39.83*65 gives 2588.95! >> >> I think you'll find it is a rounding issue. If you change the format >> >> of A2 >> >> so that you see more decimals, you'll see that it is really 39.833333 >> >> (recurring, I presume). Multiply *that* by 65 and you'll get 2589.17. >> >> >> > I tried re-adjusting the formatting of A2, but to no avail. >> >> More specifically, what did you do? Did you change it so you see more >> decimal points? You should be able to see 39.833333... rather than just >> 39.830000. >> > I changed the number of decimal points. The cell reads 39.833333333... > >> > Strangely, if I manually calculate =39.83*65, it correctly gives >> > 3588.95. Even windows calculator gives the same result. So I don't >> > think it's a fp precision problem. >> >> It's not a precision thing. Calc is holding the correct value internally. >> It's just that 39.833333... is not the same number as 38.830000! > > Ahh! Finally got it. I used ROUND(A2;2)*65 to get the correct answer. Which just begs the question of what is *really* correct! 39:50 (hh:mm) is 39.833333... (recurring). 39:50 is *not* 39.8300. So if you are rounding to lose some digits, you *may* be propogating errors in subsequent calculations. It really depends on how you are defining things for your purpose; just be careful, is all I'm saying. -- Bob Long --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
