Fran�ois Van Emelen wrote:
>   210 rem v11 returns 1.1E-2 readable=no, correct=?  , should display .011

1.1E-2 means 1.1 * 10^(-2) = 1.1 * 0.01 = 0.011

This is by the pretty similar to the way the number is saved
internally.

>   240 v20=12345678.123
>   250 v21=12345678.011
>   260 r20=v20-v21
>   270 print v20,v21,r20
>   290 rem v20 returns 1.234568E7 readable=no, correct=?  , should 
> display 12345678.123

I'm currently not certain about the actual limits (and due to the flu
and currently laying in bed I'm too lazy to look them up), but you're
probably asking too much from the poor floating point values. Floating
point values can be big numbers or numbers with a fairly high
precision after the ".", but not both at the same time.
You will always get rounding errors.

FP values are much more evil than most expect anyway. It's not even
possible to represent simple values like "0.1" (0.5, 0.25, 0.125...
all no problem. But 0.1 is). That you do get a printed "0.1" in the
end is only due to rounding technics in the FP print code. This is the
reason why currency calculations are normally done either with fixed
point arithmetic or in the lowest possible unit (cent, pence) without
any fractional part at all.

Marcel

Reply via email to