On May 21, 12:38 pm, Mark Dickinson <[EMAIL PROTECTED]> wrote:
> >>> a+0.999 # gives expected result
> 9999999999999998.0
> >>> a+0.9999 # doesn't round correctly.
>
> 10000000000000000.0
Shouldn't both of them give 9999999999999999.0?
I wrote the same program under Flaming Thunder:
Set a to 10^16-2.0.
Writeline a+0.999.
Writeline a+0.9999.
and got:
9999999999999998.999
9999999999999998.9999
I then set the precision down to 16 decimal digits to emulate Python:
Set realdecimaldigits to 16.
Set a to 10^16-2.0.
Writeline a+0.999.
Writeline a+0.9999.
and got:
9999999999999999.0
9999999999999999.0
--
http://mail.python.org/mailman/listinfo/python-list
