On 2010-08-18, at 16:53 , Raphaël Valyi wrote:
> BTW, Ruby get the result right always using Ruby Floats, not sure how they
> differ.
Different output formatting. Floats are floats are floats (unless they're
doubles) and can't store `0.1005` exactly, but they way they're printed can
change.
Take the expression above in a Python console and compare its repr and its str.
As for Ruby, check the value it yields if you print it with a precision of 17
or more:
>> sprintf( "%.17f", v)
=> "0.10049999999999995"
>> sprintf( "%.20f", v)
=> "0.10049999999999995048"
With a precision of 16 or less, the rounding will yield `0.1005[a bunch of
zeroes]`. Python exhibits the exact same behavior:
>>> "{:.16}".format(v)
'0.1005000000000000'
>>> "{:.17}".format(v)
'0.10049999999999995'
>>> "{:.20}".format(v)
'0.10049999999999995048'
Python's repr just happens to have a default precision of 17.
_______________________________________________
Mailing list: https://launchpad.net/~openerp-expert-framework
Post to : [email protected]
Unsubscribe : https://launchpad.net/~openerp-expert-framework
More help : https://help.launchpad.net/ListHelp
