On Tue, Sep 03, 2002 at 02:04:46PM -0400, Ralph Krausse wrote:
> Can anyone explain why Value = 52 instead of 53?
>
>
> float fValue = .53;
> Int16 Value = (Int16)(fValue * (float)100);
Because the compiler interpreted ".53" as ".529999999" just to piss you
off [1], and the Int16 cast is very literal. Casting to integer typically
just truncates. If you would like 52.999999 to truncate as 53 instead
of 52, you know what to do to it before you truncate it :-).
John "Google for 'what every computer scientist should know about
floating point'"
[1] or maybe it was the 99.99999673. or...
--
For information on using the Palm Developer Forums, or to unsubscribe, please see
http://www.palmos.com/dev/support/forums/
