On Jul 27, 2006, at 1:18 PM, [EMAIL PROTECTED] wrote:
Are you sure about the "every other language" portion of your
statement?
Positive. My statement was "that I tried". I tried several of each.
In my experience it is normal for the fractional portion
of a double or single to be thrown away when assigning to an integer
property.
But there is no fractional portion to throw away.
72 / 2000 = .036, it does not equal .0359999999999999 like the
examples
have implied. .036 * 750 is 27.0, again, not 26.99999999.
I've seen lots of examples where different calculators come with
different values when faced with .9999999, but this number stops at
just 3
decimal places. If there were any .3333s or .666666s in the
calculation, I
could accept a rounding “issue”. But since there are no number to
round, it
sure looks like a bug.
--Seth
Try this code out and you'll see why it is the way it is.
It has everything to do with floating point math in a system based on
binary.
dim d as double
dim i as integer
d = 72.0 / 2000.0
msgbox " d = " + format(d,"-#,###.000000000000000000")
d = d * 750.0
msgbox " d = " + format(d,"-#,###.000000000000000000")
i = d
msgbox " i = " + format(i,"-#,###.000000000000000000")
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
