hey all,
i was tracking down a bug in a chunk of declares the other day, when
i found something which took me awhile to understand - i had written
a function to convert rb integers to fixed datatype. it read like this:
function int2fixed (i as integer) as integer
return i * &hFFFF
end
simple... easy to read... gave the wrong answers?? it appears &h is
returning a double, causing my int to be multiplied in double space
leading to rounding errors. when i switched it to use bitshiftleft
the problems went away.
my question to the list is - now that we have int64, can anyone
conceive of a reason we would want &h to return a double? to the
best of my knowledge &h will always be *trying* to represent an
integer value, correct? is it time to file a feature request? or is
there still some logic to this choice?
mike
--
Mike Woodworth
[EMAIL PROTECTED]
_______________________________________________
Unsubscribe or switch delivery mode:
<http://www.realsoftware.com/support/listmanager/>
Search the archives of this list here:
<http://support.realsoftware.com/listarchives/lists.html>
