On Wednesday 17 August 2005 22:56, Jim McGowen wrote:
I wrote a test routine that multiplies and casts every number from 0.0001
to 0. and outputs to a text file via a conduit for analysis. I haven't
sifted through the whole set yet but so far I found these to be trouble
numbers:
0.141 -
Jim McGowen wrote:
Just keeps getting stranger... now 0.1551 * 1.0 is giving me 1550. I
have proof that it wasn't doing this yesterday!
Anyway, after some discussion with another programmer I work with, we
decided to change our app to do strictly integer math. That should
eliminate our
Henk Jonas wrote:
Jim McGowen wrote:
Anyway, after some discussion with another programmer I work with, we
decided to change our app to do strictly integer math. That should
eliminate our floating woes.
And it will be much faster too.
Will it? I'm not sure how floating point works on
Logan Shaw wrote:
And it will be much faster too.
Will it? I'm not sure how floating point works on Palm (you can
consider that a question...), but if it works via trapped F-line
instructions on 68k Palms, then the 68k CPU emulator used on ARM
Palms can probably emulate those instructions
Jim McGowen wrote:
Found something strange... Looks like Palm OS 5 changes certain numbers
multiplying. For example:
double x = 0.141;
long y = (long)( x * 1.0 );
y becomes 1409.
It's related to the limited resolution of floats I guess. Have you
searched google for such problems?
Casting to integer truncates the fraction part and almost never gives
the expected result.
If you want to isolate the digits you can round with something like:
long y = (long)(x*1 + 0.5);
For positive integer it will usually gives the expected result.
Like others said, it is all related
I only cast doubles to store them, less space and better precision.
I was writing a rounding function when I discovered this problem. My round
function correctly rounded 0.1405 to 0.141, it's only when I multiply the
result out and cast it for storage that it gets messed up.
Chris Tutty
I wrote a test routine that multiplies and casts every number from 0.0001 to
0. and outputs to a text file via a conduit for analysis. I haven't
sifted through the whole set yet but so far I found these to be trouble
numbers:
0.141 - 0.143
0.172 - 0.174
0.204
0.284 - 0.286
0.344 - 0.349
Just keeps getting stranger... now 0.1551 * 1.0 is giving me 1550. I
have proof that it wasn't doing this yesterday!
Anyway, after some discussion with another programmer I work with, we
decided to change our app to do strictly integer math. That should
eliminate our floating woes.
Jim
Found something strange... Looks like Palm OS 5 changes certain numbers
multiplying. For example:
double x = 0.141;
long y = (long)( x * 1.0 );
y becomes 1409.
Also:
double x = 1410.0;
double test = 0.141;
double y = test * 1.0;
double z = 0.141 * 1.0;
x = = 1410.0 = =
From: Jim McGowen [EMAIL PROTECTED]
Found something strange... Looks like Palm OS 5 changes certain numbers
multiplying. For example:
double x = 0.141;
long y = (long)( x * 1.0 );
y becomes 1409.
Also:
double x = 1410.0;
double test = 0.141;
double y = test * 1.0;
I have also seen this problem with multiplication and division. I have been
adding float support to iScript (WMLScript library) and noticed this. Adding
and subtracting floating numbers is fine. And math with integers are
perfect.
All I do is a * b or a / b. I can post some tests later. The
12 matches
Mail list logo
