OK, OK, I recognize that the error comes through because the exponent of (y)
is 0 thus the precision is 1E-16, but after all, the answer from
FlpCorrectedSub should be clean.
I make my FlpCorrectedCorrectedSub
FlpCompDouble FlpCorrectedCorrectedSub(FlpCompDouble x, FlpCompDouble y)
{
Int16 ex;
Int16 accuracy = 15; // 15 decimals to the right of the biggest
exponent
ex = max(F64GetExp(x), F64GetExp(y)) ; // get the max exponent (the
F64GetExp is my func)
x.fd = FlpCorrectedSub(x.fd, y.fd, FlpAccuracy); // subtract the
two numbers
ex = accuracy - ex;
y.d = pow(10, ex); // fortunately I'm using MathLib (thanks
Rick Huebner)
x.d = x.d * y.d;
x.d = round(x.d); // clean the garbage
x.d = x.d / y.d;
return (x); // return a clean number
}
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of John
Marshall
Sent: Martes, 19 de Septiembre de 2000 11:36 am
To: Palm Developer Forum
Subject: Re: Serious Palm FloatNumber problem
Pablo Valle <[EMAIL PROTECTED]> wrote:
> [...snip... 0.999999 - 1 ...] will give a result of
> -9.999999998067111E-7
> ...and we all know that the answer is -0.0000010000000000
You do realise that
|-9.999999998067111E-7 - -0.0000010000000000| = 1.932889E-16 < really
small
and that that's about all the accuracy you should expect, right?
You have read "What every computer scientist should know about floating
point", right?
John "trolled again"
--
For information on using the Palm Developer Forums, or to unsubscribe,
please see http://www.palmos.com/dev/tech/support/forums/
--
For information on using the ACCESS Developer Forums, or to unsubscribe, please
see http://www.access-company.com/developers/forums/
