You got me there. I admit its been several years since I converted
floating point to binary/hex. I never sat down to check the actual exponent
+ mantissa + sign math on the Palm. However, it is my understanding that
while Floats and Doubles cover a large domain of numbers, the actual number
set is finite. I would expect a float to be able to represent a simple
number such as 20.5 extremely accurately and a number such as 20.50003425
with the possibility of some accuracy loss.
The first time I brought up this subject on this group, I got the rounding
answer. (and it works for my applications which require accuracy to .0001)
If there is another answer answer that doesn't require any rounding voodoo,
I'm all for it.
>Where does the rounding come in, in a straight declaration and
initialization?
I found in my applications, I needed to tweak my numbers after performing
rather simple interpolation or other functions that required multiple
multiplication and/or division operations. I ended up checking numbers
during calculations to figure out where 'rounding' errors were being
injected, and modified results where needed. I was able to use a
spreadsheet on my PC to check the calculations and compare them to the
results from the Palm.
-----Original Message-----
From: Mike Davis <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Date: Tuesday, May 11, 1999 1:52 PM
Subject: Re: Double precision number problem. Why 20.5 SOMETIMES is stored
as 20.499999000?
I'm puzzled by this and the solution. 20.5 should never experience
any rounding by just declaring double var; var = 20.5.
20.5 = 40 34 80 00 00 00 00 00 , as stored in double format.
403 = + 16; 34 80 00 00 00 00 00 = .28125 the implied 1.0 makes this
number 16 * 1.28125 = 20.5. But the point is that 20.5 can easily be
saved in the 64 bit format without rounding.
Where does the rounding come in, in a straight declaration and
initialization?
Date sent: Mon, 10 May 1999 12:30:17 -0400
From: "Dave Lippincott" <[EMAIL PROTECTED]>
Subject: Re: Double precision number problem. Why 20.5 SOMETIMES is
stored as 20.499999000?
To: <[EMAIL PROTECTED]>
Send reply to: [EMAIL PROTECTED]
> I had the same problem. Best answer I got was the Palm OS does not
perform
> rounding like other OSs. It displays the number sorted in memory,
quantized
> errors and all. I ended up injecting error into my calculations to offset
> this problem. (i.e. I added 0.000001 to some variables in critical
> calculations to get the correct results.
>
> D
> -----Original Message-----
> From: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> Date: Monday, May 10, 1999 11:34 AM
> Subject: Double precision number problem. Why 20.5 SOMETIMES is stored as
> 20.499999000?
>
>
> >I am using CW 5.0 without the patch and PalmOS 3.0.
> >
> >when I compile the following code:
> > double var ;
> > var=20.5 ;
> >
> >during the debug, most of the time I see that var=20.4999990000, but
> >sometimes the right value (20.5) is stored.
> >
> >What is causing this mismatch and how to prevent it?
> >Is there any document where double float operations on PalmOS would be
> >explained?
> >
> >FYI,
> >I use 8-byte double setting of the Compiler
> >I am not using NewFloatMgr
> >
> >
> >Marcin Frelek
> >
> >
> >
> >
>
>
>
--
-----------------------------------------------------------------
Discussion Group: http://www.halcyon.com/ipscone/wwwboard/
Protect your constitutional rights. Your favorite one may be next!
-----------------------------------------------------------------
