I'm puzzled by this and the solution. 20.5 should never experience
any rounding by just declaring double var; var = 20.5.
20.5 = 40 34 80 00 00 00 00 00 , as stored in double format.
403 = + 16; 34 80 00 00 00 00 00 = .28125 the implied 1.0 makes this
number 16 * 1.28125 = 20.5. But the point is that 20.5 can easily be
saved in the 64 bit format without rounding.
Where does the rounding come in, in a straight declaration and
initialization?
Date sent: Mon, 10 May 1999 12:30:17 -0400
From: "Dave Lippincott" <[EMAIL PROTECTED]>
Subject: Re: Double precision number problem. Why 20.5 SOMETIMES is
stored as 20.499999000?
To: <[EMAIL PROTECTED]>
Send reply to: [EMAIL PROTECTED]
> I had the same problem. Best answer I got was the Palm OS does not perform
> rounding like other OSs. It displays the number sorted in memory, quantized
> errors and all. I ended up injecting error into my calculations to offset
> this problem. (i.e. I added 0.000001 to some variables in critical
> calculations to get the correct results.
>
> D
> -----Original Message-----
> From: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> Date: Monday, May 10, 1999 11:34 AM
> Subject: Double precision number problem. Why 20.5 SOMETIMES is stored as
> 20.499999000?
>
>
> >I am using CW 5.0 without the patch and PalmOS 3.0.
> >
> >when I compile the following code:
> > double var ;
> > var=20.5 ;
> >
> >during the debug, most of the time I see that var=20.4999990000, but
> >sometimes the right value (20.5) is stored.
> >
> >What is causing this mismatch and how to prevent it?
> >Is there any document where double float operations on PalmOS would be
> >explained?
> >
> >FYI,
> >I use 8-byte double setting of the Compiler
> >I am not using NewFloatMgr
> >
> >
> >Marcin Frelek
> >
> >
> >
> >
>
>
>
--
-----------------------------------------------------------------
Discussion Group: http://www.halcyon.com/ipscone/wwwboard/
Protect your constitutional rights. Your favorite one may be next!
-----------------------------------------------------------------
