From: "Dave Lippincott" <[EMAIL PROTECTED]>
> You got me there. I admit its been several years since I converted
> floating point to binary/hex. I never sat down to check the actual
> exponent + mantissa + sign math on the Palm. However, it is my
> understanding that while Floats and Doubles cover a large domain of
> numbers, the actual number set is finite. I would expect a float to be
> able to represent a simple number such as 20.5 extremely accurately and a
> number such as 20.50003425 with the possibility of some accuracy loss.
You may have seen my other post requesting help with DoubleToAscii
routine I am working on. I am seeing some rounding also that I
cannot figure out. But some numbers should be able to be represented
100% accurate. Most of this response is "thinking out loud".
The example, to which this post, responded indicated 20.5 which is
one of those. Should be 100% accurate.
Other numbers that are within that domain should also be able to be
represented accurately.
4503599627370495 should be the largest number that can be represented.
Any whole number (without decimal) less than this should be 100%
accurate.
The odd thing I am seeing is that for any double with the integer
part with 8 or less digits, that the decimal part is accurately
represented to 8 digits. But for 9 digit integer parts the decimal
part drops to 7 digits of accuracy.
I think I have just answered my own (other post) email. As the
number of digits, in the integer part, increases, the number of
remaining bits to define the decimal part decreases. Therefore
there might be rounding, depending on the number (decimal part).
Not all decimal values can be represented in this float format. As
the integer part gets larger there are fewer bits remaining for the
decimal part. Hence inaccuracies. Or something like that.
> The first time I brought up this subject on this group, I got the rounding
> answer. (and it works for my applications which require accuracy to
> .0001) If there is another answer answer that doesn't require any rounding
> voodoo, I'm all for it.
>
> -----Original Message-----
> From: Mike Davis <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> Date: Tuesday, May 11, 1999 1:52 PM
> Subject: Re: Double precision number problem. Why 20.5 SOMETIMES is stored
> as 20.499999000?
>
>
> I'm puzzled by this and the solution. 20.5 should never experience
> any rounding by just declaring double var; var = 20.5.
>
> 20.5 = 40 34 80 00 00 00 00 00 , as stored in double format.
>
> 403 = + 16; 34 80 00 00 00 00 00 = .28125 the implied 1.0 makes this
> number 16 * 1.28125 = 20.5. But the point is that 20.5 can easily be
> saved in the 64 bit format without rounding.
>
> Where does the rounding come in, in a straight declaration and
> initialization?
>
>
> Date sent: Mon, 10 May 1999 12:30:17 -0400
> From: "Dave Lippincott" <[EMAIL PROTECTED]>
> Subject: Re: Double precision number problem. Why 20.5 SOMETIMES is
> stored as 20.499999000? To: <[EMAIL PROTECTED]> Send
> reply to: [EMAIL PROTECTED]
>
> > I had the same problem. Best answer I got was the Palm OS does not
> perform
> > rounding like other OSs. It displays the number sorted in memory,
> quantized
> > errors and all. I ended up injecting error into my calculations to
> > offset this problem. (i.e. I added 0.000001 to some variables in
> > critical calculations to get the correct results.
> >
> > D
> > -----Original Message-----
> > From: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
> > Date: Monday, May 10, 1999 11:34 AM
> > Subject: Double precision number problem. Why 20.5 SOMETIMES is stored
> > as 20.499999000?
> >
> >
> > >I am using CW 5.0 without the patch and PalmOS 3.0.
> > >
> > >when I compile the following code:
> > > double var ;
> > > var=20.5 ;
> > >
> > >during the debug, most of the time I see that var=20.4999990000, but
> > >sometimes the right value (20.5) is stored.
> > >
> > >What is causing this mismatch and how to prevent it?
> > >Is there any document where double float operations on PalmOS would be
> > >explained?
> > >
> > >FYI,
> > >I use 8-byte double setting of the Compiler
> > >I am not using NewFloatMgr
> > >
> > >
> > >Marcin Frelek
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
> --
> -----------------------------------------------------------------
> Discussion Group: http://www.halcyon.com/ipscone/wwwboard/
>
> Protect your constitutional rights. Your favorite one may be next!
> -----------------------------------------------------------------
>
>
>
>
>
--
-----------------------------------------------------------------
Discussion Group: http://www.halcyon.com/ipscone/wwwboard/
Protect your constitutional rights. Your favorite one may be next!
-----------------------------------------------------------------
