On Fri, 26 Dec 2003 05:23:58, Dave Mottorn writes: >I have my own conversion routines to get decimal strings to floating point >numbers and they seem to work. However when I add the resulting floating >point numbers I get errors. I looked at a few additions in the debugger and >it's definitely doing it wrong. For example when I add 52333.08 to 23821.71 >I get 76154.80 instead of 76154.79. It gets an answer that's a penny high >in the cases I've seen and it's fairly frequent. I have a little report >with about 15 subtotals on it and two of them are a penny high.
In the case above, I would check the correctness of your number conversion routines. They appear not to work correctly, unlike your initial presumption (at least as far as correct rounding is concerned). I general, if you want results exact to the penny, you should calculate in pennys, not dollars. Then you won't get any of the fractional rounding errors implicit in the binary/decimal fp conversion process if you are only doing addition/subtraction operations without mantissa overflow. IMHO. YMMV. Ron Nicholson HotPaw Productions http://www.hotpaw.com/rhn/palm -- For information on using the Palm Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
