On Wednesday 17 August 2005 22:56, Jim McGowen wrote: > I wrote a test routine that multiplies and casts every number from 0.0001 > to 0.9999 and outputs to a text file via a conduit for analysis. I haven't > sifted through the whole set yet but so far I found these to be "trouble > numbers": > 0.141 - 0.143 > 0.172 - 0.174 > 0.204 > 0.284 - 0.286 > 0.344 - 0.349 > 0.407 - 0.409 > 0.563 - 0.573 > > I was thinking I might find a pattern and maybe special case these numbers > but I'm not smart enough to see a pattern here. [...]
There is a text "What Every Computer Scientist Should Know About Floating-Point Arithmetic" by David Goldberg somewhere in the web, but unfortunately I have lost the URL. It explains (among tons of other stuff) why certain numbers can't be represented in the expected precision: it has all to do with the binary system... Bodo -- For information on using the PalmSource Developer Forums, or to unsubscribe, please see http://www.palmos.com/dev/support/forums/
