I hate to state the obvious, but 0.295 cannot be represented exactly in floating point. It depends on how the internal representation is rounded up or down to get close to 0.295. If it rounded down then any special logic to treat it as 0.295 instead of the actual internal number could lead to other rounding problems. ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
- Re: [Jprogramming] dyad ": rounding error Don Guinn
- Re: [Jprogramming] dyad ": rounding error bill lam
- Re: [Jprogramming] dyad ": rounding error Devon McCormick
- Re: [Jprogramming] dyad ": rounding error Devon McCormick
- Re: [Jprogramming] dyad ": rounding error Dan Bron
- Re: [Jprogramming] dyad ": rounding error Robert P. Rumble
- Re: [Jprogramming] dyad ": rounding error bill lam
- Re: [Jprogramming] dyad ": rounding error Joey K Tuttle
- Re: [Jprogramming] dyad ": rounding err... John Randall
