On Dec 14, 2009, at 2:43 PM, William Cassis wrote: > Can anyone help explain the following Ruby (specifically ruby > 1.9.1p243 > (2009-07-16 revision 24175)) Float rounding behavior?:
There are actually several accepted (and used) methods for rounding fractional numbers. In school we get taught that anyth ending with a 5 or higher gets rounded up, but that's not actually the method used by many financial, scientific, and computational models. I don't specifically know Ruby's method or whether it has any specific software override of what the processor itself does, but doing floating point math on a computer (esp a 32-bit computer) often does not follow the exact results you'd generate yourself on paper. If you need numbers like 0.575 to always yield 0.58, then you probably need to work with integers and handle the scaling and precision rounding yourself. -- greg willits -- SD Ruby mailing list [email protected] http://groups.google.com/group/sdruby
