Thanks to everyone for their feedback and suggestions! I originally came across the Float rounding functionality when looking at the Rails number_with_precision method (http://api.rubyonrails.org/classes/ActionView/Helpers/NumberHelper.html #M001687).
If you look at how the rounding is done, they use Floats which causes the rounding problems I saw: > rounded_number = (Float(number) * (10 ** precision)).round.to_f / 10 ** precision I wonder if this is intended? -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Stephen Waits Sent: Tuesday, December 15, 2009 10:09 PM To: [email protected] Subject: Re: [SDRuby] Ruby Float rounding behavior On Dec 14, 2009, at 2:43 PM, William Cassis wrote: Can anyone help explain the following Ruby (specifically ruby 1.9.1p243 (2009-07-16 revision 24175)) Float rounding behavior?: Here's a quick hack that might help you. I only guarantee two things about it. It is untested and buggy. :) The basic idea is to round it like a human would. That is, look at the last digit you care about. If it's >= 5, then round up. We'll use integer math for this. For example, 0.565.round(2): First we'll create an integer 565, then look at the last digit, which is 565%10, or 5. Since that's >= 5, add 10 to make it 575. Chop off the last digit since we're finished with it, giving us 57, then convert it back down to the float you care about 0.57. [~/tmp] 1013% irb -rbetter_round irb(main):001:0> 0.555.rounder(2) => 0.56 irb(main):002:0> 0.565.rounder(2) => 0.57 irb(main):003:0> 0.575.rounder(2) => 0.58 irb(main):004:0> 0.585.rounder(2) => 0.59 class Float def is_negative? self < 0.0 end def rounder(ndigits = 0) # convert to an int of just the digits we care about x = ( self.abs * 10.0**(ndigits + 1) ).to_i # look at the last digit, round if necessary x += 10 if ( x % 10 ) >= 5 # remove the last digit (which was only used to determine rounding) x /= 10 # convert back to a float, handle the negative case if is_negative? -x.to_f / 10.0**ndigits else x.to_f / 10.0**ndigits end end end -- Stephen Waits [email protected] WAR IS PEACE FREEDOM IS SLAVERY IGNORANCE IS STRENGTH 1984, George Orwell -- SD Ruby mailing list [email protected] http://groups.google.com/group/sdruby
