Gu stav wrote:
> I ended up taking the chicken way out, converting my floats to strings
> in new columns and adapting my find conditions to look for those
> instead. Works like charm. Thanks anyways, I wonder why it didnt work.
As indicated by Frederick that's just the nature of a float data type.
It doesn't matter whether it's Ruby, SQL, Java or C. It is never safe to
compare equality on floating point values due to the inherent nature of
how they are stored.
Example: You may be seeing 11.967 but that value is going to get
converted to an IEEE floating point hexadecimal value. Depending on the
loss of precision it is quite possible to have 11.967 == 11.967 =>
false. Depending on how the two values get translated to and from the
IEEE floating point hexadecimal values. 11.967 might actually be stored
in the hex equivalent of 11.9669999999999.
There are a few techniques for dealing with this problem:
Given:
x = 11.967
y = 11.966999999999 <-- loss of precision due to hex<->decimal
conversion
1. Compare based on a range (as suggested in a prior post)
11.9670 <= x < 11.968.
2. Compare string representations of the floating point values:
x == y
=> false
("%.3f" % x) == ("%.3f" % y)
=> true
3. Store values using a fixed point data type (such as DECIMAL in
MySQL):
lng DECIMAL(5,3)
x -> lng
=> 11.967
y -> lng
=> 11.967 <-- rounded based on the rules defined in the database
Making it safe to compare equality on the two values
4. Store fixed decimal values as integers in the database and convert
them when displaying them. I only mention this technique because it is
an option for storing currency values. Rather than store dollars, store
cents instead and convert to dollars with displaying the values on the
view.
$1.60 -> 160 (in the database)
---------
In your case you have a slightly combined approach in that you are
storing the value in the database as a string. This technique will also
work as long as you ensure proper rounding from Float to String.
Option #3 above would be the approach I would personally take in this
situation. It allows storing numeric decimal values while preserving a
given precision.
More useful related information:
http://dev.mysql.com/doc/refman/5.1/en/precision-math.html
--
Posted via http://www.ruby-forum.com/.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Ruby
on Rails: Talk" group.
To post to this group, send email to rubyonrails-talk@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---
