On 04/29/2015 08:56 AM, Rohit Amarnath wrote:
I have a SQLite table that is defined as:
DB.create_table :airport_codes do
column :airport_code, :text
column :airport_latitude, :real
column :airport_longitude, :real
end
The row stored is:
{:airport_code=>"DXB",
:airport_latitude=>25.25,
:airport_longitude=>55.366667
}
Row returned by Sequel is:
{:airport_code=>"DXB",
:airport_latitude=>25.25,
:airport_longitude=>55.366668701171875
}
How can I have it return exactly what is in the database without the
precision change?
Thanks for the help!
Rohit
--
You received this message because you are subscribed to the Google
Groups "sequel-talk" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected]
<mailto:[email protected]>.
To post to this group, send email to [email protected]
<mailto:[email protected]>.
Visit this group at http://groups.google.com/group/sequel-talk.
For more options, visit https://groups.google.com/d/optout.
This is a classic problem with floating point numbers. The number
55.366667 closest representation in a 32-bit float is
55.366668701171875. But it appears Ruby and SQLite are using 64-bit
floating points so I wouldn't expect this. Somewhere it is converted to
32-bit. But, even with 64-bit floats you can run into this problem its
just less likely. The wikipedia article has some explanation
http://en.wikipedia.org/wiki/Floating_point#Representable_numbers.2C_conversion_and_rounding
I am not able the reproduce the problem with the following:
require 'sequel'
DB = Sequel.sqlite
DB.create_table :airport_codes do
column :airport_code, :text
column :airport_latitude, :real
column :airport_longitude, :real
end
insert = {:airport_code=>"DXB",
:airport_latitude=>25.25,
:airport_longitude=>55.366667
}
DB[:airport_codes].insert(insert)
result = DB[:airport_codes].first
result[:airport_longitude] == insert[:airport_longitude]
result is: {:airport_code=>"DXB", :airport_latitude=>25.25,
:airport_longitude=>55.366667}
and the last expression is true.
I am using sqlite3 1.3.10, sequel 4.21.0
Quinn
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sequel-talk.
For more options, visit https://groups.google.com/d/optout.
