Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Tue, Nov 30, 2010 at 12:29 AM, Shane Harrelson wrote: > > I've written two articles about this topic that summarize the problem and how it may be fixed: - http://www.exploringbinary.com/fifteen-digits-dont-round-trip-through-sqlite-reals/ This covers the original

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Tue, Nov 30, 2010 at 12:29 AM, Shane Harrelson wrote: > It turns out the same problem exists on Linux (I used sqlite3-3.7.3.bin); for example: sqlite> create table t1(d float); sqlite> insert into t1 values(9.87e+31); sqlite> select * from t1; 9.870001e+31 So it

Re: [sqlite] Some floats of 15 digits or less do not round-trip

In Visual C, a "long double" and a "double" are the same, and only offer 53 bits of precision. On GCC, a "long double" has 80 bits of precision. Unfortunately, I don't think there's a way to have Visual C use more precision. Because of this, round off error will always differ between the two

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Mon, Nov 29, 2010 at 3:04 PM, Rick Regan wrote: > So the question remains -- why does the Windows build get it wrong? > I think I figured it out. It is due to the use of extended precision. In sqlite3.c, if you change "#define LONGDOUBLE_TYPE long double" to

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Sun, Nov 28, 2010 at 8:55 AM, Black, Michael (IS) wrote: > > I find it interesting though that the math is different between the two > compilers...I wonder which is actually "correct" ?? > > > I ran a GMP version of the section of code we've highlighted, converting

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Mon, Nov 29, 2010 at 10:08 AM, Doug Currie wrote: > > The Grisu2 strings do "round trip" correctly, which I think is where this > thread started. > > Grisu addresses the "floating-point to decimal to floating-point" round-trip. The bug I found (it will become a bug

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Nov 29, 2010, at 9:37 AM, Rick Regan wrote: > "For IEEE 754 double-precision numbers and 64-bit integers roughly > 99.4% of all numbers can be processed efficiently. The remaining 0.6% are > rejected and need to be printed by a slower complete algorithm." > > Hmmm. What's involved in the

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Sun, Nov 28, 2010 at 8:09 PM, Doug Currie wrote: > The paper compares the performance of sprintf from glibc 2.11 and Grisu. I > don't know if glibc sprintf is based on Gay's code; at one point I thought > it was, but I cannot find an authoritative reference. > > In any

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Nov 28, 2010, at 6:19 PM, Rick Regan wrote: > On Sun, Nov 28, 2010 at 5:52 PM, Doug Currie wrote: > >> >> There is a new publication on this subject that may be of interest to those >> looking at providing solutions: >> >>

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Sun, Nov 28, 2010 at 5:52 PM, Doug Currie wrote: > > There is a new publication on this subject that may be of interest to those > looking at providing solutions: > > http://portal.acm.org/citation.cfm?id=1806623 > > It (Grisu2) works without bignums if you are willing

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Nov 28, 2010, at 5:37 PM, Rick Regan wrote: > On Sun, Nov 28, 2010 at 4:01 PM, Doug Currie wrote: > >> On Nov 28, 2010, at 11:18 AM, Rick Regan wrote: >> >>> Michael, >>> Thanks for the very thorough analysis. >> >> This is a difficult problem; fortunately it was

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Sun, Nov 28, 2010 at 4:01 PM, Doug Currie wrote: > On Nov 28, 2010, at 11:18 AM, Rick Regan wrote: > > > Michael, > > Thanks for the very thorough analysis. > > This is a difficult problem; fortunately it was solved 20 years ago... > > Well, it's not solved on Windows.

Re: [sqlite] Some floats of 15 digits or less do not round-trip

On Nov 28, 2010, at 11:18 AM, Rick Regan wrote: > Michael, > Thanks for the very thorough analysis. This is a difficult problem; fortunately it was solved 20 years ago... http://www.mail-archive.com/sqlite-users@sqlite.org/msg09529.html e ___

Re: [sqlite] Some floats of 15 digits or less do not round-trip

BTW, I get the CORRECT result when running the precompiled command line on Linux: SQLite version 3.7.3 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> create table t1(d float); sqlite> insert into t1 values(8.948471e15); sqlite> select * from t1; 8.948471e+15

Re: [sqlite] Some floats of 15 digits or less do not round-trip

Michael, Thanks for the very thorough analysis. Yes, sorry, I forgot to state I was running on Windows. The results I got came from running the command window (the precompiled Windows binary sqlite-3_7_3.zip). I also got the same results from compiling the amalgamation

Re: [sqlite] Some floats of 15 digits or less do not round-trip

There's a difference between GCC and MS Visual Studio 2008 Express. GCC works fine. But Visual Studio shows this problem. The difference comes in this section of code where the rounding error during computation is different. if( realvalue>0.0 ){ while( realvalue>=1e32 && exp<=350 ){

[sqlite] Some floats of 15 digits or less do not round-trip

I expected floats of 15 significant decimal digits or less to round-trip: sqlite> create table t1(d float); sqlite> insert into t1 values(8.948471e15); sqlite> select * from t1; 8.9484710001e+15 The double value stored in the database is correct (I traced the code and printed the value of