There's a difference between GCC and MS Visual Studio 2008 Express.
GCC works fine. But Visual Studio shows this problem.
The difference comes in this section of code where the rounding error during
computation is different.
if( realvalue>0.0 ){
while( realvalue>=1e32 && exp<=350 ){ realvalue *= 1e-32; exp+=32; }
while( realvalue>=1e8 && exp<=350 ){ realvalue *= 1e-8; exp+=8; }
while( realvalue>=10.0 && exp<=350 ){ realvalue *= 0.1; exp++; }
while( realvalue<1e-8 ){ realvalue *= 1e8; exp-=8; }
while( realvalue<1.0 ){ realvalue *= 10.0; exp--; }
}
Then it makes the mistake here in rounding
if( xtype!=etFLOAT ){
realvalue += rounder;
if( realvalue>=10.0 ){ realvalue *= 0.1; exp++; }
}
The first section rounds different when running the two of the while loops
Gcc ends up with 8.9484710000000032
MS ends up with 8.9484710000000049
Then the rounder takes them to
Gcc: 8.9484710000000085
MS: 8.9484710000000103
So that's where your round error is. I assume you're using a Microsoft product
to compile?
I think the actual fix is to change the two %!.15g entries to %!.14g
It's a common misconception that double precision is 15 digits to the right of
the decimal point.
That's incorrect. It's 15 significant digits TOTAL regardless of the location
of the decimal point.
By using 15g you're actually showing 16 significant digits and double-precision
actually has 15.95 significant digits so you'll hit rounding errors with 16.
I find it interesting though that the math is different between the two
compilers...I wonder which is actually "correct" ??
Michael D. Black
Senior Scientist
Advanced Analytics Directorate
Northrop Grumman Information Systems
________________________________
From: sqlite-users-boun...@sqlite.org on behalf of Rick Regan
Sent: Sat 11/27/2010 11:09 AM
To: sqlite-users@sqlite.org
Subject: EXTERNAL:[sqlite] Some floats of 15 digits or less do not round-trip
I expected floats of 15 significant decimal digits or less to round-trip:
sqlite> create table t1(d float);
sqlite> insert into t1 values(8.948471e15);
sqlite> select * from t1;
8.94847100000001e+15
The double value stored in the database is correct (I traced the code and
printed the value of 'realvalue' in sqlite3VXPrintf()) -- it's
0x1.fca96433ce600p+52
= 11111110010101001011001000011001111001110011000000000 = 8948471000000000 =
8.948471e15. In other words, the original decimal to floating-point
conversion was done correctly, but the floating-point to decimal conversion
was not.
(This is different than the issue of bug
http://www.sqlite.org/src/tktview?name=1248e6cda8 .)
Rick Regan
--
Check out my blog: http://www.exploringbinary.com
<http://www.exploringbinary.com/>
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
