Olaf Schmidt <[EMAIL PROTECTED]> wrote: > > select round(0.95, 1) > gives 0 - not 0.9 (in case rounding down) and > also not 1.0 (in case rounding up) >
I can reproduce the problem by running sqlite3.exe on windows. But I cannot explain it. The problem occurs in printf.c on line 454: if( xtype==etFLOAT ) realvalue += rounder; Going into this statement on both Linux and Windows, the exact same values are in realvalue and in rounder: realvalue: 0xf333333333333000 0.94999999999999995559107901499373838 rounder: 0x3fa999999999999a 0.05000000000000000300000000000000000 Both machines execute the exact same 4 ix86 opcodes: fldl <address of rounder> fldt <address of realvalue> faddp %st,%st(1) fstpt <address of realvalue> But they end up with different answers. Linux gets what I believe is the correct answer: realvalue: 0xfffffffffffffd00 0.99999999999999995836663657655662973 Windows gets the wrong answer: realvalue: 0x8000000000000000 1.00000000000000000000000000000000000 Can somebody please suggest to me what the difference is? Does windows somehow initialize the floating point hardware differently so that it does some kind of rounding on its own? Why should the same machine-language instructions generate a different answer on windows than it does on linux? -- D. Richard Hipp <[EMAIL PROTECTED]> ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
