On 9/3/07, Doug Currie <[EMAIL PROTECTED]> wrote: > On Monday, September 03, 2007 Arjen Markus wrote: > > > Serena Lien wrote: > > >>round(98926650.5, 1) -> 98926650.5000001 > >>round(85227887.01, 1) -> 85227887.0000001 > > > They are in fact rounded, but the internal binary representation can > > not be turned into the appropriate decimal (and human readable) > > representation due to the finite precision. > > 98926650.5 is represented exactly in IEEE double; something else is > mucking up the round or the display of the result, maybe both. > > 85227887.0 is represented exactly in IEEE double, though 85227887.01 > is not. Nevertheless, there are ways to print floating point numbers > readably. http://portal.acm.org/citation.cfm?id=93559
This made me to remember there was a bug some time ago about the rounding algorithm (but can't remember at what version it was fixed), so I just tested it. "official" amalgamated sqlite 3.4.0 downloaded from the site some time ago: SQLite version 3.4.0 Enter ".help" for instructions sqlite> select round(98926650.5, 1) ; 98926650.5 sqlite> select round(85227887.01, 1) ; 85227887.0 sqlite> select round(85227887.01, 2) ; 85227887.01 sqlite> select round(98926650.50001, 1) ; 98926650.5 Linux [K]Ubuntu 7.04 (feisty) sqlite3 package 3.3.13-0ubuntu1: SQLite version 3.3.13 Enter ".help" for instructions sqlite> select round(98926650.5, 1); 98926650.5 sqlite> select round(85227887.01, 1); 85227887.0 sqlite> select round(85227887.01, 2); 85227887.01 sqlite> select round(98926650.50001, 1) ; 98926650.5 $ uname -a Linux ubuno 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC 2007 i686 GNU/Linux So it seems SQLite is already doing the right job. Maybe some OS specific error? Wasn't there some discussion earlier about the Microsoft compiler not using the full double precision by default? Regards, ~Nuno Lucas > e > > -- > Doug Currie > Londonderry, NH, USA ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
