I stand corrected.. I thought I recalled that the IEEE for double precision offered 18 digits of accuracy (been years since I looked at it) but a little research shows me 15.
- michael dykman On Wed, 2005-02-02 at 14:02, Roger Baklund wrote: > Michael Dykman wrote: > [...] > > The MySQL implementation also supports this optional > > precision specification, but the precision value is used only to > > determine storage size. > > Right. This means you can not have 15 decimals precision using DOUBLE: > > mysql> use test > Database changed > mysql> create table dtest(d double(18,15)); > Query OK, 0 rows affected (0.01 sec) > > mysql> insert into dtest values (6.984789027653891),(39.484789039053891); > Query OK, 2 rows affected (0.02 sec) > Records: 2 Duplicates: 0 Warnings: 0 > > mysql> select * from dtest; > +--------------------+ > | d | > +--------------------+ > | 6.984789027653892 | > | 39.484789039053894 | > +--------------------+ > 2 rows in set (0.00 sec) > > Last digit is "wrong" in both test rows. Increasing precision does not help: > > mysql> create table d2test(d double(18,16)); > Query OK, 0 rows affected (0.02 sec) > > mysql> insert into d2test values (6.984789027653891),(39.484789039053891); > Query OK, 2 rows affected (0.00 sec) > Records: 2 Duplicates: 0 Warnings: 0 > > mysql> select * from d2test; > +---------------------+ > | d | > +---------------------+ > | 6.9847890276538909 | > | 39.4847890390538940 | > +---------------------+ > 2 rows in set (0.00 sec) > > This is no error, it is the approximate data type at work... it simply > can not store the exact value. -- - michael dykman - [EMAIL PROTECTED] -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe: http://lists.mysql.com/[EMAIL PROTECTED]
