Andreas Pflug <[EMAIL PROTECTED]> writes:
Well, to me it's not well-known that floating-point addition is not associative, do I need to re-learn my math?
regression=# select (1.0::float8 + (-1.0::float8)) + 1.0e-20::float8; ?column? ---------- 1e-20 (1 row)
regression=# select 1.0::float8 + ((-1.0::float8) + 1.0e-20::float8); ?column? ---------- 0 (1 row)
Hi Tom,
I already suspected an example like this. Obviously in a pure math world, the second example is wrong, caused by implicite rounding. Fortunately, if omitting the float8 casts numeric is used, delivering ultimate precision.
Just for curiousity: on MSSQL2000, the first will deliver 9.99999999999995E-21, and if the type decimal(30,25) is used both give 0.00000000000. Even better, CAST( CAST(1E-20 AS DECIMAL(30,25) AS FLOAT) is 0.0 :->
Oracle 9.2 will calculate correctly with float down to 1.0e-40.
Regards, Andreas
---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives?
http://archives.postgresql.org
