Philip Warner <[EMAIL PROTECTED]> writes:
> pjw=# select float8(float4(10.1));
> float8
> ------------------
> 10.1000003814697
> (1 row)
> I would have expected the latter to be at worst 10.10000000000000 +/-
> .00000000000001.
float4 is good to about 7 decimal digits (24 mantissa bits) on
IEEE-standard machines. Thus the above result is actually closer
than you have any right to expect.
Don't they teach people about float arithmetic in CS 101 anymore?
regards, tom lane
- [GENERAL] Trouble with float4 after upgrading from 6.5.3... Romanenko Mikhail
- Re: [GENERAL] Trouble with float4 after upgrading f... Tom Lane
- Re: [HACKERS] Re: [GENERAL] Trouble with float4... Thomas Lockhart
- Re: [HACKERS] Re: [GENERAL] Trouble with float4... Philip Warner
- Re: [HACKERS] Re: [GENERAL] Trouble with fl... Thomas Lockhart
- Re: [HACKERS] Re: [GENERAL] Trouble wit... Philip Warner
- Re: [HACKERS] Re: [GENERAL] Trouble with fl... Tom Lane
- Re: [HACKERS] Re: [GENERAL] Trouble wit... Philip Warner
