On 25/08/2013 19:50, Florian Weimer wrote:
I noticed this:
Poly/ML 5.5.1 Development
val a = 0.99999999999999972244424384371086489409208297729492187;;
val a = 1.0: real
val b = 0.999999999999999722444243843710864894092082977294921876;;
val b = 1.0: real
a < b;;
val it = false: bool
a > b;;
val it = false: bool
So both values are equal according to Poly/ML. But the correct
conversion for the first one is (in hexadecimal floating point
notation) 0x1.ffffffffffffdp-1, and for the second
0x1.ffffffffffffep-1, so the values should be different.
Thanks for reporting that. I find IEEE arithmetic a bit of a mystery so
I'll take your word for the fact that there's an error. Conversion
between real and string is done using a slightly modified version of
David Gray's open source dtoa code. I've downloaded a newer version and
this appears to produce results in line with what you're saying is correct.
> val a = 0.99999999999999972244424384371086489409208297729492187;;
val a = 1.0: real
> val b = 0.999999999999999722444243843710864894092082977294921876;;
val b = 1.0: real
> a < b;
val it = true: bool
If you can check out the current SVN (1838) that would be helpful.
David
_______________________________________________
polyml mailing list
[email protected]
http://lists.inf.ed.ac.uk/mailman/listinfo/polyml
