https://bugs.documentfoundation.org/show_bug.cgi?id=138360
--- Comment #4 from Eike Rathke <[email protected]> ---
(In reply to b. from comment #2)
> 0,5 ULP precision at
> granularity borders ... produces overlapping 'definition ranges'? as e.g.
> 0,5 ULP of 2^53 reaches 'down' to including '2^53 - 1'
Say what? I don't understand what you are trying to tell.
> while 2^53-1 itself
> covers the range from 2^53-1-0,5 to 2^53-1+0,4999~,
It does not cover a range. In an IEEE-754 binary64 magnitude with a distance
precision of 1, i.e. values in the interval [2^52,2^53) are integers, any value
or result that would lie in between integer values is tied to the nearest
representable value. There is no range between two integer values in that
interval.
Btw, it would help people and specifically developers reading (and maybe
trying) your examples if you used '.' decimal point instead of ',' decimal
comma.
> that leads to some
> ambiguity as e.g. 2^53-1+0,4 is falling into both 'ULP-precision-ranges', is
> that handeled in a meaningful way?
What both "ULP-precision-ranges"? 2^53-1+0.4 results in 2^53-1 as that is the
nearest representable value. There is no ambiguity.
--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Libreoffice-bugs mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/libreoffice-bugs
