16b8000000000000401 cannot be exactly represented. There is a band of floats near
2^63 that are converted to (<:2^63).
Henry Rich
On 10/23/2021 1:42 AM, Raul Miller wrote:
I do not see why I should expect these to be the same number:
2^.92233720368547768323
66.3219
2^.16b8000000000000401
63
One is larger than can be represented using a 64 bit integer, and so
datatype 92233720368547768323
floating
That said, I do think that there's a problem with the other one. That
one is also larger than can be represented using a 64 bit signed
integer:
(2^63x)-16b8000000000000401
1
(2^63x)-16b8000000000000000
1
(2^63x)-16b7fffffffffffffff
1
So there's a certain range of numbers here which get treated ... oddly.
Thanks,
--
This email has been checked for viruses by AVG.
https://www.avg.com
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
