The issue I find surprising is that integer literals in the range 16b8000000000000000 to 16b80000000000004ff, i.e. INT_MAX+1 to INT_MAX+1024, all parse into the integer precision 9223372036854775807, i.e. INT_MAX. This is not the same thing as (<:2^63) which has floating precision.
My expectation was that non-exact representations for (2^63x) and above--- e.g. 16b8000000000000000---would parse into floating precision numbers. As it stands, this means that integer precision values equal to 16b7fffffffffffffff cannot be trusted and must be handled specially, as demonstrated nicely in Raul's examples below. Henry Rich <[email protected]> wrote: > 16b8000000000000401 cannot be exactly represented. There is a band of floats > near 2^63 that are converted to (<:2^63). > > Henry Rich > > > > On 10/23/2021 1:42 AM, Raul Miller wrote: > > I do not see why I should expect these to be the same number: > > > > 2^.92233720368547768323 > > 66.3219 > > 2^.16b8000000000000401 > > 63 > > > > One is larger than can be represented using a 64 bit integer, and so > > > > datatype 92233720368547768323 > > floating > > > > That said, I do think that there's a problem with the other one. That > > one is also larger than can be represented using a 64 bit signed > > integer: > > > > (2^63x)-16b8000000000000401 > > 1 > > (2^63x)-16b8000000000000000 > > 1 > > (2^63x)-16b7fffffffffffffff > > 1 > > > > So there's a certain range of numbers here which get treated ... oddly. > > > > Thanks, > > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
