I do not see why I should expect these to be the same number: 2^.92233720368547768323 66.3219 2^.16b8000000000000401 63
One is larger than can be represented using a 64 bit integer, and so datatype 92233720368547768323 floating That said, I do think that there's a problem with the other one. That one is also larger than can be represented using a 64 bit signed integer: (2^63x)-16b8000000000000401 1 (2^63x)-16b8000000000000000 1 (2^63x)-16b7fffffffffffffff 1 So there's a certain range of numbers here which get treated ... oddly. Thanks, -- Raul On Fri, Oct 22, 2021 at 10:07 PM ethiejiesa via General <[email protected]> wrote: > > Look closer. The following two strings should represent the same number: > > 92233720368547768323 > 9.22337e19 > 16b8000000000000401 > 9223372036854775807 > > Worse, integer comparison is not guaranteed to be exact: > > (datatype;=/) 9223372036854776832 9223372036854775807 > ┌───────┬─┐ > │integer│1│ > └───────┴─┘ > > What am I missing? > > > Raul Miller <[email protected]> wrote: > > Yes.. if you want arbitrary precision numbers, suffix them with an 'x' > > > > 9223372036854776832 99999999999999999x > > 9223372036854776832 99999999999999999 > > > > I hope this helps, > > > > -- > > Raul > > > > On Fri, Oct 22, 2021 at 3:53 PM ethiejiesa via General > > <[email protected]> wrote: > > > > > > Split out from other post: > > > > > > 9223372036854776832 ; 9223372036854775808 > > > ┌───────────────────┬───────────────────┐ > > > │9223372036854775807│9223372036854775807│ > > > └───────────────────┴───────────────────┘ > > > > > > 9223372036854776833 ; 9223372036854775807 > > > ┌──────────┬───────────────────┐ > > > │9.22337e18│9223372036854775807│ > > > └──────────┴───────────────────┘ > > > > > > 9223372036854776832 99999999999999999 > > > 9223372036854775807 100000000000000000 > > > > > > 9223372036854776832 9999999999999998 > > > 9223372036854775807 9999999999999998 > > > > > > I'm seeing the same on both J903-beta-u and J902-b. Can others reproduce? > > > ---------------------------------------------------------------------- > > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
