I think that comparison tolerance is the surprise.

z =: 281474976711954

datatype z

integer

datatype z%32

floating
  ((z%32)-<.z%32)%z%32

_4.9738e_14

9!:18 '' NB. comparison tolerance

5.68434e_14

<.!.0 z%32

8796093022248

<.(x: z)%32

8796093022248


<.!.0 or <.@:% "work" but only kick the can a few bits down the road as
z%32 is floating.

Safest it to use rationals or exact integers.

On Thu, Jul 29, 2021 at 3:02 AM Elijah Stone <[email protected]> wrote:

> On Thu, 29 Jul 2021, Ben Gorte wrote:
>
> > After
> >
> > z =: 281474976711954
> >
> > compare
> >
> > <.z%32
> > 8796093022249
> > and
> >
> > z <.@:% 32
> >
> > 8796093022248
>
> <.@:% is recognised as a special combination, see the wiki page:
>
> https://code.jsoftware.com/wiki/Vocabulary/SpecialCombinations#Integer_Operations
>
> I am not sure why that happens; it certainly seems incorrect.  The libc
> 'floor' function does not exhibit the same behaviour.  If precision is
> important, using rationals may be a temporary workaround.
> ----------------------------------------------------------------------
> For information about J forums see http://www.jsoftware.com/forums.htm
>
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm

Reply via email to