Yet again I found myself resorting to Pari GP for a calculation; my J
function had been giving
correct answers to a problem for lowish inputs, but apparently gave up
at some stage for
higher values; I then coded the calculation in Pari GP which gave the
same results for low
inputs, but diverged from J at the business end.
Looking for inconsistencies between the two functions, the divergence
seems to appear
around this case:
m =. 134235395
2^.m NB. plenty of room for multiplication in 64-bits???
27.0002
2!m
9009570568285316
datatype 2!m
integer
(-:<.@*<:)m
9009570568285316
((*<:)m)<.@%2
9009570568285315
_1 (33 b.) (*<:)m
9009570568285315
datatype (*<:)m
integer
So - why am I getting 2!m returned as integer but wrong? If there's
overflow,
why isn't it a float? Why does (-:<.@*<:)m return the wrong integer
when
((*<:)m)<.@%2 yields the correct integer?
This was in J.04,
Engine: j9.4.2/j64avx2/windows
Build: commercial/2023-04-10T01:19:53/clang-15-0-7/SLEEF=1
I haven't checked behaviour in earlier releases. I didn't try extended
integers
for this problem.
Thanks,
Mike
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
