As (x!y) is coded, the calculation is done in floating-point and then
converted to integer if the result will fit. Loss of significance
during the calculation will make the result inaccurate.
I think it's a JE error to return an integer value when that value might
be wrong. Unfortunately, the way the internal interfaces are, it's
difficult to leave the value as floating-point, so you cannot use the
fact that an integer was returned as a guarantee of accuracy.
Henry Rich
On 4/13/2023 11:34 AM, 'Michael Day' via Programming wrote:
Yet again I found myself resorting to Pari GP for a calculation; my J
function had been giving
correct answers to a problem for lowish inputs, but apparently gave
up at some stage for
higher values; I then coded the calculation in Pari GP which gave the
same results for low
inputs, but diverged from J at the business end.
Looking for inconsistencies between the two functions, the divergence
seems to appear
around this case:
m =. 134235395
2^.m NB. plenty of room for multiplication in 64-bits???
27.0002
2!m
9009570568285316
datatype 2!m
integer
(-:<.@*<:)m
9009570568285316
((*<:)m)<.@%2
9009570568285315
_1 (33 b.) (*<:)m
9009570568285315
datatype (*<:)m
integer
So - why am I getting 2!m returned as integer but wrong? If there's
overflow,
why isn't it a float? Why does (-:<.@*<:)m return the wrong
integer when
((*<:)m)<.@%2 yields the correct integer?
This was in J.04,
Engine: j9.4.2/j64avx2/windows
Build: commercial/2023-04-10T01:19:53/clang-15-0-7/SLEEF=1
I haven't checked behaviour in earlier releases. I didn't try
extended integers
for this problem.
Thanks,
Mike
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
