Arie Groeneveld wrote:
Using dfh from convert.ijs
returns wrong values for hex strings
with length GE 14:
(TITS: for my 32bit version of J)
hfd x: dfh 13#'f'
FFFFFFFFFFFFF
but:
hfd x: dfh 14#'1'
11111111111112
hfd x: dfh 14#'f'
100000000000000
hfd dfh 21#'e'
|domain error: hfd
| hfd dfh 21#'e'
hfd x: dfh 21#'e'
EEEEEEEEEEEEF00000000
As far as I can see this minor change will
correct this:
dfh17=: 16 #. 16x | (H,h) i. ]
hfd dfh17 14#'1'
11111111111111
hfd dfh17 14#'f'
FFFFFFFFFFFFFF
hfd dfh17 21#'e'
EEEEEEEEEEEEEEEEEEEEE
This has the disadvantage that dfh would always return an extended
precision number, and I don't think that is desirable. I suppose it
could return normal precision where appropriate, and extended otherwise,
but where to draw the line? I am inclined to leave it as is, unless
there is a better suggestion.
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
