it gets surprising/ineffective/unnapplicable if extended numbers are used:
T=: 2^_34x 2 = 2+T%2 0 2 =!.T 2+ 2^_44166x 0 Though to fix your issue: 2 =!.T 2+ 2^_35 1 2 =!.T 2+ 2^_33 1 2 =!.T 2+ 2^_32 0 ----- Original Message ----- From: Ian Clark <earthspo...@gmail.com> To: Programming forum <programm...@jsoftware.com> Cc: Sent: Wednesday, February 19, 2014 8:24:09 PM Subject: Re: [Jprogramming] Getting Fit (!.) to work Okay, I was reading that as *less* than 2^_34 All is not as it seems however... ] T=: 2^_34 5.82077e_11 2 =!.T 2+T 1 2 =!.T 2+T+T NB. I'd expect 0 here 1 2 =!.T 2+T+T+T 0 NB. at last! On Thu, Feb 20, 2014 at 12:37 AM, Henry Rich <henryhr...@nc.rr.com> wrote: > Comparison tolerance cannot be more than 2^_34. > > (So that integer comparisons on 32-bit machines do not have to worry about > tolerance) > > Henry Rich > > > > On 2/19/2014 7:03 PM, Ian Clark wrote: > >> I'm trying to construct an example using Fit (!.) to customize a given >> primitive (e.g. =) to set its tolerance. >> Try as I might I cannot get variants of this to work: >> >> T=: 0.001 >> 2 =!.T 2.0001 >> |domain error >> | 2 =!.T 2.0001 >> >> What am I doing wrong? >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm >> >> ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
