Ah, so it appears there is a very low global upper bound (2^_34) on the tolerance of all verbs that can use Fit.
Perhaps this upper bound could be removed, at least for x:, since I believe the main use cases for tolerance on this verb would be bigger values, as illustrated in my earlier example. At least in this context, I can't see anything catastrophic happening by increasing this tolerance, on the contrary. But maybe this change cannot be made in such an isolated way. Thanks. ________________________________ From: Programming <programming-boun...@forums.jsoftware.com> on behalf of Raul Miller <rauldmil...@gmail.com> Sent: Thursday, August 1, 2019 11:12 AM To: Programming forum <programm...@jsoftware.com> Subject: Re: [Jprogramming] Extended precision tolerance Looks like this specifies comparison tolerance. https://code.jsoftware.com/wiki/Essays/Tolerant_Comparison "Appendix C, System Conventions and Limits, says that the tolerance is less than or equal to 2^_34 . Historically, the upper bound on the tolerance was chosen so that comparisons involving 32-bit integers are exact (as if the tolerance is 0)." FYI, -- Raul On Thu, Aug 1, 2019 at 9:33 AM Nimp O <tr...@outlook.com> wrote: > > Hello everyone, > > What is the use of non-zero tolerance for the x: verb? > > A little example: > > a=:0.1667 0.8333 0.4286 0.1111 > > x:a > 1667r10000 8333r10000 2143r5000 1111r10000 > > > In Mathematica: > In[1]:= Rationalize[{0.1667, 0.8333, 0.4286, 0.1111}] > Out[1]= {1667/10000, 8333/10000, 2143/5000, 1111/10000} > > In[2]:= Rationalize[{0.1667, 0.8333, 0.4286, 0.1111}, 0.01] > Out[2]= {1/6, 5/6, 3/7, 1/9} > > Back to J: > I was expecting the that Fit could be used the same way, as in a tolerance > for the rationalization, but I get: > > x:(!.0.01) a > |domain error > | x: (!.0.01)a > > I was waiting for this expression to return my desired output: > 1r6 5r6 3r7 1r9 > > Thanks. > > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
