> The only difficulty is to determine the cutoffs! As Seth Breidbart once told me, if it were easy they wouldn't pay us so much!
----- Original Message ----- From: "Sherlock, Ric" <[email protected]> Date: Monday, September 14, 2009 22:23 Subject: Re: [Jprogramming] Chi squared CDF definition unstable To: Programming forum <[email protected]> > That covers situations B & C, but A is still a problem AFAICS. > > One approach is a cover verb that checks and categorises the > various combinations of x and y into "OK", "A or B" and "C" > before calling chisqcdf for the "OK" group and amending the > array with 1s & 0s for the others. The only difficulty is to > determine the cutoffs! > > > From: Roger Hui > > In that case you can do chisqcdf :: > errhandler "0 > > where errhandler will return 0 or 1 as appropriate, > > after it has examined x and y . > > > > > From: "Sherlock, Ric" > > > > > > As I see it there are 3 specific situations that need attention > > > and chisqcdf :: 0: will only solve one of them. > > > > > > A. For some large test statistics where the answer should be 1, > > > 0 is returned instead. How large depends on the degrees of > freedom:> > Eg: > > > 9 chisqcdf 1415 + i.10 > > > 1 1 0 0 0 0 0 0 0 0 > > > > > > B. For even larger test statistics where the answer should > be 1, > > > a NaN error is returned instead. > > > Eg: > > > 9 chisqcdf 1471 > > > 0 > > > 9 chisqcdf 1472 > > > |NaN error: chisqcdf > > > | (1 H.(1+x)%x&((*^)*(^-)~))y > > > > > > C. For large degrees of freedom where the answer should be > 0, a > > > NaN error is returned instead. The number of degrees of freedom > > > is dependent on the size of the test statistic: > > > Eg: > > > 400 chisqcdf 73 + i.10 > > > 0 0 0 0 0 0 0 0 0 0 > > > 400 chisqcdf 83 > > > |NaN error: chisqcdf > > > | 400 chisqcdf 83 > > > > > > > > > > From: Roger Hui > > > > > > > > Would you be satisfied with chisqcdf :: 0: ? > > > > > > > > The values you gave that led to NaN errors are > > > > ridiculously large in the context of the chi-squared > distribution.> > > A&S http://www.math.sfu.ca/~cbm/aands/page_941.htm > > > > consider degrees of freedom > 30 to be "large". > > > > > > > > > > > > > > > > ----- Original Message ----- > > > > From: "Sherlock, Ric" <[email protected]> > > > > Date: Monday, September 14, 2009 18:20 > > > > Subject: [Jprogramming] Chi squared CDF definition unstable > > > > To: Programming forum <[email protected]> > > > > > > > > > The definition for the Chi squared Cumulative Distribution > > > > > Function given in the essay > > > > > http://www.jsoftware.com/jwiki/Essays/Chi%20Squared%20CDF > > > and in > > > > > the distribution.ijs script of the stats/base addon, seems > > > to be > > > > > unstable for large values. > > > > > For example: > > > > > > > > > > 9 chisqcdf 5 10 30 100 1000 1400 1420 1470 1471 > > > > > 0.16569174 0.64951479 0.99956128 1 1 1 0 0 0 > > > > > 9 chisqcdf 1472 > > > > > |NaN error: chisqcdf > > > > > | (1 H.(1+x)%x&((*^)*(^-)~))y > > > > > > > > > > Also: > > > > > 100 200 400 600 3545 chisqcdf 3 > > > > > 4.8195188e_57 9.8668347e_142 0 0 0 > > > > > 3546 chisqcdf 3 > > > > > |NaN error: chisqcdf > > > > > | 3546 chisqcdf 3 > > > > > > > > > > Has anyone come up with more robust implementations? ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
