2010/11/7 Phil Steitz <phil.ste...@gmail.com>: > Switching to the right list... > > - >>>> >>>> What we need there is a good algorithm for approximating the KS >>>> distribution. I have been corresponding with the author of a very good >>>> one >>>> with a Java implementation but have thus far failed in getting consent >>>> to >>>> release under ASL. So at this point, I am looking for an alternative >>>> good >>>> algorithm to implement. All suggestions / unencumbered patches welcome! >>>> >>>> See comments on the MATH-431 for other questions. >>>> >>> Just to be sure of what you mean: >>> Do you want to have a two-sample Kolmogorov-Smirnov test for equality >>> of distributions in addition to the Mann-Whitney? Or do you need the >>> Kolmogorov-Smirnov distribution (as stated for example at >>> >>> >>> http://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test#Kolmogorov_distribution >>> ) in regards to the MATH-428? Sorry, but I'm at bit confused :-). >> >> The goal is to implement the KS test for equality of distributions (or >> homogeneity against a reference distribution). To do that we need at >> least >> critical values of the Kolmogorov distribution. The natural way for us to >> do that would be to implement the full distribution which would be nice to >> have in the distributions package. >> >> Phil > > Have you read "Evaluating Kolmogorov’s Distribution" by Marsaglia et > al. available on http://www.jstatsoft.org/v08/i18/paper ? And do you > think their approach would be the way to go? > > I am not sure it is best. See the comments here: > http://www.iro.umontreal.ca/~lecuyer/myftp/papers/ksdist.pdf > > Phil Thanks. It looks quite thorough, indeed. Was it the Java implementation you didn't get a consent to release under ASL? > > >>>>>> >>>>>> Interesting approach for the exact algorithm for Wilcoxon. If we stay >>>>>> with this, we should ack the original author of the algorithm in the >>>>>> javadoc. Looks OK to use. >>>>> >>>>> Agree - both on the approach and legal part! Does the author need to >>>>> sign anything but write a mail? >>>>>> >>>>>> Regarding the difference from R, what I usually do in this case is >>>>>> look >>>>>> at the R sources to try to explain the difference. Most likely in >>>>>> this >>>>>> case, what is going on is they are using a different estimation >>>>>> algorithm >>>>>> for small n or treating ties differently. The ranking options that we >>>>>> use >>>>>> were largely adapted from R, so if that is the problem, it should be >>>>>> easy to >>>>>> test. We need to convince ourselves that ours is better or at least a >>>>>> legitimate alternative. I will take a close look this evening, but it >>>>>> looks >>>>>> like the algorithm you are using should be exact. If we can't >>>>>> reconcile the >>>>>> difference with R, it would be good to find a way to validate correct >>>>>> functioning of the algorithm by manufacturing reference data with >>>>>> known >>>>>> p. >>>>> >>>>> I'll try to investigate the difference, hopefully tomorrow, so that >>>>> formal tests can be written and included. >>>>>> >>>>>>> New tests: Wilcoxon signed-rank test and Mann-Whitney U >>>>>>> ------------------------------------------------------- >>>>>>> >>>>>>> Key: MATH-431 >>>>>>> URL: https://issues.apache.org/jira/browse/MATH-431 >>>>>>> Project: Commons Math >>>>>>> Issue Type: New Feature >>>>>>> Reporter: Mikkel Meyer Andersen >>>>>>> Assignee: Mikkel Meyer Andersen >>>>>>> Priority: Minor >>>>>>> Attachments: MannWhitneyUTest.java, MannWhitneyUTestImpl.java, >>>>>>> WilcoxonSignedRankTest.java, WilcoxonSignedRankTestImpl.java >>>>>>> >>>>>>> Original Estimate: 4h >>>>>>> Remaining Estimate: 4h >>>>>>> >>>>>>> Wilcoxon signed-rank test and Mann-Whitney U are commonly used >>>>>>> non-parametric statistical hypothesis tests (e.g. instead of various >>>>>>> t-tests >>>>>>> when normality is not present). >>>>>> >>>>>> -- >>>>>> This message is automatically generated by JIRA. >>>>>> - >>>>>> You can reply to this email to add a comment to the issue online. >>>>>> >>>>>> >>>> >>>> >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org > For additional commands, e-mail: dev-h...@commons.apache.org > >
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org For additional commands, e-mail: dev-h...@commons.apache.org
