A user recently asked about the feasibility of adding a two tailed Fisher's exact test to NSP. Right now we support a left and right sided Fisher's test.
This turns out to be a fairly simple thing to, and we'll likely include that in the next version which is tentatively scheduled for release in mid December. Just a bit of quick review - Fisher's test is based on summing the probablity of some set of possible 2x2 tables given that the marginal counts of the observed 2x2 table are fixed. The left sided, right sided, and two sided version of the test do that summing in slightly different ways. The left sided test (leftFisher) is based on adding the probabilities of all the possible 2x2 contingency tables that have n11 values less than or equal to that of the observed value. In effect this tells us how likely it is to sample an observation where the two words represented in the bigram are less dependent/related than the currently observed value. The right sided test (rightFisher) does something similar in that it sums all the probabilities of the 2x2 contintency tables that have n11 values greater than or equal to what is currently observed. This tells us how likely it would be to sample an observation where the two words are more dependent than currently observed. Now, the two tailed test takes the sum of the probabilities of all the possible 2x2 tables that are less than the probability of the observed data. This tells us, I think, how likely it is to sample a bigram that is "more extreme" than what is currently observed. I am not sure how this should be interpreted in terms of finding collocations, but it's an interesting and useful test in general, and maybe there are some nice things that can be done with it. In any case, it strikes me as a minor modification to make this work, so we'll plan on including this in the next release. Questions or comments are of course welcome! Thanks, Ted -- Ted Pedersen http://www.d.umn.edu/~tpederse ------------------------ Yahoo! Groups Sponsor --------------------~--> Fair play? Video games influencing politics. Click and talk back! http://us.click.yahoo.com/eMf55D/tzNLAA/TtwFAA/dpFolB/TM --------------------------------------------------------------------~-> Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/ngram/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
