Hi Bano, Thanks!
There is a paper "Collocation Extraction Based on Modifiability Statistics" by Wermter and Hahn. I liked how they evaluated their measure. They used human judges to manually compile a test set and then calculated the precision and recall at 10\% increments to determine how well the algorithm performs. This is pretty much what we did, only the human judge was me. It is alot harder than it sounds to determine whether an ngram is a collocation :) Thanks, Bridget On Fri, 22 Oct 2004, Satanjeev Banerjee wrote: > Awesome, good luck Bridget - I'm sure it will be great! The abstract sounds > very interesting. BTW, just curious, how do you evaluate collocation > identification? Is there a paper on this I could look at? > > > > Good luck again! > > Bano > > > > _____ > > From: ted pedersen [mailto:[EMAIL PROTECTED] > Sent: Thursday, October 21, 2004 11:43 PM > To: NLP @ UMD > Subject: [nlpatumd] Bridget's title/abstract > > > > > Bridget's thesis defense will be at 3pm on Friday October 30. Here's the > title of her thesis and the associated abstract: > > ====================================== > > Extending the Log Likelihood Measure to Improve Collocation Identification > > Automatically identifying collocations in a text can be useful for > applications such as machine translation and building lexicons or > knowledge bases. This thesis presents an extension of the Log Likelihood > measure ($G^{2}$) to automatically identify collocations that consist of > more than two words. $G^{2}$ is the ratio between how often an ngram > occurs compared to how often it would be expected to occur given a model. > In the 2-dimensional case, i.e., collocations that consist of only two > words, the only possible model is that of independence. $G^{2}$ > calculates the observed count of an ngram and compares it to the count > that would be expected if the words were statistically independent. The > score that $G^{2}$ produces reflects the degree to which the observed > and expected values diverge. Calculating the expected values based on > independence is commonly carried over to the three dimensional case but > as the dimensions grow, so does the number of models in which the words > can be compared to. > > Our approach calculates the $G^{2}$ of an ngram for each of the different > possible models and iteratively determines what model `fits'' the ngram > the best. The score the models return allow us to rank the ngrams such > that ngrams that have a high Log Likelihood score are collocations while > ones with low Log Likelihood scores are not. > > To calculate the Log Likelihood measure, various co-occurrence and > individual frequency counts of the tokens in the ngram are needed. > Traditionally, the method used for obtaining frequency counts for ngrams > is to count the number of times they appear in a corpus and store them. > However, this becomes very limiting because of memory constraints that > make it infeasible to process large data sets. This problem has lead to > an increasing discussion on the feasibility of using the World Wide Web > as a corpus. We explore the use of document frequencies returned by the > search engines Google and Alta Vista as the term frequencies and > marginal counts of the ngrams in order to calculate ngram statistics. > > > > > > > Yahoo! Groups Sponsor > > > > ADVERTISEMENT > > <http://us.ard.yahoo.com/SIG=129aqfo4s/M=295196.4901138.6071305.3001176/D=gr > oups/S=1705082353:HM/EXP=1098543289/A=2128215/R=0/SIG=10se96mf6/*http:/compa > nion.yahoo.com> click here > > > > <http://us.adserver.yahoo.com/l?M=295196.4901138.6071305.3001176/D=groups/S= > :HM/A=2128215/rand=703389578> > > > > _____ > > Yahoo! Groups Links > > * To visit your group on the web, go to: > http://groups.yahoo.com/group/nlpatumd/ > > * To unsubscribe from this group, send an email to: > [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > * Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service > <http://docs.yahoo.com/info/terms/> . > > ------------------------ Yahoo! Groups Sponsor --------------------~--> $9.95 domain names from Yahoo!. Register anything. http://us.click.yahoo.com/J8kdrA/y20IAA/yQLSAA/x3XolB/TM --------------------------------------------------------------------~-> Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/nlpatumd/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
