Hi Mercè, Would you try using Text-NSP 1.27 and see if the newest version is working for you? That is the only difference I am seeing. I have run this on a number of different OS: ubuntu, debian and mint. I am not able to reproduce your error.
The only other thing I can think of is there is something with your file that is not coming across with a copy and paste. If you send me your file actual file through an attachment I can see. I attached mine (test.4). Hopefully this will tell us something. Thanks, Bridget On Wed, Apr 3, 2013 at 4:19 AM, mercevg <merc...@yahoo.es> wrote: > ** > > > Hi Bridget, > > I am using Text-NSP-1.25, Perl 5.10.1 and GNU/Linux. > > Thanks for your help! > Mercè > > --- In ngram@yahoogroups.com, Bridget McInnes <btmcinnes@...> wrote: > > > > Hi Mercè, > > > > That is puzzling. I get the following, which really doesn't help you > much. > > > > 3214 > > de<>recuperación<>de<>información<>1 1531.9192 8 274 17 286 33 8 18 9 17 > 16 > > 27 8 8 8 16 > > la<>recuperación<>de<>información<>2 1266.9553 7 115 17 286 33 8 54 11 17 > > 16 27 8 7 11 16 > > recuperación<>de<>información<>y<>3 1009.9596 5 15 287 30 63 15 14 5 25 > 11 > > 5 14 5 5 5 > > el<>procesamiento<>del<>lenguaje<>4 610.5006 6 95 19 57 22 11 9 6 10 9 > 20 6 > > 6 6 9 > > procesamiento<>del<>lenguaje<>natural<>5 521.5067 9 19 55 22 19 10 9 9 20 > > 16 18 9 9 9 16 > > > > What version of Text-NSP are you using? and I guess also what version of > > perl? And your OS? Maybe if we can see the difference between your system > > and mine we can track down the error. > > > > I am using: Text-NSP-1.27; Perl 5.10.1; ubuntu. Do you have similar > > versions, especially with NSP? > > > > In the mean time, I will take a look at the code that the error is being > > generated at to see if something comes to light. > > > > Thanks, > > > > Bridget > > > > On Wed, Mar 27, 2013 at 2:34 PM, mercevg <mercevg@...> wrote: > > > > > ** > > > > > > > > > Hi Bridget, > > > > > > I've been doing the same process as you, but the error continues to > occur. > > > > > > My test.4 file contains: > > > > > > 3214 > > > procesamiento<>del<>lenguaje<>natural<>9 19 55 22 19 10 9 9 20 16 18 9 > 9 9 > > > 16 > > > de<>recuperación<>de<>información<>8 274 17 286 33 8 18 9 17 16 27 8 8 > 8 > > > 16 > > > la<>recuperación<>de<>información<>7 115 17 286 33 8 54 11 17 16 27 8 > 7 11 > > > 16 > > > el<>procesamiento<>del<>lenguaje<>6 95 19 57 22 11 9 6 10 9 20 6 6 6 9 > > > recuperación<>de<>información<>y<>5 15 287 30 63 15 14 5 25 11 5 14 5 > 5 5 > > > > > > Then I run the Log Likelihood for 4-grams > > > statistic.pl --ngram 4 ll test.4ll test.4 > > > > > > And this is the error message: > > > Use of uninitialized value $Text::NSP::Measures::4D::expected_values in > > > string eq at /etc/perl/Text/NSP/Measures/4D.pm line 869, <SRC> line > 816.^C > > > > > > Thank you for your help! > > > > > > Mercè > > > > > > --- In ngram@yahoogroups.com, Bridget McInnes <btmcinnes@> wrote: > > > > > > > > Hi Mercè, > > > > > > > > Would you send me your file? I am not able to reproduce the error. I > > > > apologize if you already sent it. I am not seeing it in the thread. > > > > > > > > I put what I did to test it below so you could reproduce what I have > done > > > > on an example test set. > > > > > > > > I will check on : > > > > In the folder "MyNSP/man/man3" I've got > "Text::NSP::Measures::4D::MI:: > > > > ll.3pm" > > > > > > > > There shouldn't be a ll.3pm in 4D. I must have something wrong in > there. > > > > > > > > Thanks, > > > > > > > > Bridget > > > > ------------- > > > > > > > > Here is what I am doing: > > > > > > > > The text file contains the following: > > > > this is a test sentence > > > > just a sentence > > > > this is a test sentence > > > > > > > > I save that to a file called test.txt. > > > > > > > > Then I run the following: > > > > bridget@atlas:~/nsp-test$ count.pl --ngram 4 test.4 test.txt > > > > > > > > The test.4 file contains: > > > > 10 > > > > this<>is<>a<>test<>2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 > > > > is<>a<>test<>sentence<>2 2 3 2 3 2 2 2 2 2 2 2 2 2 2 > > > > sentence<>this<>is<>a<>1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 > > > > test<>sentence<>just<>a<>1 1 2 1 2 1 1 1 1 1 1 1 1 1 1 > > > > just<>a<>sentence<>this<>1 1 3 2 1 1 1 1 1 1 1 1 1 1 1 > > > > a<>sentence<>this<>is<>1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 > > > > a<>test<>sentence<>just<>1 2 1 2 1 1 1 1 1 1 1 1 1 1 1 > > > > sentence<>just<>a<>sentence<>1 2 1 3 3 1 1 1 1 1 1 1 1 1 1 > > > > > > > > Then I run the Log Likelihood for 4-grams over it: > > > > bridget@atlas:~/nsp-test$ statistic.pl --ngram 4 ll test.4ll test.4 > > > > > > > > Please note here that input file is the count.pl file generated > from the > > > > above step (test.4). This may be the cause of the error. > > > > > > > > The test.4ll contains: > > > > 10 > > > > is<>a<>test<>sentence<>1 29.8708 2 2 3 2 3 2 2 2 2 2 2 2 2 2 2 > > > > this<>is<>a<>test<>2 29.6804 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 > > > > sentence<>just<>a<>sentence<>3 27.3805 1 2 1 3 3 1 1 1 1 1 1 1 1 1 1 > > > > just<>a<>sentence<>this<>4 22.4273 1 1 3 2 1 1 1 1 1 1 1 1 1 1 1 > > > > sentence<>this<>is<>a<>5 19.9354 1 2 1 1 2 1 1 1 1 1 1 1 1 1 1 > > > > test<>sentence<>just<>a<>5 19.9354 1 1 2 1 2 1 1 1 1 1 1 1 1 1 1 > > > > a<>sentence<>this<>is<>5 19.9354 1 2 2 1 1 1 1 1 1 1 1 1 1 1 1 > > > > a<>test<>sentence<>just<>5 19.9354 1 2 1 2 1 1 1 1 1 1 1 1 1 1 1 > > > > > > > > Let me know if you get anything different. > > > > > > > > > > > > On Wed, Mar 27, 2013 at 11:45 AM, mercevg <mercevg@> wrote: > > > > > > > > > ** > > > > > > > > > > > > > > > Ted, > > > > > > > > > > Changing the command line the following error message appears: > > > > > > > > > > "Use of uninitialized value > $Text::NSP::Measures::4D::expected_values > > > in > > > > > string eq at /etc/perl/Text/NSP/Measures/4D.pm line 839, <SRC> line > > > 1265.^C" > > > > > > > > > > It could be due to files already installed? > > > > > In the folder "MyNSP/man/man3" I've got > > > > > "Text::NSP::Measures::4D::MI::ll.3pm" > > > > > > > > > > Thank you, > > > > > Mercè > > > > > > > > > > --- In ngram@yahoogroups.com, Ted Pedersen <tpederse@> wrote: > > > > > > > > > > > > I think there is a slight typo in your command : > > > > > > > > > > > > statistic.pl --ngram 4 ll4.pm output.txt intput.txt > > > > > > > > > > > > (the module name should be ll4.pm) > > > > > > > > > > > > I hope this helps! Let me know if you continue to have any > trouble... > > > > > > > > > > > > Good luck, > > > > > > Ted > > > > > > > > > > > > On Wed, Mar 27, 2013 at 9:06 AM, mercevg <mercevg@> wrote: > > > > > > > Ted, > > > > > > > > > > > > > > I've received your answer without problem. I'll try to follow > up > > > with > > > > > another email address. > > > > > > > > > > > > > > A sample of my 4-grams file: > > > > > > > procesamiento<>del<>lenguaje<>natural<>9 19 55 22 19 10 9 9 20 > 16 > > > 18 9 > > > > > 9 9 16 > > > > > > > recuperación<>de<>información<>textual<>4 15 287 30 5 15 14 4 > 25 4 > > > 5 > > > > > 14 4 4 4 > > > > > > > estadístico<>del<>lenguaje<>natural<>3 5 55 22 19 3 3 3 20 16 > 18 3 > > > 3 3 > > > > > 16 > > > > > > > aparición<>en<>el<>documento<>2 4 93 95 22 3 2 3 18 6 4 2 3 2 3 > > > > > > > > > > > > > > Command line: > > > > > > > statistic.pl --ngram 4 ll.3pm 4-grams-ll.txt 4-grams.txt > > > > > > > > > > > > > > Program answer: > > > > > > > "Measure not defined for 4-grams" > > > > > > > > > > > > > > I've got Text-NSP v.1.25. > > > > > > > > > > > > > > Thank you. > > > > > > > Mercè > > > > > > > > > > > > > > --- In ngram@yahoogroups.com, Ted Pedersen <tpederse@> wrote: > > > > > > >> > > > > > > >> Merce, I got an email error when responding directly to your > > > yahoo.es > > > > > > >> account. Could you follow up with another email address or > use the > > > > > > >> group...? > > > > > > >> > > > > > > >> Thanks, > > > > > > >> Ted > > > > > > >> > > > > > > >> > > > > > > >> ---------- Forwarded message ---------- > > > > > > >> From: Ted Pedersen <tpederse@> > > > > > > >> Date: Wed, Mar 27, 2013 at 8:29 AM > > > > > > >> Subject: Re: ll4 giving me trouble with 4-grams > > > > > > >> To: mercevg <mercevg@> > > > > > > >> > > > > > > >> > > > > > > >> Hi Merce, > > > > > > >> > > > > > > >> Could you send me whatever error output you are getting, plus > a > > > small > > > > > > >> sample of your ngram file? > > > > > > >> > > > > > > >> Thanks! > > > > > > >> Ted > > > > > > >> > > > > > > >> On Wed, Mar 27, 2013 at 8:12 AM, mercevg <mercevg@> wrote: > > > > > > >> > Hi, > > > > > > >> > > > > > > > >> > I would like to know how to calculate with Statistical.pl > > > 4-grams > > > > > using log-likelihood ratio. > > > > > > >> > > > > > > > >> > To calculate 3-grams I've run the program as follows: > > > > > > >> > statistic.pl --ngram 3 tmi3.pm three.ngram.tmi3 three.ngram > > > > > > >> > > > > > > > >> > But using log-likelihood ratio it doesn't work. > > > > > > >> > > > > > > > >> > Thanks > > > > > > >> > > > > > > > >> > Mercè > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
