Barry is correct. Also kenlm doesn't care what the third field is. I just read it from the ARPA file. Using a model with lower order that it was trained for is incorrect under most smoothing methods.
On 04/30/11 16:39, Barry Haddow wrote: > Hi Alexandre > > The format of the language model specification is described here > http://www.statmt.org/moses/?n=FactoredTraining.BuildingLanguageModel > > The first number is the language model type, and the third is the > order, so to use a 5-gram kenlm model, you need "8 0 5 /path/to/lm" > > best regards - Barry > > Quoting Alexandre Helle <[email protected]> on Sat, 30 Apr > 2011 20:19:47 +0200: > >> Thanks, i think all is working, but i don't know if I'm using well the >> kenlm. >> >> Is It alright to use the kenlm doing only this?: >> >> *Create with srilm (compiled independently of moses) the lm: >> ngram-count -order 5 -interpolate -kndiscount -text data/low/train.cln.$f >> -lm lm/train.$f.arpa >> >> *and chaging in the file moses.ini the "5" to "8". >> >> >> 2011/4/30 Barry Haddow <[email protected]> >> >>> Hi Alexandre >>> >>> The mert-moses script requires an additional argument -mertdir to indicate >>> where the mert binaries are located. This is normally the mert directory of >>> the Moses distribution. >>> >>> >>> Best regards - Barry >>> >>> Sent from my ZX81 >>> >>> ----- Reply message ----- >>> From: "Alexandre Helle" <[email protected]> >>> Date: Wed, Apr 27, 2011 23:40 >>> Subject: [Moses-support] Versions of Moses >>> To: "Barry Haddow" <[email protected]>, "Lane Schwartz" < >>> [email protected]> >>> Cc: <[email protected]> >>> >>> >>> Hi >>> >>> The first problem encountered was executing the compiled moses-svn with the >>> giza compiled (1.0.5) and srilm compiled (1.5.12), in the step "Confirm >>> Setup Success" of the step-by-step guide on the moses site. >>> >>> The second problem was compiling moses with the giza and srilm from >>> repositories (with its "-devel" installed), telling me that the libraries >>> was not compiled inside. >>> >>> Finally I have used again the moses and srilm from repositories, and I had >>> compiled only the moses-scripts with the "make release", but I had problems >>> with the "mert-moses.pl": >>> >>>> Not executable: >>>> >>> /home/alex/moses/moses-decoder/scripts/bin/scripts-20110426-1407/../mert/extractor >>>> at >>>> >>> /home/alex/moses/moses-decoder/scripts/bin/scripts-20110426-1407/training/ >>>> mert-moses.pl line 303. >>> >>> but, this binary file doesn't exist. I think it is created with the >>> compiling of the moses. >>> >>> As you said, I'm going to test compiling moses without the srilm. >>> >>> I use the the step-by-step guide on the moses site and the paper "AERFAI >>> Summer School on New Trends in Pattern Recognition for Language >>> Technologies, Bilbao, June 25, 2008 Practice Session with Moses[1]" of >>> Germán Sanchis-Trilles. >>> >>> P.D.: sorry, I didn't save the output of the first two errors. >>> >>> Thanks, >>> Alexandre >>> >>> 2011/4/27 Barry Haddow <[email protected]> >>> >>>> Hi Alexandre >>>> >>>> Using the latest versions of these other tools should work. Although as >>>> Lane points out, using kenlm should ease compatibility issues. >>>> >>>> Could you be more specific about the problems you've encountered? >>>> >>>> Have you tried the step-by-step guide on the Moses site? It's slightly >>>> out-of-date in places, but mostly valid. >>>> >>>> Best regards - Barry >>>> >>>> Sent from my ZX81 >>>> >>>> >>>> ----- Reply message ----- >>>> From: "Alexandre Helle" <[email protected]> >>>> Date: Wed, Apr 27, 2011 15:51 >>>> Subject: [Moses-support] Versions of Moses >>>> To: <[email protected]> >>>> >>>> Hi all, >>>> >>>> finally I have installed moses from svn, but it doesn't worked well. I >>>> thing >>>> the problem is related to the versions of giza, srilm and irstlm. >>>> >>>> Can you tell me the versions that you use? >>>> >>>> Does someone use Moses-svn in OpenSuse without problems? >>>> >>>> Thanks all. >>>> >>>> -- >>>> Alexandre Helle >>>> >>>> >>>> >>>> The University of Edinburgh is a charitable body, registered in >>>> Scotland, with registration number SC005336. >>>> >>>> >>> >>> >>> -- >>> Saludos, >>> Alexandre Helle >>> >>> >>> >>> The University of Edinburgh is a charitable body, registered in >>> Scotland, with registration number SC005336. >>> >>> >> >> >> -- >> Saludos, >> Alexandre Helle >> > > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
