Thanks, Kenneth, This was revision 3796, I will try with a newer version.
> Hi, > > What revision of Moses are you using? Does this still happen > after you > run svn up and recompile Moses? > > Kenneth > > On 02/07/11 10:53, Kārlis Goba wrote: > > Hi, > > > > My preferred way to build large LMs has been IRSTLM as it can handle > large corpora nicely by splitting the task. The produced binary LMs > work well with Moses. Then I decided to try the new and shiny KenLM. > However, when converting the result to KenLM format, the converted LM > gives a lot worse BLEU score. > > > > I am building the LMs in this way: > > > > 1) build-lm.sh - build iARPA LM > > 2) prune-lm > > 3) compile-lm - convert iARPA to binary IRSTLM (gives me 0.3346 BLEU) > > 4) compile-lm --text=yes - convert iARPA to ARPA > > 5) build_binary trie - convert ARPA to KenLM (gives me 0.2543 BLEU) > > > > The moses.ini for both cases is the same, differing only in the LM > line (1 0 5 "/mnt/smt/lm/mt4-lv-lcase/lm-pruned.blm" vs "8 0 5 > /mnt/smt/lm/mt4-lv-lcase/lm-pruned.mmap"). I have observed this in more > than one case. > > > > I don't know who blame - either the conversion from iARPA to ARPA or > KenLM. > > > > What is your best practice for estimating large LMs to be converted > to KenLM? Is it SRILM? > > > > -- > > Karlis > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
