Hi Miles. You said that moses will append <s> and </s> pseudo-words to the start and end of a sentence and that will change the probabilities, but actually
echo "after three hours ." | ngram -order 5 -lm ../marie/lm/train.tok.en.lm -ppl - echo "<s> after three hours .</s>" | ngram -order 5 -lm ../marie/lm/train.tok.en.lm -ppl - return the same value (perplexity and logprob) so I suppose that ngram added those pseudo-words as well, didn't it? It seems more like a base difference, as explained by Barry. Moses is outputting natural logarithms, srilm is outputting base 10. log(10) * -7.40966 = -17.0614 Thank you all. -- Carlos A. Henríquez Q. [email protected] ----- Mensaje original ---- De: Miles Osborne <[email protected]> Para: Carlos Henriquez <[email protected]> CC: [email protected] Enviado: jueves, 5 de marzo, 2009 11:26:29 Asunto: Re: [Moses-support] How is the final LM score obtained? a couple of points: --you are asking ngram for perplexities scores, but Moses uses log probs --Moses will append <s> and </s> pseudo-words to the start and end ot a sentence; this will change the probabilities Miles 2009/3/5 Carlos Henriquez <[email protected]>: > > Hi all. > > > I'm making some tests extracting the nbest list from moses (-n-best-list > option) with all models' weights set to 1 and I don't understand how do you > get the final LM score. I'm using srilm. > > For instance, my best translation from Chinese to English on sentence 9 was > > 9 ||| after three hours . ||| d: 0 lm: -17.0614 tm: -7.41812 -0.944461 > -4.79107 -2.87243 w: -4 ||| -37.0874 > > but if I run ngram alone with the same output sentence > > echo "after three hours ." | ngram -order 5 -lm ../marie/lm/train.tok.en.lm > -ppl - > > the result is very different > > file -: 1 sentences, 4 words, 0 OOVs > 0 zeroprobs, logprob= -7.40966 ppl= 30.3341 ppl1= 71.1892 > > I tried with some other values from my nbest list and I always found a big > difference between the two scores. > > If my initial weight is 1, why are the scores so different? I suppose I am > misunderstanding something. > > The moses command to obtain the n-best-list was > > moses -f moses.ini -i ../../corpus/dev.zh -d 1 -tm 1 1 1 1 -lm 1 -w 1 > -n-best-list devout.moses.nbest 10 -include-alignment-in-n-best true > > devout.moses 2> /dev/null > > (yep, I'm not using the last tm weight) and the moses.ini file does not have > any weights. > > -- > Carlos A. Henríquez Q. > [email protected] > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
