Yes.
On 10/25/12 15:30, Nicola Bertoldi wrote: > Moses uses log_e > while Kenlm uses log_10 > > log_e(P) = log_e(10) log_10(P) > > -17.715 = log_e(10)*-7.69363 > > > Am I right? > > > Nicola > > On Oct 25, 2012, at 4:09 PM, Adam Teichert wrote: > >> Hi all, >> >> After decoding, I get the output "I am ." from moses. I expected >> the lm feature of the winning hypothesis to match the language model >> score of the output (i.e. running kenlm on "I am .") but it doesn't >> match. >> >> moses decoder says the lm feature should be: -17.7152 >> kenlm says: -7.69363 >> >> Any ideas? (more verbose details below) >> >> Thanks, >> >> --Adam >> >> >> moses: >> 0 ||| I am . ||| d: 0 lm: -17.7152 0 w: -3 tm: -13.6791 -13.4943 >> -7.13198 -2.7697 2.99969 ||| -17.7152 >> >> kenlm: >> I=11 2 -0.960569 am=16 2 -3.58125 .=2 1 -2.85073 </s>=12 2 >> -0.301086 Total: -7.69363 OOV: 0 >> >> lm.arpa: >> >> >> \data\ >> ngram 1=24 >> ngram 2=61 >> ngram 3=55 >> ngram 4=37 >> ngram 5=15 >> >> \1-grams: >> -1.468523 <unk> >> -1.847186 , -1.369804 >> -1.949801 . -1.569853 >> -1.963158 and -1.181599 >> -2.323228 is -1.180351 >> -2.361128 that -1.115681 >> -2.58365 : -0.9018961 >> -2.668149 it -0.9380397 >> -2.68633 this -0.947085 >> -2.810327 we -1.037582 >> -2.848784 not -0.6793468 >> -2.959159 I -1.039103 >> -3.07579 </s> >> -3.192658 my -0.771203 >> -3.257855 no -0.6133907 >> -99 <s> -2.155778 >> -4.495705 am -0.2612615 >> -3.793312 little -0.4288335 >> -3.485892 me -0.7101038 >> -4.420267 surprise -0.5936328 >> -4.396488 surprised -0.7797279 >> -4.735315 surprises -0.5008686 >> -4.552582 surprising -0.5104726 >> -4.420267 wonder -0.7249804 >> >> \2-grams: >> -0.3010858 .</s> >> -0.7091757 surprises me -0.2860005 >> -0.8920909 surprises . -0.666689 >> -0.9196912 surprised that -0.3720784 >> -0.9494263 surprise , -0.2865064 >> -0.9513271 surprising , -0.3125141 >> -0.9605687 <s> I -2.082714 >> -1.027536 surprises , -0.1418063 >> -1.03712 surprising that -0.234587 >> -1.113948 surprising . -0.818023 >> -1.128903 wonder , -0.201676 >> -1.129378 surprise that -0.2199759 >> -1.144491 <s> we -1.811983 >> -1.176047 surprised , -0.3082836 >> -1.223108 me , -0.4326107 >> -3.267176 , . -0.6048141 >> -1.942681 , I -1.324145 >> -1.624891 , and -1.041633 >> -2.088251 , is -0.7182704 >> -2.00548 , it -1.121896 >> -2.659058 , no -0.426425 >> -2.169326 , not -0.7878548 >> -2.013858 , that -0.741353 >> -4.511329 <s> am -1.611718 >> -2.70271 <s> no -0.6921422 >> -1.498533 I am -0.6396686 >> -2.678573 I wonder -0.4021412 >> -1.723108 am I -0.1898816 >> -1.983081 am not -0.05170299 >> -3.118301 and . -0.6289005 >> -4.704317 and surprising -0.03819112 >> -4.235544 and wonder -0.3394165 >> -2.49379 is . -1.147667 >> -2.878376 is it -0.3021726 >> -2.286608 is no -0.2445829 >> -1.610847 is not -0.7426848 >> -4.088284 is surprising -0.2584699 >> -1.43062 it . -1.05238 >> -1.354294 it is -0.4715224 >> -1.872162 little . -0.7615967 >> -1.302207 me . -0.7159881 >> -2.216844 no . -0.4275606 >> -3.09122 no surprise -0.238984 >> -3.29149 no surprises -0.1407324 >> -2.364784 not . -0.5624467 >> -3.183594 not it -0.0237771 >> -3.857746 not me -0.1115348 >> -3.5716 not surprise -0.333057 >> -3.570876 not surprised -0.2835279 >> -3.621675 not surprising -0.1414568 >> -1.247403 surprise . -0.5275136 >> -1.554344 surprise me -0.1131797 >> -1.231814 surprised . -1.243741 >> -1.353927 surprised me -0.2527946 >> -2.455974 that . -0.8608306 >> -1.859256 that is -0.5548451 >> -1.854284 that it -0.8317314 >> -2.064787 this . -0.8331819 >> -2.12582 this is -0.4543184 >> -2.882952 we . -0.9427112 >> -1.525799 wonder . -1.009658 >> >> \3-grams: >> -0.01054467 <s> am I -0.4033368 >> -0.0125701 surprised .</s> >> -0.02177685 wonder .</s> >> -0.0255149 we .</s> >> -0.02761195 is .</s> >> -0.03434359 surprising .</s> >> -0.03933057 little .</s> >> -0.04950372 surprises .</s> >> -0.05429693 and .</s> >> -0.06596278 it .</s> >> -0.06643473 , .</s> >> -0.069778 surprise .</s> >> -0.08091767 me .</s> >> -0.0885259 that .</s> >> -0.0999171 this .</s> >> -0.1734623 no .</s> >> -0.1389831 not .</s> >> -2.214 , I wonder -0.9099113 >> -1.619141 I am not -0.4131633 >> -4.240031 , and . -0.2288982 >> -3.244178 , is . -0.4621631 >> -2.173454 , is no -0.5102387 >> -2.217342 it is . -0.6322967 >> -1.657225 it is not -0.3576518 >> -2.566752 that is . -0.5047822 >> -3.111846 that is it -0.1018341 >> -2.304894 this is . -0.7973652 >> -2.962754 this is it -0.08542722 >> -0.6746925 , it is -0.9160917 >> -1.739163 is it . -0.1419258 >> -1.454397 not it . 0 >> -0.8604823 that it is -0.6332519 >> -0.5766525 not me . -0.3372244 >> -0.9619833 surprise me . -0.1180146 >> -0.8732141 surprised me . -0.5590731 >> -2.067063 , no . -0.7058067 >> -1.440352 <s> no . -1.143604 >> -2.403745 is no . -0.5590731 >> -2.466519 is no surprise -0.2155667 >> -3.440862 , not me -0.5590731 >> -2.300408 am not . -0.07168788 >> -2.422223 am not surprised -0.08383179 >> -2.473987 is not . -0.834681 >> -3.623306 is not it -0.07617349 >> -3.54658 is not surprising -0.1496836 >> -1.219092 no surprise . -0.1016625 >> -1.127067 not surprise me -0.066174 >> -1.265616 not surprised . -0.228898 >> -0.6443897 no surprises . -0.3372243 >> -0.8107078 and surprising . -0.2288982 >> -0.8876699 is surprising . -0.4621631 >> -1.04663 not surprising . 0.004644036 >> -0.9365889 , that is -0.8274354 >> -1.870765 I wonder . -0.2288982 >> -1.371585 and wonder . -0.2288982 >> >> \4-grams: >> -0.004286873 this is .</s> >> -0.00737656 not surprised .</s> >> -0.008448888 that is .</s> >> -0.009329482 , is .</s> >> -0.0103978 <s> no .</s> >> -0.01154472 is surprising .</s> >> -0.01272331 I wonder .</s> >> -0.01272331 and wonder .</s> >> -0.01440236 it is .</s> >> -0.017762 is not .</s> >> -0.01994449 and surprising .</s> >> -0.0208705 surprised me .</s> >> -0.02207431 no surprises .</s> >> -0.02910873 , no .</s> >> -0.03122709 , and .</s> >> -0.04654508 is it .</s> >> -0.03536305 not me .</s> >> -0.06025264 surprise me .</s> >> -0.04138191 is no .</s> >> -0.1147482 am not .</s> >> -0.05426025 no surprise .</s> >> -0.03472797 not surprising .</s> >> -3.038089 , it is . -0.42836 >> -2.386421 that it is . -0.759353 >> -2.281374 , that is . -1.259269 >> -1.008098 that is it . -0.2749732 >> -0.9806256 this is it . -0.2749732 >> -0.7204503 is not it . -0.6572088 >> -0.09846061 , not me . -0.5780276 >> -0.8568227 not surprise me . -0.3711458 >> -1.815292 , is no . -0.4811176 >> -2.135054 I am not . -0.6190532 >> -1.894238 it is not . -1.09429 >> -0.8054344 is no surprise . -0.2930098 >> -0.9460168 am not surprised . -0.2210943 >> -0.8717432 is not surprising . -0.4107019 >> -2.35131 , I wonder . -0.2210943 >> >> \5-grams: >> -0.0004608329 , that is .</s> >> -0.001402984 it is not .</s> >> -0.002472492 that it is .</s> >> -0.004418593 am not surprised .</s> >> -0.005315542 , it is .</s> >> -0.007602483 , I wonder .</s> >> -0.009067477 , not me .</s> >> -0.01316113 is not surprising .</s> >> -0.01323633 , is no .</s> >> -0.01368839 is not it .</s> >> -0.02408934 that is it .</s> >> -0.02408934 this is it .</s> >> -0.02462093 not surprise me .</s> >> -0.0249456 I am not .</s> >> -0.02678858 is no surprise .</s> >> >> \end\ >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
