Hi Sanne. Those papers usually explain how the systems were built. Did you check that you are using the same corpora for training and tuning. What about preprocessing, like case sensitivity.
I think I can help you with some of your questions: > 2) does it affect the reordering model if I choose another ngram length > during training? Nope. The language model information you placed on the training script is used at the end, when the moses.ini file is created. The information used in step 7 (learning reordering model) is a file generated in step 5 (phrase extraction) named extract.o.gz. You may open it to see its content. > 4) Wat do the 6 and the 5 mean in the reordering table line. I'm not sure about the 5 but the 6 should mean the number of weights in your reordering model. Those are two for monotone, two for swap and two for discontinous phrase moves. You can open the reordering-table file to check it ;) -- Carlos A. Henríquez Q. [email protected] ----- Mensaje original ---- De: "Korzec, Sanne" <[email protected]> Para: [email protected] Enviado: lun,1 marzo, 2010 11:48 Asunto: [Moses-support] lower BLEU score than normal Hi all, I am running some experiments where the BLEU score in my experiments is a little lower than from BLEU scores published in recent papers. I am trying to figure out why this is, but I am running out of ideas. Any help would be appreciated. I am making a list of possible points which might be relevant. I am doing a FR -> EN translation on wmt 08 europarl v3b. Koehn in 2003 reports a bleu score of 0.3389 with a language model of ngram 3 mylonakis in 2008 report a bleu score of 0.3303 with a language model of ngram 5 my model gets a bleu score of 0.3145 on a language model of ngram 4 and 0.3178 on a LM of ngram 5. I have a moses compilation of 8-5-2008 and doing the normal baseline instructions as found under baseline system wmt08. Questions: 1) is it possible I have some faulty moses compilation. Should I redownload another version from svn? 2) does it affect the reordering model if I choose another ngram length during training? 3) my moses.ini prints: ============================ # limit on how many phrase translations e for each phrase f are loaded # 0 = all elements loaded [ttable-limit] 20 0 # distortion (reordering) files [distortion-file] 0-0 msd-bidirectional-fe 6 /scratch/res_10/mosespl5pc1.0/model/reordering-table. msd-bidirectional-fe.0.5.0-0.gz =================================== 3) why are there two lines under table-limit? should there be just one? 4) Wat do the 6 and the 5 mean in the reordering table line. regards, Sanne _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
