ok, I'll check that. But I think I am doing that. any other suggestions?
________________________________ From: Carlos Henriquez [mailto:[email protected]] Sent: Mon 3/1/2010 12:22 PM To: Korzec, Sanne Cc: [email protected] Subject: Re: [Moses-support] lower BLEU score than normal Hi Sanne. Those papers usually explain how the systems were built. Did you check that you are using the same corpora for training and tuning. What about preprocessing, like case sensitivity. I think I can help you with some of your questions: > 2) does it affect the reordering model if I choose another ngram length > during training? Nope. The language model information you placed on the training script is used at the end, when the moses.ini file is created. The information used in step 7 (learning reordering model) is a file generated in step 5 (phrase extraction) named extract.o.gz. You may open it to see its content. > 4) Wat do the 6 and the 5 mean in the reordering table line. I'm not sure about the 5 but the 6 should mean the number of weights in your reordering model. Those are two for monotone, two for swap and two for discontinous phrase moves. You can open the reordering-table file to check it ;) -- Carlos A. Henríquez Q. [email protected] ----- Mensaje original ---- De: "Korzec, Sanne" <[email protected]> Para: [email protected] Enviado: lun,1 marzo, 2010 11:48 Asunto: [Moses-support] lower BLEU score than normal Hi all, I am running some experiments where the BLEU score in my experiments is a little lower than from BLEU scores published in recent papers. I am trying to figure out why this is, but I am running out of ideas. Any help would be appreciated. I am making a list of possible points which might be relevant. I am doing a FR -> EN translation on wmt 08 europarl v3b. Koehn in 2003 reports a bleu score of 0.3389 with a language model of ngram 3 mylonakis in 2008 report a bleu score of 0.3303 with a language model of ngram 5 my model gets a bleu score of 0.3145 on a language model of ngram 4 and 0.3178 on a LM of ngram 5. I have a moses compilation of 8-5-2008 and doing the normal baseline instructions as found under baseline system wmt08. Questions: 1) is it possible I have some faulty moses compilation. Should I redownload another version from svn? 2) does it affect the reordering model if I choose another ngram length during training? 3) my moses.ini prints: ============================ # limit on how many phrase translations e for each phrase f are loaded # 0 = all elements loaded [ttable-limit] 20 0 # distortion (reordering) files [distortion-file] 0-0 msd-bidirectional-fe 6 /scratch/res_10/mosespl5pc1.0/model/reordering-table. msd-bidirectional-fe.0.5.0-0.gz =================================== 3) why are there two lines under table-limit? should there be just one? 4) Wat do the 6 and the 5 mean in the reordering table line. regards, Sanne _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
