Hi Raphael, just to re-iterate: are you sure you used the correct (German) reference file to score your (German) translation output, and didn't mistakenly use the (English) input as reference?
best wishes, Rico On 18.11.2015 15:12, Raphael Hoeps wrote: > - When calculating the BLEU-score of this test set and the translation > of the test set with multi-bleu.perl I get the poor result of 3.76. > > Do you have any tips on how to find my mistake? > > Thanks a lot, > Raphael > > > > > > > Am 18.11.2015 um 14:56 schrieb Rico Sennrich: >> Hello Raphael, >> >> I suggest that you check if you mixed up the languages somewhere, and >> check if your translation output is actually English. >> >> 3.76 BLEU is possible to achieve without translation (because names and >> some function words are the same between English and German), and it's >> possible that you used the wrong reference file when measuring BLEU, or >> that your SMT system is broken in some way and just copies the source >> text to the output. >> >> best wishes, >> Rico >> >> >> On 18.11.2015 13:36, Raphael Hoeps wrote: >>> Hi, >>> I'm a Computer Science student from Germany working on a SMT-project. I >>> tried to get into the moses-system a little bit and did the >>> baseline-tutorial found here: >>> http://www.statmt.org/moses/?n=Moses.Baseline. I sticked to this >>> tutorial but used the German/English corpora. >>> >>> Unfortunately in the end I got a poor BLEU-score of only 3.76: >>> BLEU = 3.76, 24.1/6.2/2.2/0.9 (BP=0.906, ratio=0.910, hyp_len=68049, >>> ref_len=74753) >>> In the tutorial, a score of 23.5 was received. >>> >>> I think did everything as shown in the tutorial, except for one thing: >>> in the tuning-part I cut down the two development corpora to 500 lines >>> (from 2000), because my Laptop is quite old and I wanted to speed up the >>> process a little bit. (It still took my Laptop 6 hours). >>> Do you think that this is the reason for my poor score? >>> Is it maybe because I used the German/English corpora, so the score >>> can't be compared to the English/French system in the tutorial? >>> Or did I just make a mistake when typing all the commands? Any ideas how >>> to find this mistake? >>> >>> Thank you very much for your help, >>> Raphael >>> >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
