Hi, I think the average is OK, your variance is however quite high. Did you retrain the entire system or just optimize parameters a couple of times?
Two useful papers on the topic: https://www.cs.cmu.edu/~jhclark/pubs/significance.pdf http://www.mt-archive.info/MTS-2011-Cettolo.pdf On 22.06.2015 02:37, Hokage Sama wrote: > Hi, > > Since MT training is non-convex and thus the BLEU score varies, which > score should I use for my system? I trained my system three times > using the same data and obtained the three different scores below. > Should I take the average or the best score? > > BLEU = 17.84, 49.1/22.0/12.5/7.5 (BP=1.000, ratio=1.095, hyp_len=3952, > ref_len=3609) > BLEU = 16.51, 48.4/20.7/11.4/6.5 (BP=1.000, ratio=1.093, hyp_len=3945, > ref_len=3609) > BLEU = 15.33, 48.2/20.1/10.3/5.5 (BP=1.000, ratio=1.087, hyp_len=3924, > ref_len=3609) > > Thanks, > Hilton > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
