Dear Mikel, I'd check Daniel Cer's thesis, I'm not sure if he did anything with BLEU coefficients but he definitely convinced himself that there was no better tuning metric than BLEU.
...but we're running the tuning task at WMT this year again, so something new will hopefully show up. Cheers, Ondrej. On February 3, 2015 7:09:20 AM CET, Mikel Forcada <[email protected]> wrote: >Thanks a million, Philipp! >About these tunable metrics, has anything been published? do you have >any references? >Cheers >Mikel > > >> >> Hi, >> >> while the original BLEU paper mentions co-efficients, there are not >> used (i.e., all are set to 1/4). >> >> There have efforts to create tunable metrics, but these are not >> widely used. >> >> -phi >> >> On Mon, Feb 2, 2015 at 4:34 AM, Mikel L. Forcada <[email protected]> >> wrote: >> > Dear list, >> > >> > the correlation of BLEU with manual measurements of quality has >> > extensively been studied (Callison-Burch et al., WMT "Findings" >> > paper). But, would anyone know of any paper where people have >> > actually tuned the BLEU coefficients to approximate some kind of >> > manual measurement of quality? This is hard to search for as most >> > papers talk about tuning the coefficients of something else to >BLEU. >> > >> > Thanks a million! >> > >> > All the best, >> > >> > Mikel >> > >> > -- >> > >_______________________________________________ >Moses-support mailing list >[email protected] >http://mailman.mit.edu/mailman/listinfo/moses-support -- Ondrej Bojar (mailto:[email protected] / [email protected]) http://www.cuni.cz/~obo _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
