See also Fraser and Marcu's analysis of AER: http://aclweb.org/anthology-new/J/J07/J07-3002.pdf
On Wed, Mar 4, 2009 at 1:46 PM, Miles Osborne <[email protected]> wrote: > one thing to remember is that the link between AER and BLEU is not > obvious; in my view at least AER-like scores should be treated with > skepticism and the real merit of an alignment approach should be the > corresponding translation performance (BLEU etc). > > can you provide associated BLEU scores for those AER numbers? > > Miles > > 2009/3/4 J.Tiedemann <[email protected]>: > > > > hi, > > > > I'm just wondering if Och's "refined" heuristics is also implemented > > in Moses. The "grow-diag" is not exactly the same as far as I > > understand. > > > > The reason why I'm asking is because I found out that in all of my > > experiments with europarl data the "intersection" always produces the > > best results in terms of AER (for example using the wpt03 data) > > whereas I see better performances reported for refined compared with > > intersection in various papers (also for the wpt03 data). However, I > > cannot believe that the grow-heuristics would perform so much worse > > than the original "refined" approach. > > > > My AER scores with standard GIZA settings and moses heuristics for > > wpt03 data are the following: > > > > moses.intersect AER = 0.0613 > > moses.grow-diag AER = 0.0843 > > moses.grow-diag-final-and AER = 0.0926 > > moses.grow-diag-final AER = 0.1312 > > moses.srctotgt AER = 0.1039 > > moses.tgttosrc AER = 0.1162 > > moses.union AER = 0.1444 > > > > does this sound reasonable? > > > > > > Jorg > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
