yep, that sounds reasonable. in that case it is good to remember that those heuristics are all designed for eventual translation and not for doing well at AER. i can easily imagine some other set of heuristics which will do well at word alignment-like tasks and not necessarily pan-out into good bleu scores etc.
Miles 2009/3/4 J.Tiedemann <[email protected]>: > > it depends on what you want to do. I was interested in the word alignment in > particular. not necessarily for running MT with moses. > > for SMT I usually just use the default grow-diag-final-and which probably > gives the best input anyway. this is, I guess, because it's better on > recall. AER seems to strongly prefer precision. > > jorg > > > On Wed, 4 Mar 2009 13:46:36 +0000 > Miles Osborne <[email protected]> wrote: >> >> one thing to remember is that the link between AER and BLEU is not >> obvious; in my view at least AER-like scores should be treated with >> skepticism and the real merit of an alignment approach should be the >> corresponding translation performance (BLEU etc). >> >> can you provide associated BLEU scores for those AER numbers? >> >> Miles >> >> 2009/3/4 J.Tiedemann <[email protected]>: >>> >>> hi, >>> >>> I'm just wondering if Och's "refined" heuristics is also implemented >>> in Moses. The "grow-diag" is not exactly the same as far as I >>> understand. >>> >>> The reason why I'm asking is because I found out that in all of my >>> experiments with europarl data the "intersection" always produces the >>> best results in terms of AER (for example using the wpt03 data) >>> whereas I see better performances reported for refined compared with >>> intersection in various papers (also for the wpt03 data). However, I >>> cannot believe that the grow-heuristics would perform so much worse >>> than the original "refined" approach. >>> >>> My AER scores with standard GIZA settings and moses heuristics for >>> wpt03 data are the following: >>> >>> moses.intersect AER = 0.0613 >>> moses.grow-diag AER = 0.0843 >>> moses.grow-diag-final-and AER = 0.0926 >>> moses.grow-diag-final AER = 0.1312 >>> moses.srctotgt AER = 0.1039 >>> moses.tgttosrc AER = 0.1162 >>> moses.union AER = 0.1444 >>> >>> does this sound reasonable? >>> >>> >>> Jorg >>> _______________________________________________ >>> Moses-support mailing list >>> [email protected] >>> http://mailman.mit.edu/mailman/listinfo/moses-support >>> >> >> >> >> -- >> The University of Edinburgh is a charitable body, registered in >> Scotland, with registration number SC005336. > > -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
