Hi Yvette, Barry was spot on as usual. The code is in a branch:
svn co https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/branches/mert-mtm5/ moseslrscore You could also use the latest version of moses and just take the files I changed for this branch. There weren't too many. There are a few more parameters to use the mert with LRscore. In the config.ini file put: ### tuning script to be used tuning-script = "$moses-script-dir/training/mert-moses-new.pl --jobs 10" refalign = $working-dir/tuning/source/Tune.berkeley tuning-settings = "-mertdir $moses-src-dir/mert -mertargs=' --sctype KENDALL,BLEU --scconfig refalign:$refalign.0+$refalign.1+$refalign.2+$refalign.3,source:$input,weights:0.2623+0.7377 ' " So you need to align your dev set source/translation and in this config they are called $working-dir/tuning/source/Tune.berkeley.0-3, they are in the format: 0-0 11-16 3-4 8-11 1-2 4-6 9-13 8-10 0-1 2-3 5-7 6-7 7-9 --testalign is the alignment info between the decoded translations and the source. --refalign are the alignments between each reference and the source. For the source/reference alignments, you could either use a test set with gold standard alignments, or you could align using the Berkeley/GIZA++ aligner trained on the training set. For the test set, you could either get your decoder to output alignments, or again automatically align using Berkeley/GIZA++. I have used Berkeley because you can train it once and then run it separately with different test sets. Philipp also reports slightly better results with Berkeley. On Tue, Jun 12, 2012 at 8:52 AM, Barry Haddow <[email protected]> wrote: > Hi Yvette > > The LRScore metric was implemented in the mert-mtm5 branch (see > PermutationScorer) but it doesn't look like it was merged into trunk. You'd > also need to use InterpolatedScorer to interpolate the permutation metric with > bleu. > > mert in general seems to be missing some documentation, and in particular the > alternative scorers are not documented (as far as I can see). However you can > use a scorer by passing the "--sctype TYPE" argument to mert, where the > scorers are listed in ScorerFactory.cpp, > > cheers - Barry > > On Monday 11 June 2012 15:15:58 [email protected] wrote: >> Hi there, >> >> I want to optimize for lr-score instead of bleu using hierarchical moses. >> Could you tell me if there are instructions available anywhere? >> >> Thanks a lot, >> Yvette >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > > -- > Barry Haddow > University of Edinburgh > +44 (0) 131 651 3173 > > -- > The University of Edinburgh is a charitable body, registered in > Scotland, with registration number SC005336. > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
