Hi Neda, this happens because the seed used in the MERT optimizer depends by default on the moment on which you launch it. If you need deterministic MERT runs, you can set the seed by adding this switch to the mert-moses.pl call:
--mertargs=" -r $seed " Patrik Le 29/10/2011 18:11, [email protected] a écrit : > Message: 3 > Date: Sat, 29 Oct 2011 17:05:11 +0100 > From: Barry Haddow<[email protected]> > Subject: Re: [Moses-support] mert-moses.pl script > To: [email protected] > Message-ID:<[email protected]> > Content-Type: Text/Plain; charset="utf-8" > > Hi Neda > > Yes, this is quite normal. The best plan is to do several runs and take the > average bleu. See this paper for a discussion > > http://www.cs.cmu.edu/~jhclark/pubs/significance.pdf > > cheers - Barry > > On Saturday 29 Oct 2011 10:09:29 Neda NoorMohammadi wrote: >> hello, >> >> In my experiment this is happening: >> >> I am running mert-moses.pl script on a same develop set and configure file >> in a same condition but by each execution I gain different weights which >> leads to different Bleu (about 1%) (note that these executions are >> independent of each other and are initialized to predefine weights of >> moses.ini). >> >> The used features in configure file are: LM: 3gram, Reordering: >> msd-bidirectional-fe, Alignment: grow-diag-final-and, phrase scores. >> >> Is it normal? What is your suggestion? >> In this situation what is my baseline? The best one or the worth? >> >> Neda Noormohammadi >> > > ------------------------------ > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > End of Moses-support Digest, Vol 60, Issue 46 > ********************************************* _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
