Hi Neda There is also a parameter of mert-moses.pl "--predictable-seed" (see the help) which makes MERT deterministic.
In general, MERT procedure relies on an initial seed seed_t at each iteration t for the creation of the (20 by default) random starting points for the optimization; not that seed_i differs from seed_j (if i differs from j) If you activate this flag "--predictable-seed", seed_i still differs from seed_j BUT the sequence seed_1, seed_2, ... seed_N is always the same; and hence, the random starting points considered for the optimization are always the same, so that the final results of the MERT is always the same. best regards Nicola On Oct 29, 2011, at 6:37 PM, Patrik Lambert wrote: > > Hi Neda, > > this happens because the seed used in the MERT optimizer depends by > default on the moment on which you launch it. > If you need deterministic MERT runs, you can set the seed by adding this > switch to the mert-moses.pl call: > > --mertargs=" -r $seed " > > Patrik > > > Le 29/10/2011 18:11, [email protected] a écrit : >> Message: 3 >> Date: Sat, 29 Oct 2011 17:05:11 +0100 >> From: Barry Haddow<[email protected]> >> Subject: Re: [Moses-support] mert-moses.pl script >> To: [email protected] >> Message-ID:<[email protected]> >> Content-Type: Text/Plain; charset="utf-8" >> >> Hi Neda >> >> Yes, this is quite normal. The best plan is to do several runs and take the >> average bleu. See this paper for a discussion >> >> http://www.cs.cmu.edu/~jhclark/pubs/significance.pdf >> >> cheers - Barry >> >> On Saturday 29 Oct 2011 10:09:29 Neda NoorMohammadi wrote: >>> hello, >>> >>> In my experiment this is happening: >>> >>> I am running mert-moses.pl script on a same develop set and configure file >>> in a same condition but by each execution I gain different weights which >>> leads to different Bleu (about 1%) (note that these executions are >>> independent of each other and are initialized to predefine weights of >>> moses.ini). >>> >>> The used features in configure file are: LM: 3gram, Reordering: >>> msd-bidirectional-fe, Alignment: grow-diag-final-and, phrase scores. >>> >>> Is it normal? What is your suggestion? >>> In this situation what is my baseline? The best one or the worth? >>> >>> Neda Noormohammadi >>> >> >> ------------------------------ >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> End of Moses-support Digest, Vol 60, Issue 46 >> ********************************************* > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
