great, I'll try that!
On Thu, Apr 12, 2012 at 10:09 PM, Philipp Koehn <[email protected]> wrote:
> Hi,
>
> the option "-v" will give you the information you need
> to find out why a step could not be re-used.
>
> For each step, the same steps in other runs are checked,
> and the "-v" option gives you output such as:
>
> re_use 41 TRAINING:run-giza-inverse (v26) 3 ?
> check parameter count current: 3, old: 3
> check '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]'
> eq '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]' -> ok
> check 'USED /home/pkoehn/experiment/wmt12-de-en/training/prepared.*' eq
> 'USED /home/pkoehn/experiment/wmt12-de-en/training/prepared.26' -> ok
> check '-mgiza -mgiza-cpus 8 -sort-buffer-size 30G' eq '-mgiza -mgiza-cpus
> 8 -sort-buffer-size 30G' -> ok
> all parameters match
> re-usable
>
> A step may not be re-usable, if the number of parameters has
> changed, if any of the settings was changed, or if a timestamp
> has changed.
>
> For instance:
>
> re_use 36 TRAINING:build-ttable (v24) ?
> RECURSIVE_RE_USE{36,24,TRAINING:extract-phrases} = 8
> RECURSIVE_RE_USE{36,24,TRAINING:build-lex-trans} = 3
> check parameter count current: 5, old: 5
> check '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]'
> eq '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]' -> ok
> check 'USED /home/pkoehn/experiment/wmt12-de-en/model/extract.*
> /home/pkoehn/experiment/wmt12-de-en/model/lex.*' eq 'USED
> /home/pkoehn/experiment/wmt12-de-en/model/extract.8
> /home/pkoehn/experiment/wmt12-de-en/model/lex.3' -> ok
> check 'true' eq 'true' -> ok
> check '-mgiza -mgiza-cpus 8 -sort-buffer-size 30G' eq '-mgiza -mgiza-cpus
> 8 -sort-buffer-size 30G' -> ok
> check ' --GoodTuring --MinCountHierarchical 2' eq ' --GoodTuring' ->
> mismatch
> parameters from info file do not match
> not re-usable
>
> A step is also not re-usable if it depends on a preceeding step which
> is not re-usable, i.e., the whole chain is invalidated. This is not
> clear from this debug output, but it is quite straight-forward to always
> check the first step that failed.
>
> One annoying problem is that occasionally due to minor fixes,
> recompilation, etc., the timestamps of scripts and binaries change,
> but you know that it has no impact on your experiment.
>
> I checked in a script "scripts/ems/fix-info.perl" that allows you to
> update all timestamps for a file in the *.INFO records, for instance:
> fix-info.perl /home/pkoehn/moses/scripts/training/train-model.perl
>
> A bit of a hack, but very handy.
>
> -phi
>
>
>
> On Thu, Apr 12, 2012 at 9:26 AM, Barry Haddow <[email protected]>
> wrote:
> > Hi Eva
> >
> > ems will also check the date stamps and filenames of any tools and data
> you
> > use, and may rerun steps if any have changed. The dependencies are made
> > explicit in experiment.meta, and you can trace what it's doing if you run
> > experiment.perl with the -v option (the output is quite verbose though).
> >
> > Or if you want to reuse symmetrised alignments, then just set the word-
> > alignment variable in the TRAINING section of your config file,
> >
> > cheers - Barry
> >
> > On Thursday 12 April 2012 17:13:59 Eva Hasler wrote:
> >> actually, something similar is happening in a different experiment, I
> >> deleted the training.build-ttable and training.create-config steps and
> it
> >> wants to restart from training.prepare-data, even though I haven't
> changed
> >> anything in the config file
> >>
> >> On Thu, Apr 12, 2012 at 2:01 PM, Barry Haddow <[email protected]>
> wrote:
> >> > Hi Eva
> >> >
> >> > If you have an ini file with tuned weights then you can force ems to
> use
> >> > it by
> >> > setting config-with-reused-weights in the TUNING section. It should
> then
> >> > continue with evaluation.
> >> >
> >> > cheers - Barry
> >> >
> >> > On Thursday 12 Apr 2012 12:35:30 Eva Hasler wrote:
> >> > > Hi,
> >> > >
> >> > > my experiment failed at the evaluation.filter step, I fixed it and
> now
> >> > > I want to continue. But when I run experiment.perl -continue, it
> wants
> >> > > to start again from training.create-config. I've had this problem a
> >> > > couple
> >> >
> >> > of
> >> >
> >> > > times, is there a way to convince experiment.perl that I do not
> want to
> >> >
> >> > do
> >> >
> >> > > the whole tuning step again?
> >> > >
> >> > > Eva
> >>
> >
> > --
> > Barry Haddow
> > University of Edinburgh
> > +44 (0) 131 651 3173
> >
> > --
> > The University of Edinburgh is a charitable body, registered in
> > Scotland, with registration number SC005336.
> >
> > _______________________________________________
> > Moses-support mailing list
> > [email protected]
> > http://mailman.mit.edu/mailman/listinfo/moses-support
>
>
--
@__^.^
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
