Hi,
the option "-v" will give you the information you need
to find out why a step could not be re-used.
For each step, the same steps in other runs are checked,
and the "-v" option gives you output such as:
re_use 41 TRAINING:run-giza-inverse (v26) 3 ?
check parameter count current: 3, old: 3
check '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]'
eq '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]' -> ok
check 'USED /home/pkoehn/experiment/wmt12-de-en/training/prepared.*' eq
'USED /home/pkoehn/experiment/wmt12-de-en/training/prepared.26' -> ok
check '-mgiza -mgiza-cpus 8 -sort-buffer-size 30G' eq '-mgiza -mgiza-cpus
8 -sort-buffer-size 30G' -> ok
all parameters match
re-usable
A step may not be re-usable, if the number of parameters has
changed, if any of the settings was changed, or if a timestamp
has changed.
For instance:
re_use 36 TRAINING:build-ttable (v24) ?
RECURSIVE_RE_USE{36,24,TRAINING:extract-phrases} = 8
RECURSIVE_RE_USE{36,24,TRAINING:build-lex-trans} = 3
check parameter count current: 5, old: 5
check '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]'
eq '/home/pkoehn/moses/scripts/training/train-model.perl [1317810495]' -> ok
check 'USED /home/pkoehn/experiment/wmt12-de-en/model/extract.*
/home/pkoehn/experiment/wmt12-de-en/model/lex.*' eq 'USED
/home/pkoehn/experiment/wmt12-de-en/model/extract.8
/home/pkoehn/experiment/wmt12-de-en/model/lex.3' -> ok
check 'true' eq 'true' -> ok
check '-mgiza -mgiza-cpus 8 -sort-buffer-size 30G' eq '-mgiza -mgiza-cpus
8 -sort-buffer-size 30G' -> ok
check ' --GoodTuring --MinCountHierarchical 2' eq ' --GoodTuring' ->
mismatch
parameters from info file do not match
not re-usable
A step is also not re-usable if it depends on a preceeding step which
is not re-usable, i.e., the whole chain is invalidated. This is not
clear from this debug output, but it is quite straight-forward to always
check the first step that failed.
One annoying problem is that occasionally due to minor fixes,
recompilation, etc., the timestamps of scripts and binaries change,
but you know that it has no impact on your experiment.
I checked in a script "scripts/ems/fix-info.perl" that allows you to
update all timestamps for a file in the *.INFO records, for instance:
fix-info.perl /home/pkoehn/moses/scripts/training/train-model.perl
A bit of a hack, but very handy.
-phi
On Thu, Apr 12, 2012 at 9:26 AM, Barry Haddow <[email protected]>
wrote:
> Hi Eva
>
> ems will also check the date stamps and filenames of any tools and data
you
> use, and may rerun steps if any have changed. The dependencies are made
> explicit in experiment.meta, and you can trace what it's doing if you run
> experiment.perl with the -v option (the output is quite verbose though).
>
> Or if you want to reuse symmetrised alignments, then just set the word-
> alignment variable in the TRAINING section of your config file,
>
> cheers - Barry
>
> On Thursday 12 April 2012 17:13:59 Eva Hasler wrote:
>> actually, something similar is happening in a different experiment, I
>> deleted the training.build-ttable and training.create-config steps and it
>> wants to restart from training.prepare-data, even though I haven't
changed
>> anything in the config file
>>
>> On Thu, Apr 12, 2012 at 2:01 PM, Barry Haddow <[email protected]>
wrote:
>> > Hi Eva
>> >
>> > If you have an ini file with tuned weights then you can force ems to
use
>> > it by
>> > setting config-with-reused-weights in the TUNING section. It should
then
>> > continue with evaluation.
>> >
>> > cheers - Barry
>> >
>> > On Thursday 12 Apr 2012 12:35:30 Eva Hasler wrote:
>> > > Hi,
>> > >
>> > > my experiment failed at the evaluation.filter step, I fixed it and
now
>> > > I want to continue. But when I run experiment.perl -continue, it
wants
>> > > to start again from training.create-config. I've had this problem a
>> > > couple
>> >
>> > of
>> >
>> > > times, is there a way to convince experiment.perl that I do not want
to
>> >
>> > do
>> >
>> > > the whole tuning step again?
>> > >
>> > > Eva
>>
>
> --
> Barry Haddow
> University of Edinburgh
> +44 (0) 131 651 3173
>
> --
> The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
> _______________________________________________
> Moses-support mailing list
> [email protected]
> http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
