I experienced an error that I've not seen before using train-model.perl
from Moses Release 1.0. Here are the final lines of the output log as
the script terminated. The error message "Illegal reordering type used:
d" is new to me.
(7) learn reordering model @ Sun Jun 1 00:12:55 IST 2014
(7.1) [no factors] learn reordering model @ Sun Jun 1 00:12:55 IST 2014
(7.2) building tables @ Sun Jun 1 00:12:55 IST 2014
Executing:
/usr/local/lib/mosesdecoder/scripts/../bin/lexical-reordering-score
/opt/domy/TRAININGS/alignments/align-td_fr_ui-en_us-fr_fr/giza.extract.en_us-fr_fr/ext.7-gram.o.sorted.gz
0.5
/opt/domy/ENGINES/tables/phrase-s=en_us-t=fr_fr-p=td_fr_ui-a=giza-g=7/reordering-table.
--model "wbe msd wbe-msd-bidirectional-fe"
Lexical Reordering Scorer
scores lexical reordering models of several types (hierarchical,
phrase-based and word-based-extraction
*Illegal reordering type used: d*
Exit code: 1
ERROR: Lexical reordering scoring failed at
/usr/local/bin/train-model.perl line 1705.
I suspect two potential causes.
1) the system only has about 50 GB of free hard drive space. There are
several tools in the training toolchain that create atomic temp files
that disappear immediately on an error. Is it possible this is one of
those place, and would this error occur if the system ran out of hard
drive space during this stage? I.e. it needs more than 50GB temp file
space?
2) the log has several warning messages that sentence pairs violate 9:1
ratio. E.g.
WARNING: The following sentence pair has source/target sentence length
ration more than the maximum allowed limit for a source word fertility
source length = 1 target length = 15 ratio 15 ferility limit : 9
Shortening sentence
Sent No: 212107 , No. Occurrences: 1
This should not be happening with our corpus preparation tools. There
doesn't seem to be anything particularly different about this training
corpus. So, we will look for our "leak" that let them pass our
preparation filters. Nonetheless, would these ratio violations cause the
fatal error in step 7 as logged above?
Can anyone can confirm my suspensions or suggest an alternate cause.
Thanks,
Tom
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
