Hi,
I'm trying to do pivot translation between Arabic dialects via English. The
Egyptian to English side trains and tunes fine in about 4 hours, but the
English to Levantine side (although it is of comparable size and
preprocessed in exactly the same way) stalls before completing the first
run during the tuning step (training finished without error and the tuning
step never yields an error, it just never finishes the first run).
There are about 175,000 sentences in the training corpus for English to
Levantine, all of this data coming from weblogs used in the BOLT corpus.
The LM for Levantine is mostly comprised of the same data, with some
additional data news commentary and twitter. I tried tuning originally with
2000 sentences from BOLT, then 1000, and finally 500, but I ran into the
same stall each time while the Egyptian - English side never had an issue.
Do you have any idea what is going on or how to resolve this?
Here is the output file:
run 1 start at Sun Mar 12 23:26:52 GST 2017
Parsing --decoder-flags: |-v 0|
Saving new config to: ./run1.moses.ini
(1) run decoder to produce n-best lists
params = -v 0
decoder_config = -weight-overwrite 'PhrasePenalty0= 0.043478 WordPenalty0=
-0.217391 TranslationModel0= 0.043478 0.043478 0.043478 0.043478
Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217 0.065217
0.065217 0.065217 0.065217 0.065217'
and here is the error file:
Loading module 'mosesdecoder/3.0'
Loading module 'mgiza/2015.01'
mkdir: cannot create directory
Using SCRIPTS_ROOTDIR: /share/apps/NYUAD3/mosesdecoder/3.0/scripts
Assuming the tables are already filtered, reusing filtered/moses.ini
Using cached features list: ./features.list
MERT starting values and ranges for random generation:
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
LexicalReordering0 = 0.300 ( 0.00 .. 1.00)
Distortion0 = 0.300 ( 0.00 .. 1.00)
LM0 = 0.500 ( 0.00 .. 1.00)
WordPenalty0 = -1.000 ( 0.00 .. 1.00)
PhrasePenalty0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
TranslationModel0 = 0.200 ( 0.00 .. 1.00)
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: LexicalReordering0=0.300000
featlist: Distortion0=0.300000
featlist: LM0=0.500000
featlist: WordPenalty0=-1.000000
featlist: PhrasePenalty0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
featlist: TranslationModel0=0.200000
Saved: ./run1.moses.ini
Normalizing lambdas: 0.300000 0.300000 0.300000 0.300000 0.300000 0.300000
0.300000 0.500000 -1.000000 0.200000 0.200000 0.200000 0.200000 0.200000
DECODER_CFG = -weight-overwrite 'PhrasePenalty0= 0.043478 WordPenalty0=
-0.217391 TranslationModel0= 0.043478 0.043478 0.043478 0.043478
Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217 0.065217
0.065217 0.065217 0.065217 0.065217'
Executing: /share/apps/NYUAD/mosesdecoder/3.0/bin/moses -v 0 -config
filtered/moses.ini -weight-overwrite 'PhrasePenalty0= 0.043478
WordPenalty0= -0.217391 TranslationModel0= 0.043478 0.043478 0.043478
0.043478 Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217
0.065217 0.065217 0.065217 0.065217 0.065217' -n-best-list
run1.best100.out 100 distinct -input-file
/scratch/ae1541/unComparableCorpora/evaluation/silverPhrases/eng-lev/train.bolt.lev.eng
> run1.out
Executing: /share/apps/NYUAD/mosesdecoder/3.0/bin/moses -v 0 -config
filtered/moses.ini -weight-overwrite 'PhrasePenalty0= 0.043478
WordPenalty0= -0.217391 TranslationModel0= 0.043478 0.043478 0.043478
0.043478 Distortion0= 0.065217 LM0= 0.108696 LexicalReordering0= 0.065217
0.065217 0.065217 0.065217 0.065217 0.065217' -n-best-list
run1.best100.out 100 distinct -input-file
/scratch/ae1541/unComparableCorpora/evaluation/silverPhrases/eng-lev/train.bolt.lev.eng
> run1.out
Initializing LexicalReordering..
Loading table into memory...done.
Thanks,
--
Alex Erdmann
PhD Student in Linguistics at The Ohio State University
Visiting Scholar at NYU Abu Dhabi
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
