I'm tuning a new ES-EN translation model. The tables were trained with about 1.75 million pairs from the Europarl v6 data using Moses w/KenLM SVN rev 4011 and IRSTLM 5.60.03. The attachments herewith include the run1.moses.ini file and the output log from mert-moses.pl that also includes the command line.
If I run from a terminal command line: "$ moses -f run1.moses.ini < mert.es > run0.out" Moses terminates with the same error in the mert-moses.pl.log file. Piping any other file into moses as above also terminates with the same error. I also removed the [threads] value to run single threaded, and again, same terminal error. If I run in a terminal: "$ moses -f run1.moses.ini" then, copy lines from the mert.es file and paste into the terminal, they translate fine. Also, three days ago, a tuning/training session with the same moses build competed fine. It used different training corpus started from the same data and used clean-corpus-n.perl with max tokens = 78. This corpus uses max tokens = 65 and extracted a different 2500 pairs for tuning. Those are the only differences in the two training corpora. I'm baffled. Any suggestions? Tom
COMMAND LINE:
nohup nice -10 mert-moses.pl \
/opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5/mert.es \
/opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5/mert.en \
/usr/local/bin/moses \
/opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5/run0.moses.ini \
--working-dir /opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5 \
--rootdir /usr/local/lib/mosesdecoder/scripts \
--mertdir /usr/local/bin \
--nbest 100 \
--no-filter-phrase-table \
--maximum-iterations 25 \
--decoder-flags "-v 0 -threads 4"
Using SCRIPTS_ROOTDIR: /usr/local/lib/mosesdecoder/scripts
Executing: mkdir -p /opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5
run 1 start at Tue Jun 28 20:55:54 ICT 2011
Parsing --decoder-flags: |-v 0 -threads 4|
Saving new config to: ./run1.moses.ini
Saved: ./run1.moses.ini
Normalizing lambdas: -1 0.5000 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.20 0.20 0.20 0.20 0.20
DECODER_CFG = -w %.6f -lm %.6f -d %.6f %.6f %.6f %.6f %.6f %.6f %.6f -tm %.6f %.6f %.6f %.6f %.6f
values = -0.217391304347826 0.108695652173913 0.0652173913043478 0.0652173913043478 0.0652173913043478 0.0652173913043478 0.0652173913043478 0.0652173913043478 0.0652173913043478 0.0434782608695652 0.0434782608695652 0.0434782608695652 0.0434782608695652 0.0434782608695652
Executing: /usr/local/bin/moses -v 0 -threads 4 -config /opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5/run0.moses.ini -inputtype 0 -w -0.217391 -lm 0.108696 -d 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 -tm 0.043478 0.043478 0.043478 0.043478 0.043478 -n-best-list run1.best100.out 100 -input-file /opt/library/TRAININGS/merts/mert-s=es-t=en-p=europarl6_ce-a=giza-g=7-l=europarl6_ce-T=irstlmken-n=5/mert.es > run1.out
(1) run decoder to produce n-best lists
params = -v 0 -threads 4
decoder_config = -w -0.217391 -lm 0.108696 -d 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 -tm 0.043478 0.043478 0.043478 0.043478 0.043478
Creating lexical reordering...
weights: 0.065 0.065 0.065 0.065 0.065 0.065
binary file loaded, default OFF_T: -1
Translating line 0 in thread id 140258570172160
Translating line 1 in thread id 140258578564864
reading bin ttable
size of OFF_T 8
Translating line 2 in thread id 140258561779456
Translating line 3 in thread id 140258553386752
reading bin ttable
size of OFF_T 8
reading bin ttable
size of OFF_T 8
reading bin ttable
size of OFF_T 8
[ERROR] Malformed input at
Expected input to have words composed of 1 factor(s) (form FAC1|FAC2|...)
but instead received input with 0 factor(s).
Aborted
Exit code: 134
The decoder died. CONFIG WAS -w -0.217391 -lm 0.108696 -d 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 0.065217 -tm 0.043478 0.043478 0.043478 0.043478 0.043478
run1.moses.ini
Description: application/wine-extension-ini
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
