You are using one factor (with index 0) but in the configuration file you are loading a LM for the factor with index 1 and no one with index 0
the correct line is 1 0 5 /home/loki/Downloads/irstlm-5.70.04/scripts/train.irstlm.gz Futhermore, you are loading a textual version of the LM, and this take a long time Finished loading LanguageModels : [2945.000] second I strongly suggest you to compile it n binary format using the command compile-lm of tIRSTLM toolkit, as follows (with the path) compile-lm rain.irstlm.gz train.irstlm.blm see the manual for more details: the Moses configuration file should be changed accordingly (with the correct path) 1 0 5 train.irstlm.blm You spend some time to binarize the LM once, but then loading it in Moses will be very fast. best Nicola Bertoldi On Apr 11, 2012, at 8:58 AM, Loki Cheng wrote: Hi, everyone, I want to use the script "mert-moses.pl<http://mert-moses.pl/>" to tuning the parameters in the "mose.ini" file, but the running process fail eventually, here is the log information: --------------------------------------------------------------------------------------------------------------------------------- loki@ubuntu:~/Downloads/moses/scripts/target/scripts-20120222-0301/training$ ./mert-moses.pl<http://mert-moses.pl/> /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/development_corpus/input/dev.clean.en /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/development_corpus/references/dev.zh.utf8 /home/loki/Downloads/moses/moses-cmd/src/moses /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/model/moses.ini --mertdir /home/loki/Downloads/moses/mert --working-dir /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/tunning --rootdir $SCRIPTS_ROOTDIR main::create_extractor_script() called too early to check prototype at ./mert-moses.pl<http://mert-moses.pl/> line 666. Using SCRIPTS_ROOTDIR: /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301 Assuming the tables are already filtered, reusing filtered/moses.ini Using cached features list: ./features.list MERT starting values and ranges for random generation: d = 0.600 ( 0.00 .. 1.00) lm = 0.500 ( 0.00 .. 1.00) w = -1.000 ( 0.00 .. 1.00) tm = 0.200 ( 0.00 .. 1.00) tm = 0.200 ( 0.00 .. 1.00) tm = 0.200 ( 0.00 .. 1.00) tm = 0.200 ( 0.00 .. 1.00) tm = 0.200 ( 0.00 .. 1.00) run 1 start at Tue Apr 10 21:15:19 PDT 2012 Parsing --decoder-flags: || Saving new config to: ./run1.moses.ini Saved: ./run1.moses.ini (1) run decoder to produce n-best lists params = Normalizing lambdas: 0.600000 0.500000 -1.000000 0.200000 0.200000 0.200000 0.200000 0.200000 DECODER_CFG = -w -0.322581 -lm 0.161290 -d 0.193548 -tm 0.064516 0.064516 0.064516 0.064516 0.064516 decoder_config = -w -0.322581 -lm 0.161290 -d 0.193548 -tm 0.064516 0.064516 0.064516 0.064516 0.064516 Executing: /home/loki/Downloads/moses/moses-cmd/src/moses -config filtered/moses.ini -inputtype 0 -w -0.322581 -lm 0.161290 -d 0.193548 -tm 0.064516 0.064516 0.064516 0.064516 0.064516 -n-best-list run1.best100.out 100 -input-file /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/development_corpus/input/dev.clean.en > run1.out Defined parameters (per moses.ini or switch): config: filtered/moses.ini distortion-limit: 6 input-factors: 0 input-file: /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/development_corpus/input/dev.clean.en inputtype: 0 lmodel-file: 1 1 5 /home/loki/Downloads/irstlm-5.70.04/scripts/train.irstlm.gz mapping: 0 T 0 n-best-list: run1.best100.out 100 ttable-file: 1 0 0 5 /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/model/phrase-table ttable-limit: 20 weight-d: 0.193548 weight-l: 0.161290 weight-t: 0.064516 0.064516 0.064516 0.064516 0.064516 weight-w: -0.322581 Loading lexical distortion models...have 0 models Start loading LanguageModel /home/loki/Downloads/irstlm-5.70.04/scripts/train.irstlm.gz : [0.000] seconds In LanguageModelIRST::Load: nGramOrder = 5 Language Model Type of /home/loki/Downloads/irstlm-5.70.04/scripts/train.irstlm.gz is 1 iARPA loadtxt_ram() 1-grams: reading 443439 entries done level1 2-grams: reading 8644803 entries .done level2 3-grams: reading 48970019 entries .........done level3 4-grams: reading 103946151 entries ....................done level4 5-grams: reading 140766631 entries ............................done level5 done OOV code is 443438 OOV code is 443438 IRST: m_unknownId=443438 Finished loading LanguageModels : [2945.000] seconds Start loading PhraseTable /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/model/phrase-table : [2945.000] seconds filePath: /home/loki/Downloads/moses/scripts/target/scripts-20120222-0301/training/model/phrase-table Finished loading phrase tables : [2945.000] seconds Created input-output object : [2945.000] seconds Translating line 0 in thread id 0 Translating: What I 'm going to show you first , as quickly as I can , is some foundational work , some new technology that we brought to Microsoft as part of an acquisition almost exactly a year ago . This is Seadragon . reading bin ttable size of OFF_T 8 binary phrasefile loaded, default OFF_T: -1 Collecting options took 14.310 seconds Segmentation fault Exit code: 139 The decoder died. CONFIG WAS -w -0.322581 -lm 0.161290 -d 0.193548 -tm 0.064516 0.064516 0.064516 0.064516 0.064516 ------------------------------------------------------------------------------------ I don't know what's wrong with it, is the physical memory not enough? If yes, how can I solve this issue? Any suggestions will be appreciated Best regards Moonloki _______________________________________________ Moses-support mailing list [email protected]<mailto:[email protected]> http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
