Hi, tokenization of the reference of evaluation sets is only needed for some of the analysis steps (there must be references to the script analysis.perl in your config file).
To determine, why the step crashed, you should look at the file steps/*/EVALUATION:test:tokenize-reference.*.STDERR where * is the step number. -phi On Mon, Feb 14, 2011 at 1:02 PM, amin farajian <[email protected]> wrote: > Hello Dear Koehn, > During training the system, I faced another problem. > In the first steps, ems produces this error: > > executing > /home/amin/mt/work/Basic/3lm/steps/1/EVALUATION_test_tokenize-reference.1 > via sh (2) > > executing /home/amin/mt/work/Basic/3lm/steps/1/LM_toy_binarize.1 via --- on > hold > > executing > /home/amin/mt/work/Basic/3lm/steps/1/EVALUATION_test_input-from-sgm.1 via > --- on hold > > step EVALUATION:test:tokenize-reference crashed > > But it continues without any other problem. I think this is because I'm > using Persian-English language pairs and there is no tokenizer for Persian. > But I wanted to know that can this crash affect the training procedure? _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
