Hi Chris, Thanks a million! that saved me several hours of going through a head scratching session. I catenated the both language Bible data through "sed" to remove the special characters and the training script ran through to completion. Thanks again.
Jide Otuyelu oroede.org ----- Original Message ---- From: Chris Dyer <[EMAIL PROTECTED]> To: jide Otuyelu <[EMAIL PROTECTED]> Cc: [email protected] Sent: Tuesday, August 19, 2008 12:39:06 PM Subject: Re: [Moses-support] train-factored-phrase-model.perl Couldn't find factor 0 in token That usually means that you have the '|' symbol in your input data. One way of avoiding this is using the --factor-delimiter option and setting it to some random string like +++! . You can also remove the | from your training data. Chris On Tue, Aug 19, 2008 at 8:27 AM, jide Otuyelu <[EMAIL PROTECTED]> wrote: > I am trying to run the training script but keep getting the following error > : > (1.0.5) reducing factors to produce work/corpus/new.tok.lowered.0-0.yo @ > Tue Aug 19 00:56:21 EDT 2008 > Couldn't find factor 0 in token "|" in work/corpus/new.tok.lowered.yo LINE > 2074 at /usr/bin/train-factored-phrase-model.perl line 395, <IN> line 2074. > Rechecked the data which is from the English and corresponding Yoruba > Bibles, and the they seem okay. Everything runs fine until this stage. Here > is the command I ran: > # train-factored-phrase-model.perl --scripts-root-dir > /usr/share/moses/scripts -root-dir work --corpus work/corpus/new.tok.lowered > -f yo -e en --alignment grow-diag-final-and --reordering > msd-bidirectional-fe --lm > 0:3:/home/jide/lab5/work/lm/new.tok.lowered.en.lm:1 > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
