are you sure the parallel data is encoded in UTF8? Was it tokenized, cleaned and escaped by the Moses scripts or by another external script?
Can you please send me you EMS config file too On 7 December 2013 14:03, amir haghighi <[email protected]> wrote: > Hi, > > I have also the same problem in evaluation step with EMS and I would be > thankful if you could help me. > the lexical reordering file is emtpy and the log of the output in > evaluation_test_filter.2.stderr is: > > Using SCRIPTS_ROOTDIR: > /opt/tools/mosesdecoder-RELEASE-1.0/mosesdecoder-RELEASE-1.0/scripts > (9) create moses.ini @ Sat Dec 7 04:50:15 PST 2013 > Executing: mkdir -p /opt/tools/workingEms/evaluation/test.filtered.2 > Considering factor 0 > Considering factor 0 > filtering /opt/tools/workingEms/model/phrase-table.2 -> > /opt/tools/workingEms/evaluation/test.filtered.2/phrase-table.0-0.1.1... > 0 of 2197240 phrases pairs used (0.00%) - note: max length 10 > binarizing...cat > /opt/tools/workingEms/evaluation/test.filtered.2/phrase-table.0-0.1.1 | > LC_ALL=C sort -T /opt/tools/workingEms/evaluation/test.filtered.2 | > > /opt/tools/mosesdecoder-RELEASE-1.0/mosesdecoder-RELEASE-1.0/bin/processPhraseTable > -ttable 0 0 - -nscores 5 -out > /opt/tools/workingEms/evaluation/test.filtered.2/phrase-table.0-0.1.1 > processing ptree for stdin > Segmentation fault (core dumped) > filtering > /opt/tools/workingEms/model/reordering-table.2.wbe-msd-bidirectional-fe.gz > -> > > /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe... > 0 of 2197240 phrases pairs used (0.00%) - note: max length 10 > > binarizing.../opt/tools/mosesdecoder-RELEASE-1.0/mosesdecoder-RELEASE-1.0/bin/processLexicalTable > -in > > /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe > -out > > /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe > processLexicalTable v0.1 by Konrad Rawlik > processing > > /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe > to > > /opt/tools/workingEms/evaluation/test.filtered.2/reordering-table.2.wbe-msd-bidirectional-fe.* > ERROR: empty lexicalised reordering file > > > > Barry Haddow <bhaddow@...> writes: > > > > > Hi Irene > > > > > But the output is empty. And the errors are 1. segmentation fault > > 2. error: empty lexicalized > > > reordering file > > > > Is this lexicalised reordering file empty then? > > > > It would be helpful if you could post the full log of the output when > > your run the filter command, > > > > cheers - Barry > > > > On 26/10/12 17:59, Irene Huang wrote: > > > Hi, I have trained and tuned the model, now I am using > > > > > > ~/mosesdecoder/scripts/training/filter-model-given-input.pl > > > <http://filter-model-given-input.pl> filtered-newstest2011 > > > mert-work/moses.ini ~/corpus/newstest2011.true.fr > > > <http://newstest2011.true.fr> \ > > > -Binarizer ~/mosesdecoder/bin/processPhraseTable > > > > > > to filter the phrase table. > > > > > > But the output is empty. And the errors are 1. segmentation fault > > > 2. error: empty lexicalized reordering file > > > > > > So does this mean it's out of memory error? > > > > > > Thanks > > > > > > > > > _______________________________________________ > > > Moses-support mailing list > > > Moses-support@... > > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
