Thanks Pratyush, This solved the issue for me as well. Without using it, no filtered phrase table was generated (in filtered directory under mert work directory)
Best Regards, Rasul. Quoting Pratyush Banerjee <[email protected]>: > Hi All, > > I remember facing the same issue sometime last year. I had posted into this > list and got a reply from Barry (Haddow). Here is a link to the archive > > http://comments.gmane.org/gmane.comp.nlp.moses.user/4383 > > Daniel has correctly pointed out the issue to be of using --filtercmd. > Be default when using MERT for hierarchical models, the -Hierarchical > switch is not passed to the filterring script from with mert-moses.pl > > Barry suggested using something like: > > mert-moses.perl ... --filtercmd 'filter-model-given-input.pl > -Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical' > > This sorted out my problems, so hope this helps. > > Regards, > > Pratyush > > On Mon, Jan 23, 2012 at 1:30 PM, Daniel Schaut > <[email protected]>wrote: > >> Hi Rasul, >> >> I experienced exactly the same issue two or four weeks ago: My tuning set >> contained an odd number of lines, e.g. the target side included 2000 lines >> and the source side 2003 lines. Subsequently, I removed the remaining lines >> on the source side and the issue was gone. If I remember correctly, I also >> filtered the tuning set using --filtercmd. Hope this helps! >> >> Best, >> Daniel >> >> -----Ursprüngliche Nachricht----- >> Von: [email protected] [mailto:[email protected]] >> Im >> Auftrag von [email protected] >> Gesendet: Montag, 23. Januar 2012 00:57 >> An: [email protected] >> Betreff: [Moses-support] Segmentation fault in tuning with chart decoder >> >> Hi all, >> >> I have trained a hierarchical model and trying to tune it using mert. >> I'm getting this segmentation fault error in the early stages. >> Following is the log and the command I'm using. Your idea is much >> appreciated. >> >> Best Wishes, >> Rasul. >> >> --------------------------------------------------- Log >> Executing: /tools/moses/moses-chart-cmd/src/moses_chart -v 0 -config >> filtered/moses.ini -inputtype 0 -show-weights > ./features.list In >> LanguageModelIRST::Load: nGramOrder = 5 Language Model Type of >> /en-fr/lm/irstlmse.5grams.lm.fr is 1 \data\ >> loadtxt_ram() >> 1-grams: reading 176493 entries >> done level1 >> 2-grams: reading 1332577 entries >> done level2 >> 3-grams: reading 1402000 entries >> done level3 >> 4-grams: reading 1836276 entries >> done level4 >> 5-grams: reading 1830829 entries >> done level5 >> done >> OOV code is 176492 >> OOV code is 176492 >> sh: line 1: 7057 Segmentation fault >> /tools/moses/moses-chart-cmd/src/moses_chart -v 0 -config >> filtered/moses.ini >> -inputtype 0 -show-weights > ./features.list Exit code: 139 Failed to run >> moses with the config filtered/moses.ini at >> /tools/moses/scripts/training/mert-moses.pl line 1072. >> r >> >> --------------------------------------------------- Command nohup nice >> $SCRIPTS_ROOTDIR/training/mert-moses.pl >> $EXP_ENFR/common/corpus/dev.tok.lower.en >> $EXP_ENFR/corpus/dev.tok.lower.fr >> $MOSES_ROOTDIR/moses-chart-cmd/src/moses_chart >> $EXP_ENFR/model/moses.ini >> --working-dir $EXP_ENFR/tuning/mert >> --mertdir $MOSES_ROOTDIR/mert >> --rootdir $SCRIPTS_ROOTDIR >> --decoder-flags "-v 0" >> > $EXP_ENFR/tuning/mert.log >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support >> > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
