Hi All, I remember facing the same issue sometime last year. I had posted into this list and got a reply from Barry (Haddow). Here is a link to the archive
http://comments.gmane.org/gmane.comp.nlp.moses.user/4383 Daniel has correctly pointed out the issue to be of using --filtercmd. Be default when using MERT for hierarchical models, the -Hierarchical switch is not passed to the filterring script from with mert-moses.pl Barry suggested using something like: mert-moses.perl ... --filtercmd 'filter-model-given-input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical' This sorted out my problems, so hope this helps. Regards, Pratyush On Mon, Jan 23, 2012 at 1:30 PM, Daniel Schaut <[email protected]>wrote: > Hi Rasul, > > I experienced exactly the same issue two or four weeks ago: My tuning set > contained an odd number of lines, e.g. the target side included 2000 lines > and the source side 2003 lines. Subsequently, I removed the remaining lines > on the source side and the issue was gone. If I remember correctly, I also > filtered the tuning set using --filtercmd. Hope this helps! > > Best, > Daniel > > -----Ursprüngliche Nachricht----- > Von: [email protected] [mailto:[email protected]] > Im > Auftrag von [email protected] > Gesendet: Montag, 23. Januar 2012 00:57 > An: [email protected] > Betreff: [Moses-support] Segmentation fault in tuning with chart decoder > > Hi all, > > I have trained a hierarchical model and trying to tune it using mert. > I'm getting this segmentation fault error in the early stages. > Following is the log and the command I'm using. Your idea is much > appreciated. > > Best Wishes, > Rasul. > > --------------------------------------------------- Log > Executing: /tools/moses/moses-chart-cmd/src/moses_chart -v 0 -config > filtered/moses.ini -inputtype 0 -show-weights > ./features.list In > LanguageModelIRST::Load: nGramOrder = 5 Language Model Type of > /en-fr/lm/irstlmse.5grams.lm.fr is 1 \data\ > loadtxt_ram() > 1-grams: reading 176493 entries > done level1 > 2-grams: reading 1332577 entries > done level2 > 3-grams: reading 1402000 entries > done level3 > 4-grams: reading 1836276 entries > done level4 > 5-grams: reading 1830829 entries > done level5 > done > OOV code is 176492 > OOV code is 176492 > sh: line 1: 7057 Segmentation fault > /tools/moses/moses-chart-cmd/src/moses_chart -v 0 -config > filtered/moses.ini > -inputtype 0 -show-weights > ./features.list Exit code: 139 Failed to run > moses with the config filtered/moses.ini at > /tools/moses/scripts/training/mert-moses.pl line 1072. > r > > --------------------------------------------------- Command nohup nice > $SCRIPTS_ROOTDIR/training/mert-moses.pl > $EXP_ENFR/common/corpus/dev.tok.lower.en > $EXP_ENFR/corpus/dev.tok.lower.fr > $MOSES_ROOTDIR/moses-chart-cmd/src/moses_chart > $EXP_ENFR/model/moses.ini > --working-dir $EXP_ENFR/tuning/mert > --mertdir $MOSES_ROOTDIR/mert > --rootdir $SCRIPTS_ROOTDIR > --decoder-flags "-v 0" > > $EXP_ENFR/tuning/mert.log > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
