Hello all, I'm new to Moses and I've been trying to set up a baseline hierarchical/syntax Moses system. I got a regular system running using the "Baseline System" tutorial and now I'm following the Syntax Tutorial http://www.statmt.org/moses/?n=Moses.SyntaxTutorial
The error I get is when I try to tune my system using MERT: Here's my tuning command: > $ /home/ben/moses/mosesdecoder/scripts/training/mert-moses.pl \ > /data/ben/hiertest/yomiuri_tuning.ja /data/ben/hiertest/yomiuri_tuning.en \ > /home/ben/moses/mosesdecoder/bin/moses /data/ben/hiertest/model/moses.ini \ > --mertdir /home/ben/moses/mosesdecoder/bin \ > --maximum-iterations=10 The tail of the error log is: > exec: > /mnt/home/ben/moses/mosesdecoder/scripts/training/filter-model-given-input.pl > ./filtered /data/ben/hiertest/model/moses.ini > /data/ben/hiertest/yomiuri_tuning.ja > Executing: > /mnt/home/ben/moses/mosesdecoder/scripts/training/filter-model-given-input.pl > ./filtered /data/ben/hiertest/model/moses.ini > /data/ben/hiertest/yomiuri_tuning.ja > filterphrases.out 2> filterphrases.err > Asking moses for feature names and values from filtered/moses.ini > Executing: /home/ben/moses/mosesdecoder/bin/moses -config filtered/moses.ini > -inputtype 0 -show-weights > ./features.list > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > cube-pruning-pop-limit: 1000 > distortion-file: 0-0 wbe-msd-bidirectional-fe-allff 6 > /mnt/home/ben/moses/mert-work/filtered/reordering-table.wbe-msd-bidirectional-fe > input-factors: 0 > inputtype: 0 > lmodel-file: 8 0 5 /yew/ben/trans/nict.all.en.5.blm > mapping: 0 T 0 1 T 1 > max-chart-span: 20 1000 > non-terminals: X > search-algorithm: 3 > show-weights: > ttable-file: 6 0 0 5 > /mnt/home/ben/moses/mert-work/filtered/phrase-table.0-0.1.1.gz 6 0 0 1 > /data/ben/hiertest/model/glue-grammar > ttable-limit: 20 > weight-l: 0.5000 > weight-t: 0.20 0.20 0.20 0.20 0.20 1.0 > weight-w: -1 > /home/ben/moses/mosesdecoder/bin > ScoreProducer: WordPenalty start: 0 end: 1 > ScoreProducer: !UnknownWordPenalty start: 1 end: 2 > Loading lexical distortion models...have 1 models > ERROR:Lexicalized distortion model: Not enough weights, add to [weight-d] > Exit code: 1 > Failed to run moses with the config filtered/moses.ini at > /home/ben/moses/mosesdecoder/scripts/training/mert-moses.pl line 1169. It says in the Syntax Tutorial that training a hierarchical model is just a case of adding -hierarchical and a few other options to a normal Moses training command. Here is my training command. Modified from my working non-hierarchical system to add the one -hierarchical line: > $ /home/ben/moses/mosesdecoder/scripts/training/train-model.perl \ > -root-dir /data/ben/hiertest \ > -corpus /data/ben/hiertest/hiertest.corpus -f ja -e en \ > -hierarchical -glue-grammar -max-phrase-length 5 \ > -alignment grow-diag-final-and -reordering msd-bidirectional-fe \ > -lm 0:5:/yew/ben/trans/nict.all.en.5.blm:8 \ > -external-bin-dir=/home/ben/moses/mosesdecoder/bin/ I can add the -d flag with a bunch of weights to make the error go away, but that seems like a hack, and I don't know what weights to use anyway. Am I missing a step that sets up these weights in my moses.ini or something? I didn't notice or get any issues like this with the non-hierarchical system. Ben Humphreys PhD student Kurohashi-Kawahara Lab Kyoto University _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
