Hi, I've been trying to test the Fuzzy Match Rule Table for hierarchical models, but when I run MERT, I only get syntax errors. I'm fairly new in hierarchical models, I think my moses.ini config is correct, but I may be doing something wrong. If I don't use the PhraseDictionaryFuzzyMatch, it works properly.
I saw a similar thread ( http://permalink.gmane.org/gmane.comp.nlp.moses.user/12239), but I can't even reach the translation process, it crashes before. I'd appreciate any help. This is the trace of the crash: (6.6) consolidating the two halves @ Thu Sep 10 12:59:50 CEST 2015 Executing: /home/lvapeab/smt/software/mosesdecoder/scripts/../bin/consolidate /tmp/moses.XQR9TF/fuzzyMatchFile.pt.half.f2e.gz /tmp/moses.XQR9TF/fuzzyMatchFile.pt.half.e2f.gz /dev/stdout --Hierarchical | gzip -c > /tmp/moses.XQR9TF/fuzzyMatchFile.pt.gz Consolidate v2.0 written by Philipp Koehn consolidating direct and indirect rule tables processing hierarchical rules Executing: rm -f /tmp/moses.XQR9TF/fuzzyMatchFile.pt.half.* Start loading fuzzy-match phrase model : [1.245] seconds terminate called after throwing an instance of 'util::Exception' what(): moses/TranslationModel/RuleTable/PhraseDictionaryFuzzyMatch.cpp:235 in virtual void Moses::PhraseDictionaryFuzzyMatch::InitializeForInput(const ttasksptr&) threw util::Exception'. Syntax error at /tmp/moses.XQR9TF/fuzzyMatchFile.pt.gz:0 Aborted (core dumped) Exit code: 134 The decoder died. CONFIG WAS -weight-overwrite 'TranslationModel0= 0.057143 0.057143 0.057143 0.057143 TranslationModel1= 0.285714 LM0= 0.142857 WordPenalty0= -0.285714 PhrasePenalty0= 0.057143' The file which gives the syntax error (/tmp/moses.2pkyhw/fuzzyMatchFile.pt.gz) contains: nos hemos de marchar el jueves cinco de enero por la tarde . [X] ||| should leave on Thursday January the fifth in the afternoon . [X] ||| -nan -nan ||| 0-0 1-0 1-1 10-9 11-10 2-1 3-1 3-2 4-3 5-4 5-6 6-7 7-5 7-8 8-5 9-8 ||| 0 0 0 ||| ||| (Maybe it fails because of the -nan?) Finally, my moses.ini: ######################### ### MOSES CONFIG FILE ### ######################### # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 1 T 1 [cube-pruning-pop-limit] 1000 [non-terminals] X [search-algorithm] 3 [inputtype] 3 [max-chart-span] 20 1000 # feature functions [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryMemory name=TranslationModel0 num-features=4 path=/home/lvapeab/smt/tasks/turista/esen/fuzzy/model/rule-table.gz input-factor=0 output-factor=0 PhraseDictionaryMemory name=TranslationModel1 num-features=1 path=/home/lvapeab/smt/tasks/turista/esen/fuzzy/model/glue-grammar input-factor=0 output-factor=0 tuneable=true KENLM name=LM0 factor=0 path=/home/lvapeab/smt/tasks/turista/esen/LM/5gram.en order=5 PhraseDictionaryFuzzyMatch source=/home/lvapeab/smt/tasks/turista/DATA/ training.es target=/home/lvapeab/smt/tasks/turista/DATA/training.en alignment=/home/lvapeab/smt/tasks/turista/esen/fuzzy/model/aligned.grow-diag-final-and num-features=0 # dense weights for feature functions [weight] # The default weights are NOT optimized for translation quality. You MUST tune the weights. # Documentation for tuning is here: http://www.statmt.org/moses/?n=FactoredTraining.Tuning UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2 TranslationModel0= 0.2 0.2 0.2 0.2 TranslationModel1= 1 LM0= 0.5 Thank you in advance. Cheers, -- Álvaro.
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
