hi all i have a problem need your help; i training chinese-mongolian translation. using factored model . chinese have two factors (surface,pos) mongolia have 5 factors(surface,stem , pos,morpheme and morpheme's pos). my training command like this:
/home/srgl/mtworkdir/bin/moses-scripts/scripts-20110907-2221/training/train-model.perl -corpus /home/meirong-moses/fac-sur+pos+stem/corpus/clean-fac -root-dir /home/meirong-moses/fac-sur+pos+stem/ -f ch -e mg -alignment grow-diag-final-and -reordering msd-bidirectional-fe -lm 0:3:/home/meirong-moses/fac-sur+pos+stem/corpus/surface-lm.bin:0 -lm 1:3:/home/meirong-moses/fac-sur+pos+stem/corpus/stem-lm.bin:0 -lm 3:3:/home/meirong-moses/fac-sur+pos+stem/corpus/pos-lm.bin:0 -translation-factors 0-1+1-3 -reordering-factors 0-1+1-3 -generation-factors 1,3-0 -decoding-steps t0,t1,g0 /home/meirong-moses/fac-sur+pos+stem/model/reordering-table.0-1.wbe-msd-bidirectionhome/srgl/mtworkdir/bin/moses-scripts/scripts-20110907-2221/training/mert-moses.pl /home/meirong-moses/fac-sur+pos+stem/tuning/test.ch /home/meirong-moses/fac-sur+pos+stem/tuning/re /home/srgl/mtworkdir/moses/moses-cmd/src/moses /home/meirong-moses/fac-sur+pos+stem/model/moses.ini --working-dir /home/meirong-moses/fac-sur+pos+stem/tuning/mert --rootdir /home/srgl/mtworkdir/bin/moses-scripts/scripts-20110907-2221/ --decoder-flags "-v 1" >& /home/meirong-moses/fac-sur+pos+stem/tuning/mert.out moses.ini file like this: ######################### ### MOSES CONFIG FILE ### ######################### # input factors [input-factors] 0 1 # mapping steps [mapping] 0 T 0 0 T 1 0 G 0 # translation tables: table type (hierarchical(0), textual (0), binary (1)), source-factors, target-factors, number of scores, file # OLD FORMAT is still handled for back-compatibility # OLD FORMAT translation tables: source-factors, target-factors, number of scores, file # OLD FORMAT a binary table type (1) is assumed [ttable-file] 1 0 1 5 /home/meirong-moses/fac-sur+pos+stem/model/phrase-table.0-1 1 1 3 5 /home/meirong-moses/fac-sur+pos+stem/model/phrase-table.1-3 # generation models: source-factors, target-factors, number-of-weights, filename [generation-file] 1,3 0 2 /home/meirong-moses/fac-sur+pos+stem/model/generation.1,3-0 # language models: type(srilm/irstlm), factors, order, file [lmodel-file] 0 0 3 /home/meirong-moses/fac-sur+pos+stem/corpus/3-surface.lm 0 1 3 /home/meirong-moses/fac-sur+pos+stem/corpus/5-stem.lm 0 3 3 /home/meirong-moses/fac-sur+pos+stem/corpus/7-pos.lm # limit on how many phrase translations e for each phrase f are loaded # 0 = all elements loaded [ttable-limit] 50 50 # distortion (reordering) files [distortion-file] 0-1 wbe-msd-bidirectional-fe-allff 6 /home/meirong-moses/fac-sur+pos+stem/model/reordering-table.0-1.wbe-msd-bidirectional-fe 1-3 wbe-msd-bidirectional-fe-allff 6 /home/meirong-moses/fac-sur+pos+stem/model/reordering-table.1-3.wbe-msd-bidirectional-fe # distortion (reordering) weight [weight-d] 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 0.3 # language model weights [weight-l] 0.1667 0.1667 0.1667 # translation model weights [weight-t] 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 0.2 # generation model weights [weight-generation] 0.3 0 # word penalty [weight-w] -1 [distortion-limit] 6 after filtering the moses.ini file is : ######################### ### MOSES CONFIG FILE ### ######################### # input factors [input-factors] 0 1 # mapping steps [mapping] 0 T 0 0 T 1 0 G 0 # translation tables: table type (hierarchical(0), textual (0), binary (1)), source-factors, target-factors, number of scores, file # OLD FORMAT is still handled for back-compatibility # OLD FORMAT translation tables: source-factors, target-factors, number of scores, file # OLD FORMAT a binary table type (1) is assumed [ttable-file] 1 0 1 5 /home/meirong-moses/fac-sur+pos+stem/model/phrase-table.0-1 1 1 3 5 /home/meirong-moses/fac-sur+pos+stem/model/phrase-table.1-3 # generation models: source-factors, target-factors, number-of-weights, filename [generation-file] 1,3 0 2 /home/meirong-moses/fac-sur+pos+stem/model/generation.1,3-0 # language models: type(srilm/irstlm), factors, order, file [lmodel-file] 0 0 3 /home/meirong-moses/fac-sur+pos+stem/corpus/3-surface.lm 0 1 3 /home/meirong-moses/fac-sur+pos+stem/corpus/5-stem.lm 0 3 3 /home/meirong-moses/fac-sur+pos+stem/corpus/7-pos.lm # limit on how many phrase translations e for each phrase f are loaded # 0 = all elements loaded [ttable-limit] 20 0 # distortion (reordering) files [distortion-file] 0-1 wbe-msd-bidirectional-fe-allff 6 /home/meirong-moses/fac-sur+pos+stem/evaluation/filtered.test2012/reordering-table.0-1.wbe-msd-bidirectional-fe 1-3 wbe-msd-bidirectional-fe-allff 6 /home/meirong-moses/fac-sur+pos+stem/evaluation/filtered.test2012/reordering-table.1-3.wbe-msd-bidirectional-fe # distortion (reordering) weight [weight-d] -0.020171 0.037719 -0.031663 0.007208 0.025879 0.022314 0.190458 -0.004907 -0.016917 0.035219 0.002975 0.021972 -0.062569 # language model weights [weight-l] 0.037882 0.020446 0.009571 # translation model weights [weight-t] 0.011354 0.003790 0.009319 -0.000823 -0.230286 0.006994 0.000386 0.101978 -0.042747 0.005346 # generation model weights [weight-generation] 0.012161 0.013079 # word penalty [weight-w] 0.013869 [distortion-limit] 6 my problem is that run4.cmert.log=0.5647 but finally translation test's bleu score is very small =0.1014 i don't know why ? need everyone's help thank you!
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
