hiya
using a complicated factored model by itself usually result in very bad
scores. To get good scores, you should use it in addition to a simple
model. Our paper talks a little about this:
http://homepages.inf.ed.ac.uk/pkoehn/publications/emnlp2007-factored.pdf
The ini file would look something like:
[mapping]
0 T 0
0 T 1
0 G 0
1 T 2
[ttable-file]
1 0 1 5 file1
1 1 3 5 file2
1 0,1 1,2,3 5 file3
On 08/01/2012 20:46, ?? wrote:
hi all
i have a problem need your help; i training chinese-mongolian
translation. using factored model . chinese have two factors
(surface,pos) mongolia have 5 factors(surface,stem , pos,morpheme and
morpheme's pos). my training command like this:
/home/srgl/mtworkdir/bin/moses-scripts/scripts-20110907-2221/training/train-model.perl
-corpus /home/meirong-moses/fac-sur+pos+stem/corpus/clean-fac -root-dir
/home/meirong-moses/fac-sur+pos+stem/ -f ch -e mg -alignment
grow-diag-final-and -reordering msd-bidirectional-fe -lm
0:3:/home/meirong-moses/fac-sur+pos+stem/corpus/surface-lm.bin:0 -lm
1:3:/home/meirong-moses/fac-sur+pos+stem/corpus/stem-lm.bin:0 -lm
3:3:/home/meirong-moses/fac-sur+pos+stem/corpus/pos-lm.bin:0
-translation-factors 0-1+1-3 -reordering-factors 0-1+1-3 -generation-factors
1,3-0 -decoding-steps t0,t1,g0
/home/meirong-moses/fac-sur+pos+stem/model/reordering-table.0-1.wbe-msd-bidirectionhome/srgl/mtworkdir/bin/moses-scripts/scripts-20110907-2221/training/mert-moses.pl
/home/meirong-moses/fac-sur+pos+stem/tuning/tes!
t.ch /home/meirong-moses/fac-sur+pos+stem/tuning/re
/home/srgl/mtworkdir/moses/moses-cmd/src/moses
/home/meirong-moses/fac-sur+pos+stem/model/moses.ini --working-dir
/home/meirong-moses/fac-sur+pos+stem/tuning/mert --rootdir
/home/srgl/mtworkdir/bin/moses-scripts/scripts-20110907-2221/ --decoder-flags "-v
1">& /home/meirong-moses/fac-sur+pos+stem/tuning/mert.out
moses.ini file like this:
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
1
# mapping steps
[mapping]
0 T 0
0 T 1
0 G 0
# translation tables: table type (hierarchical(0), textual (0), binary (1)),
source-factors, target-factors, number of scores, file
# OLD FORMAT is still handled for back-compatibility
# OLD FORMAT translation tables: source-factors, target-factors, number of
scores, file
# OLD FORMAT a binary table type (1) is assumed
[ttable-file]
1 0 1 5 /home/meiron!
g-moses/fac-sur+pos+stem/model/phrase-table.0-1
1 1 3 5 /home/meiro
ng-moses/fac-sur+pos+stem/model/phrase-table.1-3
# generation models: source-factors, target-factors, number-of-weights, filename
[generation-file]
1,3 0 2 /home/meirong-moses/fac-sur+pos+stem/model/generation.1,3-0
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
0 0 3 /home/meirong-moses/fac-sur+pos+stem/corpus/3-surface.lm
0 1 3 /home/meirong-moses/fac-sur+pos+stem/corpus/5-stem.lm
0 3 3 /home/meirong-moses/fac-sur+pos+stem/corpus/7-pos.lm
# limit on how many phrase translations e for each phrase f are loaded
# 0 = all elements loaded
[ttable-limit]
50
50
# distortion (reordering) files
[distortion-file]
0-1 wbe-msd-bidirectional-fe-allff 6
/home/meirong-moses/fac-sur+pos+stem/model/reordering-table.0-1.wbe-msd-bidirectional-fe
1-3 wbe-msd-bidirectional-fe-allff 6
/home/meirong-moses/fac-sur+pos+stem/model/reordering-table.1-3.wbe-msd-bidirectional-fe
# distortion (reord!
ering) weight
[weight-d]
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
0.3
# language model weights
[weight-l]
0.1667
0.1667
0.1667
# translation model weights
[weight-t]
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
0.2
# generation model weights
[weight-generation]
0.3
0
# word penalty
[weight-w]
-1
[distortion-limit]
6
after filtering the moses.ini file is :
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
1
# mapping steps
[mapping]
0 T 0
0 T 1
0 G 0
# translation tables: table type (hierarchical(0), textual (0), binary (1)),
source-factors, target-factors, number of scores, file
# OLD FORMAT is still handled for back-compatibility
# OLD FORMAT translation tables: source-factors, target-factors, number !
of scores, file
# OLD FORMAT a binary table type (1) is assumed*[ttable-file]
1 0 1 5 /home/meirong-moses/fac-sur+pos+stem/model/phrase-table.0-1
1 1 3 5 /home/meirong-moses/fac-sur+pos+stem/model/phrase-table.1-3
# generation models: source-factors, target-factors, number-of-weights, filename
[generation-file]
1,3 0 2 /home/meirong-moses/fac-sur+pos+stem/model/generation.1,3-0
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
0 0 3 /home/meirong-moses/fac-sur+pos+stem/corpus/3-surface.lm
0 1 3 /home/meirong-moses/fac-sur+pos+stem/corpus/5-stem.lm
0 3 3 /home/meirong-moses/fac-sur+pos+stem/corpus/7-pos.lm
# limit on how many phrase translations e for each phrase f are loaded
# 0 = all elements loaded
[ttable-limit]
20
0
# distortion (reordering) files
[distortion-file]
0-1 wbe-msd-bidirectional-fe-allff 6
/home/meirong-moses/fac-sur+pos+stem/evaluation/filtered.test2012/reordering-table.0-1.wbe-msd-bidirectional-fe
1-3 wbe-msd-bidirecti!
onal-fe-allff 6
/home/meirong-moses/fac-sur+pos+stem/evaluation/filtered.test2012/reordering-table.1-3.wbe-msd-bidirectional-fe
# distortion (reordering) weight
[weight-d]
-0.020171
0.037719
-0.031663
0.007208
0.025879
0.022314
0.190458
-0.004907
-0.016917
0.035219
0.002975
0.021972
-0.062569
# language model weights
[weight-l]
0.037882
0.020446
0.009571
# translation model weights
[weight-t]
0.011354
0.003790
0.009319
-0.000823
-0.230286
0.006994
0.000386
0.101978
-0.042747
0.005346
# generation model weights
[weight-generation]
0.012161
0.013079
# word penalty
[weight-w]
0.013869
[distortion-limit]
6
my problem is that run4.cmert.log=0.5647 but finally translation test's bleu score is very small =0.1014
i don't know why ? need everyone's help thank you!
*
**
*
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
*
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
