Hi all,
I am working on improving a baseline Japanese->English moses system I
developed. It achieves okay translation results already. I am
experimenting with adding part-of-speech factors, so I prepared
training data that looks like:
word0|pos0 word1|pos1 word2|pos2 ...
I tried two moses training runs on this factored data. In the first, I
wanted to use the POS factor for word alignment, translation, and
reordering, so I used these flags to the
train-factored-phrase-model.perl script:
--alignment-factors=0,1-0 --translation-factors=0,1-0
--reordering-factors=0,1-0 --decoding-steps=t0
The training finished successfully, MERT training also finished
successfully, and I could also translate and calculate BLEU score
successfully. (although it didn't give an improvement in BLEU score.
I'll look into why :-)
In the second run, I wanted to use the POS factor only in the
reordering model. I tried these flags:
--alignment-factors=0-0 --translation-factors=0-0
--reordering-factors=0,1-0 --decoding-steps=t0
The training finished successfully, but MERT training gave this error
(with some things snipped out):
Defined parameters (per moses.ini or switch):
config: filtered/moses.ini
distortion-file: 0,1-0 msd-bidirectional-fe 6
/afs/csail.mit.edu/u/j/jasonkb/moses/working-dirs/reordering-factored-pos/tuning/filtered/reordering-table.msd-bidirectional-fe.0.5.0,1-0
<snip>
ttable-file: 0 0 5
/afs/csail.mit.edu/u/j/jasonkb/moses/working-dirs/reordering-factored-pos/tuning/filtered/phrase-table.0-0
<snip>
[ERROR] Malformed input at
Expected input to have words composed of 1 factor(s) (form FAC1|FAC2|...)
but instead received input with 2 factor(s).
sh: line 1: 18883 Aborted
/afs/csail.mit.edu/u/j/jasonkb/moses/moses/moses-cmd/src/moses -config
filtered/moses.ini -inputtype 0 -w 0.000000 -lm 0.111111 -d 0.111111
0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 -tm 0.033333
0.022222 0.033333 0.022222 0.000000 -n-best-list run1.best100.out 100
-i
/afs/csail.mit.edu/u/j/jasonkb/moses/working-dirs/reordering-factored-pos/tuning/dev.input
>run1.out
Should the input be expected to have only one factor, if the
reordering model uses the second factor? Or have I set up my training
data or command-line flags wrongly?
By the way, I hope to make my system setup freely available when I
have finished a few more experiments, so I hope we will have a free
Japanese-English system that can rival the current commercial
offerings.
Thank you for any help!
--Jason
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
