Hi Jason, sounds very interesting.
There seems to be some bug in the creation of the moses.ini file. For now, the easiest fix should be to edit moses.ini so it reflects the input you expect: [input-factors] 0 1 Just curious: What kind and amount of training data do you have for Japanese-English. Is any of it publicly available? -phi On Jan 21, 2008 2:53 AM, Jason Katz-Brown <[EMAIL PROTECTED]> wrote: > Hi all, > > I am working on improving a baseline Japanese->English moses system I > developed. It achieves okay translation results already. I am > experimenting with adding part-of-speech factors, so I prepared > training data that looks like: > > word0|pos0 word1|pos1 word2|pos2 ... > > I tried two moses training runs on this factored data. In the first, I > wanted to use the POS factor for word alignment, translation, and > reordering, so I used these flags to the > train-factored-phrase-model.perl script: > > --alignment-factors=0,1-0 --translation-factors=0,1-0 > --reordering-factors=0,1-0 --decoding-steps=t0 > > The training finished successfully, MERT training also finished > successfully, and I could also translate and calculate BLEU score > successfully. (although it didn't give an improvement in BLEU score. > I'll look into why :-) > > In the second run, I wanted to use the POS factor only in the > reordering model. I tried these flags: > > --alignment-factors=0-0 --translation-factors=0-0 > --reordering-factors=0,1-0 --decoding-steps=t0 > > The training finished successfully, but MERT training gave this error > (with some things snipped out): > > Defined parameters (per moses.ini or switch): > config: filtered/moses.ini > distortion-file: 0,1-0 msd-bidirectional-fe 6 > /afs/csail.mit.edu/u/j/jasonkb/moses/working-dirs/reordering-factored-pos/tuning/filtered/reordering-table.msd-bidirectional-fe.0.5.0,1-0 > <snip> > ttable-file: 0 0 5 > /afs/csail.mit.edu/u/j/jasonkb/moses/working-dirs/reordering-factored-pos/tuning/filtered/phrase-table.0-0 > <snip> > [ERROR] Malformed input at > Expected input to have words composed of 1 factor(s) (form FAC1|FAC2|...) > but instead received input with 2 factor(s). > sh: line 1: 18883 Aborted > /afs/csail.mit.edu/u/j/jasonkb/moses/moses/moses-cmd/src/moses -config > filtered/moses.ini -inputtype 0 -w 0.000000 -lm 0.111111 -d 0.111111 > 0.111111 0.111111 0.111111 0.111111 0.111111 0.111111 -tm 0.033333 > 0.022222 0.033333 0.022222 0.000000 -n-best-list run1.best100.out 100 > -i > /afs/csail.mit.edu/u/j/jasonkb/moses/working-dirs/reordering-factored-pos/tuning/dev.input > >run1.out > > Should the input be expected to have only one factor, if the > reordering model uses the second factor? Or have I set up my training > data or command-line flags wrongly? > > By the way, I hope to make my system setup freely available when I > have finished a few more experiments, so I hope we will have a free > Japanese-English system that can rival the current commercial > offerings. > > Thank you for any help! > --Jason > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
