Hi Yulia
On 1 September 2013 22:46, Yulia Tsvetkov <[email protected]> wrote: > Dear Moses developers, > > I am trying to use the a new version of Moses, seems like things have > changed quite a bit and I have hard time finding an up-to-date > documentation. For debugging I used very small train/tune/test corpora (10 > lines each). > > First thing is running the following command produces a phrase table with > only 4 features: > train-model.perl --root-dir $root_dir --corpus $root_dir/$corpus_name --f > $src_lng --e $trg_lng --alignment grow-diag-final --lm 0:3:$LM > -external-bin-dir $external_bin_dir`; > > Here is a snippet from a produced moses.iniPhraseDictionaryMemory > name=TranslationModel0 table-limit=20 *num-features=4 > *path=/usr1/projects/mt_proj/mt_eval/baselines/fr-base-1-lats/model/phrase-table.gz > input-factor=0 output-factor=0 > Yes, the phrase-table now has 4 scores, instead of 5. The 5th score was a constant 2.718. This has now moved into it's own feature function, PhrasePenalty. it save 3% of disk space, and i think is better for research. eg. create better, non-constant phrase penalty feature functions, if we have 2 phrase tables do we need just 1 phrase penalty? etc. > Second, I am trying to run tuning and decoding of lattices in plf format. > Can you point me to example commands and moses.ini for running mert and > decoding lattices with the new Moses? > an example ini file for lattices can be seen here https://github.com/moses-smt/moses-regression-tests/blob/master/tests/phrase.lattice-surface/moses.ini Mert should run like it has always did. However, if you upgrade the decoder, you should use the upgraded mert script too. Decoding with lattice is exactly the same as for a sentence, except 2 things 1. inputtype=2. This can be on the command line of in the ini file, eg. ./moses -inputtype 2 or [inputtype] 2 2. You should use the InputFeature feature function. This is the score of the path through the lattice. You can see the InputFeature in the ini file: [feature] .... InputFeature num-features=1 num-input-features=1 real-word-count=0 [weight] ... InputFeature0 = 1 Before the refactoring, this was hacked into as an extra feature in the phrase-table > > So far I tried training and tuning on text files and decoding on lattices > because I could not figure out the right settings for tuning. > According to some old documentation I am supposed to convert the phrase > table to a binary format. Is it still needed? > You no longer need to convert it to binary format. It's good to convert to binary format to save memory, but it is not required. Lattice decoding works with all phrase-table implmentations now > > When I ran it with the following command: > moses *-inputtype 2 -weight-i 0.62 -weight-l 12.5* -f $tune_dir/moses.ini > < $eval_dir/69.plf > $eval_dir/69.plf.out > I got an error: > *Don't mix old and new ini file format* > What is the new equivalent of weight-i and weight-l? > -weight-i 0.62 now becomes -weight-overwrite 'InputFeature0= 0.62' -weight-l 12.5 now becomes -weight-overwrite 'LM0= 12.5' The updated mert script should be doing this anyway. > > Without those parameters I get a Segmentation Fault with both a .gz and a > binary phrase table. > if you're still having problems, give me your ini file and exact command you're executing and i'll try and debug it > > Could you help me figuring out the right settings? > > Thanks in advance. > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > -- Hieu Hoang Research Associate University of Edinburgh http://www.hoang.co.uk/hieu
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
