Hello everybody, I am currently trying to add an externally computed score to the phrase table. The format looks like this:
in europa ||| in europe ||| 0.5 0.03125 1 0.340909 ||| 0-0 1-1 ||| 1 1 1 ||| ||| As far as I understand the first and the second column contain the phrase in both languages and the third column contains a list of scores, separated by white spaces. I would now use an external tool to add another score to the third column. My first attempt was to simply compute the score and add it - separated by another white space - to the third column. This is what it could look like: in europa ||| in europe ||| 0.5 0.03125 1 0.340909 0.97 ||| 0-0 1-1 ||| 1 1 1 ||| ||| Since now I have 5 instead of 4 features I changed the num-features option in my moses.ini: PhraseDictionaryMemory name=TranslationModel0 table-limit=20 num-features=5 path=... input-factor=0 output-factor=0 Also, I added one more inital weight to the TranslationModel0 line: [weight] TranslationModel0= 0.2 0.2 0.2 0.2 0.2 The moses.ini that I changed is located in my working directory under "train/model/". The phrase table is saved under "/train/model/phrase-table.gz". When I run the tuning as $mosesbase/mosesdecoder/scripts/training/mert-moses.pl $mosesbase/corpus/train.100.ja $mosesbase/corpus/train.100.en $mosesbase/mosesdecoder/bin/moses train/model/moses.ini --mertdir $mosesbase/mosesdecoder/bin/ --rootdir $mosesbase/mosesdecoder/scripts/ &> translation.log the decoder exits with Size of scoreVector != number (4!=5) of score components on line 0 After looking around I found out that the phrase table filtering somehow deletes my newly added score. In my initial phrase table located in the "train/model" directory my score is present but in the phrase table in the mert-work directory "mert-work/filtered/phrase-table.0-0.1.1.gz" for some reason the score is gone. Is there anything I need to specify to keep my score? Thank you in advance! Best, Hubert _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
