I included the moses.ini file in the original post. I checked in VIM using :set list. Everything looks good. I repost it here:
# MERT optimized configuration # decoder /home/steve/mosesdecoder/bin/moses # BLEU 0.18611 on dev /home/steve/corpus/news-test2008.true.fr # We were before running iteration 5 # finished Thu Sep 15 01:53:22 PDT 2016 ### MOSES CONFIG FILE ### ######################### # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 [distortion-limit] 6 # feature functions [feature] UnknownWordPenalty WordPenalty PhrasePenalty PhraseDictionaryCompact name=TranslationModel0 num-features=4 path=/home/steve/working/binarised-model/phrase-table.minphr input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0 num-features=6 type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 path=/home/steve/working/binarised-model/reordering-table Distortion KENLM name=LM0 factor=0 path=/home/steve/lm/news-commentary-v8.fr-en.blm.en order=3 # dense weights for feature functions [threads] 4 [weight] LexicalReordering0= 0.0760973 0.0426988 0.0872591 0.0872621 0.0141898 0.0348721 Distortion0= 0.0721205 LM0= 0.0908638 WordPenalty0= -0.153776 PhrasePenalty0= 0.0874059 TranslationModel0= 0.0162038 0.0871216 0.0687163 0.0814132 UnknownWordPenalty0= 1 On Fri, Sep 16, 2016 at 12:10 AM, Hieu Hoang <[email protected]> wrote: > look inside the file > mert-work/moses.ini > is there anything strange going on? > > Hieu Hoang > http://www.hoang.co.uk/hieu > > On 16 September 2016 at 09:02, Steve Braich <[email protected]> wrote: > >> Hi Hieu, >> I changed from relative path to absolute. Still same error. >> >> Thanks, >> Steve >> >> On Thu, Sep 15, 2016 at 8:54 PM, Hieu Hoang <[email protected]> wrote: >> >>> don't use relative paths, ie. change >>> filtered-newstest2011 -> /whatever/filtered-newstest2011 >>> mert-work/moses.ini -> /whatever/mert-work/moses.ini >>> >>> >>> Hieu Hoang >>> http://www.hoang.co.uk/hieu >>> >>> On 16 September 2016 at 05:42, Steve Braich <[email protected]> wrote: >>> >>>> Hello, >>>> >>>> I am going thru the steps described in the baseline system >>>> instructions. I have no problems until the very end when I try to filter >>>> out the test data from the binarised model. >>>> >>>> >>>> >>>> Here is the step in the instructions where the error appears: >>>> >>>> *The model that we've trained can then be filtered for this test set, >>>> meaning that we only retain the entries needed translate the test set. This >>>> will make the translation a lot faster.* >>>> >>>> >>>> >>>> *cd ~/working* >>>> >>>> *~/mosesdecoder/scripts/training/filter-model-given-input.pl >>>> <http://filter-model-given-input.pl> filtered-newstest2011 >>>> mert-work/moses.ini ~/corpus/newstest2011.true.fr >>>> <http://newstest2011.true.fr> -Binarizer >>>> ~/mosesdecoder/bin/processPhraseTableMin* >>>> >>>> >>>> >>>> Here is the error message: >>>> >>>> *Can't open '< /home/steve/working/binarised-model/reordering-table' at >>>> /home/steve/mosesdecoder/scripts/training/filter-model-given-input.pl >>>> <http://filter-model-given-input.pl> line 439.* >>>> >>>> >>>> >>>> Keep in mind the following: >>>> >>>> - I compiled moses with the following parameters. It is the same >>>> compilation. I did not recompile moses at any point during the process. >>>> ./bjam -a --with-boost=/home/steve/mosesdecoder/boost_1_55_0 >>>> --with-cmph=$HOME/mosesdecoder/cmph/cmph-2.0 >>>> NOTE: I did try recompiling with the absolute path for cmph. Same >>>> error. >>>> - I checked all paths first to make sure they exist. >>>> - I am able to translate segments using the binarised model with no >>>> problem >>>> - I provided my moses.ini file below. >>>> >>>> Questions: >>>> >>>> - Can I recompile moses after training and/or tuning, and expect >>>> that it should work for testing? I ask because I want to know if the >>>> solution is to modify the compile options, do I have to start from >>>> scratch >>>> again? (Can you filter a binarised model using moses with different >>>> compile options that were used to train the model?) >>>> >>>> Thanks, >>>> Steve >>>> >>>> >>>> Moses.ini file >>>> >>>> # MERT optimized configuration >>>> >>>> # decoder /home/steve/mosesdecoder/bin/moses >>>> >>>> # BLEU 0.18611 on dev /home/steve/corpus/news-test2008.true.fr >>>> >>>> # We were before running iteration 5 >>>> >>>> # finished Thu Sep 15 01:53:22 PDT 2016 >>>> >>>> ### MOSES CONFIG FILE ### >>>> >>>> ######################### >>>> >>>> >>>> # input factors >>>> >>>> [input-factors] >>>> >>>> 0 >>>> >>>> >>>> # mapping steps >>>> >>>> [mapping] >>>> >>>> 0 T 0 >>>> >>>> >>>> [distortion-limit] >>>> >>>> 6 >>>> >>>> >>>> # feature functions >>>> >>>> [feature] >>>> >>>> UnknownWordPenalty >>>> >>>> WordPenalty >>>> >>>> PhrasePenalty >>>> >>>> PhraseDictionaryCompact name=TranslationModel0 num-features=4 >>>> path=/home/steve/working/binarised-model/phrase-table.minphr >>>> input-factor=0 output-factor=0 >>>> >>>> LexicalReordering name=LexicalReordering0 num-features=6 >>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0 >>>> path=/home/steve/working/binarised-model/reordering-table >>>> >>>> Distortion >>>> >>>> KENLM name=LM0 factor=0 path=/home/steve/lm/news-commentary-v8.fr-en.blm.en >>>> order=3 >>>> >>>> >>>> # dense weights for feature functions >>>> >>>> >>>> [threads] >>>> >>>> 4 >>>> >>>> [weight] >>>> >>>> >>>> LexicalReordering0= 0.0760973 0.0426988 0.0872591 0.0872621 0.0141898 >>>> 0.0348721 >>>> >>>> Distortion0= 0.0721205 >>>> >>>> LM0= 0.0908638 >>>> >>>> WordPenalty0= -0.153776 >>>> >>>> PhrasePenalty0= 0.0874059 >>>> >>>> TranslationModel0= 0.0162038 0.0871216 0.0687163 0.0814132 >>>> >>>> UnknownWordPenalty0= 1 >>>> >>>> >>>> _______________________________________________ >>>> Moses-support mailing list >>>> [email protected] >>>> http://mailman.mit.edu/mailman/listinfo/moses-support >>>> >>>> >>> >> >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
