ah. from the moses.ini file, it looks like you've binarized the
phrase-table (and the re-ordering model?) already.

The filtering script only works with moses.ini which has the original text
pt and reordering files

Hieu Hoang
http://www.hoang.co.uk/hieu

On 16 September 2016 at 09:13, Steve Braich <steveb...@gmail.com> wrote:

> I included the moses.ini file in the original post.
>
> I checked in VIM using :set list.  Everything looks good.  I repost it
> here:
>
> # MERT optimized configuration
> # decoder /home/steve/mosesdecoder/bin/moses
> # BLEU 0.18611 on dev /home/steve/corpus/news-test2008.true.fr
> # We were before running iteration 5
> # finished Thu Sep 15 01:53:22 PDT 2016
> ### MOSES CONFIG FILE ###
> #########################
>
> # input factors
> [input-factors]
> 0
>
> # mapping steps
> [mapping]
> 0 T 0
>
> [distortion-limit]
> 6
>
> # feature functions
> [feature]
> UnknownWordPenalty
> WordPenalty
> PhrasePenalty
> PhraseDictionaryCompact name=TranslationModel0 num-features=4
> path=/home/steve/working/binarised-model/phrase-table.minphr
> input-factor=0 output-factor=0
> LexicalReordering name=LexicalReordering0 num-features=6
> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
> path=/home/steve/working/binarised-model/reordering-table
> Distortion
> KENLM name=LM0 factor=0 path=/home/steve/lm/news-commentary-v8.fr-en.blm.en
> order=3
>
> # dense weights for feature functions
>
> [threads]
> 4
> [weight]
>
> LexicalReordering0= 0.0760973 0.0426988 0.0872591 0.0872621 0.0141898
> 0.0348721
> Distortion0= 0.0721205
> LM0= 0.0908638
> WordPenalty0= -0.153776
> PhrasePenalty0= 0.0874059
> TranslationModel0= 0.0162038 0.0871216 0.0687163 0.0814132
> UnknownWordPenalty0= 1
>
> On Fri, Sep 16, 2016 at 12:10 AM, Hieu Hoang <hieuho...@gmail.com> wrote:
>
>> look inside the file
>>    mert-work/moses.ini
>> is there anything strange going on?
>>
>> Hieu Hoang
>> http://www.hoang.co.uk/hieu
>>
>> On 16 September 2016 at 09:02, Steve Braich <steveb...@gmail.com> wrote:
>>
>>> Hi Hieu,
>>> I changed from relative path to absolute.  Still same error.
>>>
>>> Thanks,
>>> Steve
>>>
>>> On Thu, Sep 15, 2016 at 8:54 PM, Hieu Hoang <hieuho...@gmail.com> wrote:
>>>
>>>> don't use relative paths, ie. change
>>>>    filtered-newstest2011 -> /whatever/filtered-newstest2011
>>>>   mert-work/moses.ini -> /whatever/mert-work/moses.ini
>>>>
>>>>
>>>> Hieu Hoang
>>>> http://www.hoang.co.uk/hieu
>>>>
>>>> On 16 September 2016 at 05:42, Steve Braich <steveb...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I am going thru the steps described in the baseline system
>>>>> instructions.  I have no problems until the very end when I try to filter
>>>>> out the test data from the binarised model.
>>>>>
>>>>>
>>>>>
>>>>> Here is the step in the instructions where the error appears:
>>>>>
>>>>> *The model that we've trained can then be filtered for this test set,
>>>>> meaning that we only retain the entries needed translate the test set. 
>>>>> This
>>>>> will make the translation a lot faster.*
>>>>>
>>>>>
>>>>>
>>>>> *cd ~/working*
>>>>>
>>>>> *~/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>> <http://filter-model-given-input.pl> filtered-newstest2011
>>>>> mert-work/moses.ini ~/corpus/newstest2011.true.fr
>>>>> <http://newstest2011.true.fr> -Binarizer
>>>>> ~/mosesdecoder/bin/processPhraseTableMin*
>>>>>
>>>>>
>>>>>
>>>>> Here is the error message:
>>>>>
>>>>> *Can't open '< /home/steve/working/binarised-model/reordering-table'
>>>>> at /home/steve/mosesdecoder/scripts/training/filter-model-given-input.pl
>>>>> <http://filter-model-given-input.pl> line 439.*
>>>>>
>>>>>
>>>>>
>>>>> Keep in mind the following:
>>>>>
>>>>>    - I compiled moses with the following parameters.  It is the same
>>>>>    compilation.  I did not recompile moses at any point during the 
>>>>> process.
>>>>>    ./bjam -a --with-boost=/home/steve/mosesdecoder/boost_1_55_0
>>>>>    --with-cmph=$HOME/mosesdecoder/cmph/cmph-2.0
>>>>>    NOTE: I did try recompiling with the absolute path for cmph.  Same
>>>>>    error.
>>>>>    - I checked all paths first to make sure they exist.
>>>>>    - I am able to translate segments using the binarised model with
>>>>>    no problem
>>>>>    - I provided my moses.ini file below.
>>>>>
>>>>> Questions:
>>>>>
>>>>>    - Can I recompile moses after training and/or tuning, and expect
>>>>>    that it should work for testing?  I ask because I want to know if the
>>>>>    solution is to modify the compile options, do I have to start from 
>>>>> scratch
>>>>>    again?  (Can you filter a binarised model using moses with different
>>>>>    compile options that were used to train the model?)
>>>>>
>>>>> Thanks,
>>>>> Steve
>>>>>
>>>>>
>>>>> Moses.ini file
>>>>>
>>>>> # MERT optimized configuration
>>>>>
>>>>> # decoder /home/steve/mosesdecoder/bin/moses
>>>>>
>>>>> # BLEU 0.18611 on dev /home/steve/corpus/news-test2008.true.fr
>>>>>
>>>>> # We were before running iteration 5
>>>>>
>>>>> # finished Thu Sep 15 01:53:22 PDT 2016
>>>>>
>>>>> ### MOSES CONFIG FILE ###
>>>>>
>>>>> #########################
>>>>>
>>>>>
>>>>> # input factors
>>>>>
>>>>> [input-factors]
>>>>>
>>>>> 0
>>>>>
>>>>>
>>>>> # mapping steps
>>>>>
>>>>> [mapping]
>>>>>
>>>>> 0 T 0
>>>>>
>>>>>
>>>>> [distortion-limit]
>>>>>
>>>>> 6
>>>>>
>>>>>
>>>>> # feature functions
>>>>>
>>>>> [feature]
>>>>>
>>>>> UnknownWordPenalty
>>>>>
>>>>> WordPenalty
>>>>>
>>>>> PhrasePenalty
>>>>>
>>>>> PhraseDictionaryCompact name=TranslationModel0 num-features=4
>>>>> path=/home/steve/working/binarised-model/phrase-table.minphr
>>>>> input-factor=0 output-factor=0
>>>>>
>>>>> LexicalReordering name=LexicalReordering0 num-features=6
>>>>> type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
>>>>> path=/home/steve/working/binarised-model/reordering-table
>>>>>
>>>>> Distortion
>>>>>
>>>>> KENLM name=LM0 factor=0 
>>>>> path=/home/steve/lm/news-commentary-v8.fr-en.blm.en
>>>>> order=3
>>>>>
>>>>>
>>>>> # dense weights for feature functions
>>>>>
>>>>>
>>>>> [threads]
>>>>>
>>>>> 4
>>>>>
>>>>> [weight]
>>>>>
>>>>>
>>>>> LexicalReordering0= 0.0760973 0.0426988 0.0872591 0.0872621 0.0141898
>>>>> 0.0348721
>>>>>
>>>>> Distortion0= 0.0721205
>>>>>
>>>>> LM0= 0.0908638
>>>>>
>>>>> WordPenalty0= -0.153776
>>>>>
>>>>> PhrasePenalty0= 0.0874059
>>>>>
>>>>> TranslationModel0= 0.0162038 0.0871216 0.0687163 0.0814132
>>>>>
>>>>> UnknownWordPenalty0= 1
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Moses-support mailing list
>>>>> Moses-support@mit.edu
>>>>> http://mailman.mit.edu/mailman/listinfo/moses-support
>>>>>
>>>>>
>>>>
>>>
>>
>
_______________________________________________
Moses-support mailing list
Moses-support@mit.edu
http://mailman.mit.edu/mailman/listinfo/moses-support

Reply via email to