Hi I'm having the same problem, but it don't work with:
$SCRIPTS_ROOTDIR/training/mert-moses.pl data/low/tune.$f data/low/tune.$e $MOSESCHART $f-$e/model/moses.ini --working-dir $f-$e/tuning --rootdir $SCRIPTS_ROOTDIR --mertdir=/Data/moses/mert --filtercmd '/Data/moses/tools/bin/moses-scripts/training/filter-model-given-input.pl -Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical' ... loadbin_level (level 3) mapping 3265681 3-grams tableOffs 14460016 tableGaps1136-grams done (level3) loadbin_level (level 4) mapping 4729686 4-grams tableOffs 43853193 tableGaps1417-grams done (level4) loadbin_level (level 5) mapping 5476137 5-grams tableOffs 86421391 tableGaps3983-grams done (level5) done OOV code is 141515 IRST: m_unknownId=141515 Finished loading LanguageModels : [0.000] seconds Using uniform ttable-limit of 20 for all translation tables. Start loading PhraseTable /Data/experiment/kftt-moses-1.1_experimento_9/ja-en/tuning/filtered/phrase-table.0-0.1.1.bin : [0.000] seconds filePath: /Data/experiment/kftt-moses-1.1_experimento_9/ja-en/tuning/filtered/phrase-table.0-0.1.1.bin Start loading PhraseTable /Data/experiment/kftt-moses-1.1_experimento_9/ja-en/model/glue-grammar : [0.000] seconds filePath: /Data/experiment/kftt-moses-1.1_experimento_9/ja-en/model/glue-grammar Finished loading phrase tables : [0.000] seconds Start loading phrase table from /Data/experiment/kftt-moses-1.1_experimento_9/ja-en/model/glue-grammar : [0.000] seconds Start loading new format pt model : [0.000] seconds Finished loading phrase tables : [0.000] seconds Created input-output object : [0.000] seconds moses_chart: OnDiskWrapper.cpp:59: bool OnDiskPt::OnDiskWrapper::OpenForLoad(const std::string&): Assertion `m_fileSource.is_open()' failed. Aborted Exit code: 134 The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143 0.057143 0.057143 0.057143 0.057143 0.285714 My moses.ini is like that: ######################### ### MOSES CONFIG FILE ### ######################### # input factors [input-factors] 0 # mapping steps [mapping] 0 T 0 1 T 1 # translation tables: table type (hierarchical(0), textual (0), binary (1)), source-factors, target-factors, number of scores, file # OLD FORMAT is still handled for back-compatibility # OLD FORMAT translation tables: source-factors, target-factors, number of scores, file # OLD FORMAT a binary table type (1) is assumed [ttable-file] 6 0 0 5 /Data/experiment/moses-1.1_experimento_9/it-en/model/rule-table.gz 6 0 0 1 /Data/experiment/moses-1.1_experimento_9/it-en/model/glue-grammar # no generation models, no generation-file section # language models: type(srilm/irstlm), factors, order, file [lmodel-file] 1 0 5 /Data/experiment/moses-1.1_experimento_9/lm/train.en.qblm.mm # limit on how many phrase translations e for each phrase f are loaded # 0 = all elements loaded [ttable-limit] 20 # language model weights [weight-l] 0.5000 # translation model weights [weight-t] 0.20 0.20 0.20 0.20 0.20 1.0 # no generation models, no weight-generation section # word penalty [weight-w] -1 [cube-pruning-pop-limit] 1000 [non-terminals] X [search-algorithm] 3 [inputtype] 3 [max-chart-span] 20 1000 Barry Haddow <bhaddow@...> writes: > > Hi > > You don't need to change the mert-moses.perl script. Just pass the filter > command with the hierarchical argument to the script, like this > > mert-moses.perl ... --filtercmd 'filter-model-given-input.pl -Binarizer "CreateOnDiskPt > 1 1 5 100 2" --Hierarchical' > > best regards - Barry > > On Thursday 31 March 2011 10:21, Pratyush Banerjee wrote: > > Hi All, > > > > I faced the same issue when trying to tune a tree-based model using the > > mert-moses.pl script. (I am running Moses R3739) > > Not sure if leaving out the Hierarchical switch in the mert script was > > intentional or had any particular reason. > > > > I have already made the necessary changes in the mert-moses.pl which i > > have attached herewith. It requires a --herarchical switch to run the > > filterring in a proper way. > > > > Thanks and regards, > > > > Pratyush > > > > On 28/03/11 14:18, Kaveh Taghipour wrote: > > > The script "mert-moses-new.pl" filters the model and then starts > > > tuning. To filter the rule-table in tree-based models, the script > > > "filter-model-given-input.pl" should be called with the parameter > > > "-hierarchical", but "mert-moses-new.pl" calls the script without it. > > > So, using "mert-moses-new.pl" for tuning a tree-based model would > > > result in an emtpy filtered rule-table and this causes the decoder to > > > copy the input tuning file to the output and the tuning phase ends > > > with some meaningless weights without any errors. > > > > > > Using "mert-moses-new.pl" with the parameter > > > "--no-filter-phrase-table" fixes this problem, but slows the tuning > > > process. It's better to add "-hierarchical" to the script > > > "mert-moses-new.pl" (line 575). > > > _______________________________________________ > > > Moses-support mailing list > > > Moses-support@... > > > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Best regards and thanks Alex Helle**
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
