Hi
I'm having the same problem, but it don't work with:
$SCRIPTS_ROOTDIR/training/mert-moses.pl <http://mert-moses.pl>
data/low/tune.$f data/low/tune.$e $MOSESCHART $f-$e/model/moses.ini
--working-dir $f-$e/tuning --rootdir $SCRIPTS_ROOTDIR
--mertdir=/Data/moses/mert --filtercmd
'/Data/moses/tools/bin/moses-scripts/training/filter-model-given-input.pl
<http://filter-model-given-input.pl>
-Binarizer "CreateOnDiskPt 1 1 5 100 2" --Hierarchical'
...
loadbin_level (level 3)
mapping 3265681 3-grams
tableOffs 14460016 tableGaps1136-grams
done (level3)
loadbin_level (level 4)
mapping 4729686 4-grams
tableOffs 43853193 tableGaps1417-grams
done (level4)
loadbin_level (level 5)
mapping 5476137 5-grams
tableOffs 86421391 tableGaps3983-grams
done (level5)
done
OOV code is 141515
IRST: m_unknownId=141515
Finished loading LanguageModels : [0.000] seconds
Using uniform ttable-limit of 20 for all translation tables.
Start loading PhraseTable
/Data/experiment/kftt-moses-1.1_experimento_9/ja-en/tuning/filtered/phrase-table.0-0.1.1.bin
: [0.000] seconds
filePath:
/Data/experiment/kftt-moses-1.1_experimento_9/ja-en/tuning/filtered/phrase-table.0-0.1.1.bin
Start loading PhraseTable
/Data/experiment/kftt-moses-1.1_experimento_9/ja-en/model/glue-grammar
: [0.000] seconds
filePath:
/Data/experiment/kftt-moses-1.1_experimento_9/ja-en/model/glue-grammar
Finished loading phrase tables : [0.000] seconds
Start loading phrase table from
/Data/experiment/kftt-moses-1.1_experimento_9/ja-en/model/glue-grammar
: [0.000] seconds
Start loading new format pt model : [0.000] seconds
Finished loading phrase tables : [0.000] seconds
Created input-output object : [0.000] seconds
moses_chart: OnDiskWrapper.cpp:59: bool
OnDiskPt::OnDiskWrapper::OpenForLoad(const std::string&): Assertion
`m_fileSource.is_open()' failed.
Aborted
Exit code: 134
The decoder died. CONFIG WAS -w -0.285714 -lm 0.142857 -tm 0.057143
0.057143 0.057143 0.057143 0.057143 0.285714
My moses.ini is like that:
#########################
### MOSES CONFIG FILE ###
#########################
# input factors
[input-factors]
0
# mapping steps
[mapping]
0 T 0
1 T 1
# translation tables: table type (hierarchical(0), textual (0), binary
(1)),
source-factors, target-factors, number of scores, file
# OLD FORMAT is still handled for back-compatibility
# OLD FORMAT translation tables: source-factors, target-factors, number of
scores, file
# OLD FORMAT a binary table type (1) is assumed
[ttable-file]
6 0 0 5 /Data/experiment/moses-1.1_experimento_9/it-en/model/rule-table.gz
6 0 0 1 /Data/experiment/moses-1.1_experimento_9/it-en/model/glue-grammar
# no generation models, no generation-file section
# language models: type(srilm/irstlm), factors, order, file
[lmodel-file]
1 0 5 /Data/experiment/moses-1.1_experimento_9/lm/train.en.qblm.mm
<http://train.en.qblm.mm>
# limit on how many phrase translations e for each phrase f are loaded
# 0 = all elements loaded
[ttable-limit]
20
# language model weights
[weight-l]
0.5000
# translation model weights
[weight-t]
0.20
0.20
0.20
0.20
0.20
1.0
# no generation models, no weight-generation section
# word penalty
[weight-w]
-1
[cube-pruning-pop-limit]
1000
[non-terminals]
X
[search-algorithm]
3
[inputtype]
3
[max-chart-span]
20
1000
Barry Haddow <bhaddow@...> writes:
>
> Hi
>
> You don't need to change the mert-moses.perl script. Just pass the
filter
> command with the hierarchical argument to the script, like this
>
> mert-moses.perl ... --filtercmd 'filter-model-given-input.pl
<http://filter-model-given-input.pl> -Binarizer
"CreateOnDiskPt
> 1 1 5 100 2" --Hierarchical'
>
> best regards - Barry
>
> On Thursday 31 March 2011 10:21, Pratyush Banerjee wrote:
> > Hi All,
> >
> > I faced the same issue when trying to tune a tree-based model
using the
> > mert-moses.pl <http://mert-moses.pl> script. (I am running Moses
R3739)
> > Not sure if leaving out the Hierarchical switch in the mert script was
> > intentional or had any particular reason.
> >
> > I have already made the necessary changes in the mert-moses.pl
<http://mert-moses.pl> which i
> > have attached herewith. It requires a --herarchical switch to run the
> > filterring in a proper way.
> >
> > Thanks and regards,
> >
> > Pratyush
> >
> > On 28/03/11 14:18, Kaveh Taghipour wrote:
> > > The script "mert-moses-new.pl <http://mert-moses-new.pl>"
filters the model and then starts
> > > tuning. To filter the rule-table in tree-based models, the script
> > > "filter-model-given-input.pl
<http://filter-model-given-input.pl>" should be called with the parameter
> > > "-hierarchical", but "mert-moses-new.pl
<http://mert-moses-new.pl>" calls the script without it.
> > > So, using "mert-moses-new.pl <http://mert-moses-new.pl>" for
tuning a tree-based model would
> > > result in an emtpy filtered rule-table and this causes the
decoder to
> > > copy the input tuning file to the output and the tuning phase ends
> > > with some meaningless weights without any errors.
> > >
> > > Using "mert-moses-new.pl <http://mert-moses-new.pl>" with the
parameter
> > > "--no-filter-phrase-table" fixes this problem, but slows the tuning
> > > process. It's better to add "-hierarchical" to the script
> > > "mert-moses-new.pl <http://mert-moses-new.pl>" (line 575).
> > > _______________________________________________
> > > Moses-support mailing list
> > > Moses-support@...
> > > http://mailman.mit.edu/mailman/listinfo/moses-support
>
--
Best regards and thanks
Alex Helle
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
