Hello,
I am trying to configure Moses to use a language model over word classes.
To do this, I've added a generation model, which is basically just a
mapping from a word to a word class and a generation step after the
translation step.
The issue is that while loading the generation model I get a segmentation
fault. I am using a custom word mapping file, so there might be issues with
the structure of the file. I have tried to follow the layout as shown in
your factored training tutorial. A line in the mapping file is split into 4
fields: <target_word> <word_class> 1 0
The output of the command "./mosesdecoder/bin/moses -i
/work/smt2/graca/iwslt/2013/de-en/translation/pbt.all.wclm.hrm.moses.0/f-demo
-f moses.ini -v 3 > out" is given below:
I have also tried not including the generation model and the language model
over word classes and in that case I do not obtain a segmentation fault.
Regards,
Miguel
--------------------------------------
Defined parameters (per moses.ini or switch):
config: moses.ini
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
path=/work/smt2/graca/iwslt/2013/de-en/training/pbt.moses/model/filtered/phrase-table.bin
input-factor=0 output-factor=0 LexicalReordering name=LexicalReordering0
num-features=6 type=hier-msd-bidirectional-fe-allff input-factor=0
output-factor=0
path=/work/smt2/graca/iwslt/2013/de-en/training/pbt.moses/model/filtered/reordering-table.hier-msd-bidirectional-fe
Distortion KENLM name=LM0 factor=0 path=lm order=4 KENLM name=LM1 factor=1
path=wclm order=7 Generation name=GEN path=e.classes.ext input-factor=0
output-factor=1
input-factors: 0
input-file:
/work/smt2/graca/iwslt/2013/de-en/translation/pbt.all.wclm.hrm.moses.0/f-demo
mapping: 0 T 0 0 G 1
verbose: 3
weight: UnknownWordPenalty0= 1 WordPenalty0= -1 PhrasePenalty0= 0.2
TranslationModel0= 0.2 0.2 0.2 0.2 LexicalReordering0= 0.3 0.3 0.3 0.3 0.3
0.3 Distortion0= 0.3 LM0= 0.5 LM1= 0.5 GEN= 1
input type is: text input
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryOnDisk name=TranslationModel0 num-features=4
path=/work/smt2/graca/iwslt/2013/de-en/training/pbt.moses/model/filtered/phrase-table.bin
input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
DecodeFeature:
line=LexicalReordering name=LexicalReordering0 num-features=6
type=hier-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/work/smt2/graca/iwslt/2013/de-en/training/pbt.moses/model/filtered/reordering-table.hier-msd-bidirectional-fe
FeatureFunction: LexicalReordering0 start: 7 end: 12
Initializing Lexical Reordering Feature..
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM name=LM0 factor=0 path=lm order=4
FeatureFunction: LM0 start: 14 end: 14
line=KENLM name=LM1 factor=1 path=wclm order=7
FeatureFunction: LM1 start: 15 end: 15
line=Generation name=GEN path=e.classes.ext input-factor=0 output-factor=1
FeatureFunction: GEN start: 16 end: 16
DecodeFeature:
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Loading table into memory...done.
Loading Distortion0
Loading LM0
Loading LM1
Loading GEN
./queueTrans.sh: line 1: 13276 Segmentation fault (core dumped)
./mosesdecoder/bin/moses -i
/work/smt2/graca/iwslt/2013/de-en/translation/pbt.all.wclm.hrm.moses.0/f-demo
-f moses.ini -v 3 > out
------------------------------------------
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
