Hi,
that's weird. The numbers in the 3rd pass should be the same as for the
other passes. It does indeed look truncated. Could it be that you are
running out of temporary space? (Although there should be some kind of
error in that case.)
You can set the temporary folder to with -T /path/to/temp to use a
custom folder other than /tmp which is used by default. Let's see if
that helps. Since the passes are uncompressed it needs more space than
the final compressed output. Give it at least as much space as the
uncompressed text phrase table. That should be more than enough.
Best,
Marcin
W dniu 29.09.2014 21:35, Eleftherios Avramidis pisze:
Hi,
our Moses server has issues loading the compact phrase tables. We are
running Moses 2.1.1.
Attached you can find the log file from the binarization done by
processPhraseTableMin. We run two executions of the binarization (both
included in the log). I don't know if the output of the log file was
truncated, but the executions do not seem the same.
This is the error we get during loading of phrase-table.minphr by
mosesserver:
Loading table into memory...Defined parameters (per moses.ini or switch):
config:
/project/qtleap/software/mtmonkey/user/en-de/mt-2.1/models/deen/moses.ini.4.en-de
distortion-limit: 6
feature: UnknownWordPenalty WordPenalty PhrasePenalty
PhraseDictionaryCompact name=TranslationModel0 table-limit=20
num-features=4
path=/project/qtleap/pilot0/systems/en-de/model/phrase-table.2.minphr
input-factor=0 output-factor=0 LexicalReordering
name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/project/qtleap/pilot0/systems/en-de/model/reordering-table.2.wbe-msd-bidirectional-fe.gz
Distortion KENLM lazyken=0 name=LM0 factor=0
path=/project/qtleap/pilot0/systems/en-de/lm/interpolated-binlm.2 order=5
input-factors: 0
mapping: 0 T 0
weight: LexicalReordering0= 0.106673 0.0371276 0.0124599
0.0642303 0.00514413 0.0893282 Distortion0= 0.0367167 LM0= 0.0755455
WordPenalty0= -0.25141 PhrasePenalty0= 0.080744 TranslationModel0=
-0.00207054 0.0663382 0.160812 0.0113997
line=UnknownWordPenalty
FeatureFunction: UnknownWordPenalty0 start: 0 end: 0
line=WordPenalty
FeatureFunction: WordPenalty0 start: 1 end: 1
line=PhrasePenalty
FeatureFunction: PhrasePenalty0 start: 2 end: 2
line=PhraseDictionaryCompact name=TranslationModel0 table-limit=20
num-features=4
path=/project/qtleap/pilot0/systems/en-de/model/phrase-table.2.minphr
input-factor=0 output-factor=0
FeatureFunction: TranslationModel0 start: 3 end: 6
line=LexicalReordering name=LexicalReordering0 num-features=6
type=wbe-msd-bidirectional-fe-allff input-factor=0 output-factor=0
path=/project/qtleap/pilot0/systems/en-de/model/reordering-table.2.wbe-msd-bidirectional-fe.gz
FeatureFunction: LexicalReordering0 start: 7 end: 12
Initializing LexicalReordering..
line=Distortion
FeatureFunction: Distortion0 start: 13 end: 13
line=KENLM lazyken=0 name=LM0 factor=0
path=/project/qtleap/pilot0/systems/en-de/lm/interpolated-binlm.2 order=5
FeatureFunction: LM0 start: 14 end: 14
Loading UnknownWordPenalty0
Loading WordPenalty0
Loading PhrasePenalty0
Loading LexicalReordering0
Loading table into memory...done.
Loading Distortion0
Loading LM0
Loading TranslationModel0
terminate called after throwing an instance of 'util::Exception'
what():
moses/TranslationModel/CompactPT/PhraseDictionaryCompact.cpp:104 in
virtual void Moses::PhraseDictionaryCompact::Load() threw
util::Exception because `indexSize == 0 || coderSize == 0 ||
phraseSize == 0'.
Not successfully loaded
best
Lefteris, Lukas
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
_______________________________________________
Moses-support mailing list
[email protected]
http://mailman.mit.edu/mailman/listinfo/moses-support
