I built translation model several months ago with moses svn rev 3675
(~ Oct 2010). I'm trying to run under a newer moses build (svn 4011).
I re-binarized the kenlm language model to the newer supported
version (4). Running moses ended in a Segmentation fault. So, I
re-binarized the phrase table (this translation model does not have a
reordering table). Running still ends with a segmentation fault. The
Moses output running in a terminal stdin/stdout is below.
Other translation models that I've trained with the new (svn 4011)
build work fine. Have there been any changes that require that I
re-train/re-tune this model?
Thanks,
Tom
user@host:~$ moses -f
/usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf
-v 4
Defined parameters (per moses.ini or switch):
config:
/usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf
distortion-limit: 0
input-factors: 0
lmodel-file: 9 0 5
/opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm
mapping: 0 T 0
threads: 4
ttable-file: 1 0 0 5
/opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table
ttable-limit: 20
v: 0
verbose: 4
weight-d: 1.71428e-06
weight-l: 0.198294
weight-t: 0.224902 0.00343444 0.1214 6.07841e-07 -0.276618
weight-w: -0.175349
xml-input: exclusive
Added ScoreProducer(0 Distortion) index=0-0
input type is: text input
Added ScoreProducer(1 WordPenalty) index=1-1
Added ScoreProducer(2 !UnknownWordPenalty) index=2-2
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm :
[0.000] seconds
Added ScoreProducer(3 LM_5gram) index=3-3
Finished loading LanguageModels : [0.000] seconds
Creating phrase table features
Start loading PhraseTable
/opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table
: [0.000] seconds
filePath:
/opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table
DecodeFeature: input=FactorMask<0> output=FactorMask<0>
Added ScoreProducer(4 PhraseModel) index=4-8
Finished loading phrase tables : [0.000] seconds
DecodeStep():
outputFactors=FactorMask<0>
conflictFactors=FactorMask<>
newOutputFactors=FactorMask<0>
Adding decoder graph 0 to translation system default
Adding language model 0 to translation system default
IO from STDOUT/STDIN
Created input-output object : [0.000] seconds
The score component vector looks like this:
Distortion
WordPenalty
!UnknownWordPenalty
LM_5gram
PhraseModel_1
PhraseModel_2
PhraseModel_3
PhraseModel_4
PhraseModel_5
The global weight vector looks like this: 0.000 -0.175 1.000 0.198
0.225 0.003 0.121 0.000 -0.277
種
Translating line 0 in thread id 140373814597376
Translating: 種
reading bin ttable
size of OFF_T 8
binary phrasefile loaded, default OFF_T: -1
Segmentation fault
user@host:~$
ま り ヰ 洋 種
user@host:~$ moses -f
/usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf
-v 4
Defined parameters (per moses.ini or switch):
config:
/usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf
distortion-limit: 0
input-factors: 0
lmodel-file: 9 0 5
/opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm
mapping: 0 T 0
threads: 4
ttable-file: 1 0 0 5
/opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table
ttable-limit: 20
v: 0
verbose: 4
weight-d: 1.71428e-06
weight-l: 0.198294
weight-t: 0.224902 0.00343444 0.1214 6.07841e-07 -0.276618
weight-w: -0.175349
xml-input: exclusive
Added ScoreProducer(0 Distortion) index=0-0
input type is: text input
Added ScoreProducer(1 WordPenalty) index=1-1
Added ScoreProducer(2 !UnknownWordPenalty) index=2-2
Loading lexical distortion models...have 0 models
Start loading LanguageModel
/opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm :
[0.000] seconds
Added ScoreProducer(3 LM_5gram) index=3-3
Finished loading LanguageModels : [0.000] seconds
Creating phrase table features
Start loading PhraseTable
/opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table
: [0.000] seconds
filePath:
/opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table
DecodeFeature: input=FactorMask<0> output=FactorMask<0>
Added ScoreProducer(4 PhraseModel) index=4-8
Finished loading phrase tables : [0.000] seconds
DecodeStep():
outputFactors=FactorMask<0>
conflictFactors=FactorMask<>
newOutputFactors=FactorMask<0>
Adding decoder graph 0 to translation system default
Adding language model 0 to translation system default
IO from STDOUT/STDIN
Created input-output object : [0.000] seconds
The score component vector looks like this:
Distortion
WordPenalty
!UnknownWordPenalty
LM_5gram
PhraseModel_1
PhraseModel_2
PhraseModel_3
PhraseModel_4
PhraseModel_5
The global weight vector looks like this: 0.000 -0.175 1.000 0.198
0.225 0.003 0.121 0.000 -0.277
種
Translating line 0 in thread id 140373814597376
Translating: 種
reading bin ttable
size of OFF_T 8
binary phrasefile loaded, default OFF_T: -1
Segmentation fault
