I don't change the binary file format without updating the version number so old versions won't load. The recent versions shouldn't impact that.
Sounds like a case for gdb. On 06/26/11 08:22, Hieu Hoang wrote: > i believe there's been changes to the binary phrase table (to the > support word alignment) and kenlm (speed/bug fixes). > > are you binarizing the LM and pt with latest binarizer, as well as > running with the latest decoder? > > On 26/06/2011 11:15, Tom Hoar wrote: >> >> I updated my Moses to the newest svn rev 4036 which includes the >> segfault fix from 4030. It didn't help. My translations still generate >> the same segmentation fault as below. >> >> >> >> Tom >> >> >> >> On Sat, 25 Jun 2011 23:34:49 +0700, Tom Hoar >> <[email protected]> wrote: >> >>> I built translation model several months ago with moses svn rev 3675 >>> (~ Oct 2010). I'm trying to run under a newer moses build (svn 4011). >>> I re-binarized the kenlm language model to the newer supported >>> version (4). Running moses ended in a Segmentation fault. So, I >>> re-binarized the phrase table (this translation model does not have a >>> reordering table). Running still ends with a segmentation fault. The >>> Moses output running in a terminal stdin/stdout is below. >>> >>> Other translation models that I've trained with the new (svn 4011) >>> build work fine. Have there been any changes that require that I >>> re-train/re-tune this model? >>> >>> Thanks, >>> Tom >>> >>> >>> >>> user@host:~$ moses -f >>> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf >>> -v 4 >>> Defined parameters (per moses.ini or switch): >>> config: >>> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf >>> >>> distortion-limit: 0 >>> input-factors: 0 >>> lmodel-file: 9 0 5 >>> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm >>> mapping: 0 T 0 >>> threads: 4 >>> ttable-file: 1 0 0 5 >>> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table >>> ttable-limit: 20 >>> v: 0 >>> verbose: 4 >>> weight-d: 1.71428e-06 >>> weight-l: 0.198294 >>> weight-t: 0.224902 0.00343444 0.1214 6.07841e-07 -0.276618 >>> weight-w: -0.175349 >>> xml-input: exclusive >>> Added ScoreProducer(0 Distortion) index=0-0 >>> input type is: text input >>> Added ScoreProducer(1 WordPenalty) index=1-1 >>> Added ScoreProducer(2 !UnknownWordPenalty) index=2-2 >>> Loading lexical distortion models...have 0 models >>> Start loading LanguageModel >>> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm : >>> [0.000] seconds >>> Added ScoreProducer(3 LM_5gram) index=3-3 >>> Finished loading LanguageModels : [0.000] seconds >>> Creating phrase table features >>> Start loading PhraseTable >>> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table >>> : [0.000] seconds >>> filePath: >>> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table >>> DecodeFeature: input=FactorMask<0> output=FactorMask<0> >>> Added ScoreProducer(4 PhraseModel) index=4-8 >>> Finished loading phrase tables : [0.000] seconds >>> DecodeStep(): >>> outputFactors=FactorMask<0> >>> conflictFactors=FactorMask<> >>> newOutputFactors=FactorMask<0> >>> Adding decoder graph 0 to translation system default >>> Adding language model 0 to translation system default >>> IO from STDOUT/STDIN >>> Created input-output object : [0.000] seconds >>> The score component vector looks like this: >>> Distortion >>> WordPenalty >>> !UnknownWordPenalty >>> LM_5gram >>> PhraseModel_1 >>> PhraseModel_2 >>> PhraseModel_3 >>> PhraseModel_4 >>> PhraseModel_5 >>> >>> The global weight vector looks like this: 0.000 -0.175 1.000 0.198 >>> 0.225 0.003 0.121 0.000 -0.277 >>> 種 >>> Translating line 0 in thread id 140373814597376 >>> Translating: 種 >>> >>> reading bin ttable >>> size of OFF_T 8 >>> binary phrasefile loaded, default OFF_T: -1 >>> Segmentation fault >>> user@host:~$ >>> >>> ま り ヰ 洋 種 >>> >>> user@host:~$ moses -f >>> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf >>> -v 4 >>> Defined parameters (per moses.ini or switch): >>> config: >>> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_all-T=kenlm-n=5.conf >>> >>> distortion-limit: 0 >>> input-factors: 0 >>> lmodel-file: 9 0 5 >>> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm >>> mapping: 0 T 0 >>> threads: 4 >>> ttable-file: 1 0 0 5 >>> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table >>> ttable-limit: 20 >>> v: 0 >>> verbose: 4 >>> weight-d: 1.71428e-06 >>> weight-l: 0.198294 >>> weight-t: 0.224902 0.00343444 0.1214 6.07841e-07 -0.276618 >>> weight-w: -0.175349 >>> xml-input: exclusive >>> Added ScoreProducer(0 Distortion) index=0-0 >>> input type is: text input >>> Added ScoreProducer(1 WordPenalty) index=1-1 >>> Added ScoreProducer(2 !UnknownWordPenalty) index=2-2 >>> Loading lexical distortion models...have 0 models >>> Start loading LanguageModel >>> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm : >>> [0.000] seconds >>> Added ScoreProducer(3 LM_5gram) index=3-3 >>> Finished loading LanguageModels : [0.000] seconds >>> Creating phrase table features >>> Start loading PhraseTable >>> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table >>> : [0.000] seconds >>> filePath: >>> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/phrase-table >>> DecodeFeature: input=FactorMask<0> output=FactorMask<0> >>> Added ScoreProducer(4 PhraseModel) index=4-8 >>> Finished loading phrase tables : [0.000] seconds >>> DecodeStep(): >>> outputFactors=FactorMask<0> >>> conflictFactors=FactorMask<> >>> newOutputFactors=FactorMask<0> >>> Adding decoder graph 0 to translation system default >>> Adding language model 0 to translation system default >>> IO from STDOUT/STDIN >>> Created input-output object : [0.000] seconds >>> The score component vector looks like this: >>> Distortion >>> WordPenalty >>> !UnknownWordPenalty >>> LM_5gram >>> PhraseModel_1 >>> PhraseModel_2 >>> PhraseModel_3 >>> PhraseModel_4 >>> PhraseModel_5 >>> >>> The global weight vector looks like this: 0.000 -0.175 1.000 0.198 >>> 0.225 0.003 0.121 0.000 -0.277 >>> 種 >>> Translating line 0 in thread id 140373814597376 >>> Translating: 種 >>> >>> reading bin ttable >>> size of OFF_T 8 >>> binary phrasefile loaded, default OFF_T: -1 >>> Segmentation fault >>> >> >> _______________________________________________ >> Moses-support mailing list >> [email protected] >> http://mailman.mit.edu/mailman/listinfo/moses-support > > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
