Hi Tom, Hieu If there have been some changes that make old formats incompatible, it would be good to have an error message rather than a segfault.
Tom - would you be able to get a stack trace and/or make your model available somewhere in order to reproduce the problem? best regards - Barry On Sunday 26 Jun 2011 13:22:53 Hieu Hoang wrote: > i believe there's been changes to the binary phrase table (to the > support word alignment) and kenlm (speed/bug fixes). > > are you binarizing the LM and pt with latest binarizer, as well as > running with the latest decoder? > > On 26/06/2011 11:15, Tom Hoar wrote: > > I updated my Moses to the newest svn rev 4036 which includes the > > segfault fix from 4030. It didn't help. My translations still generate > > the same segmentation fault as below. > > > > Tom > > > > On Sat, 25 Jun 2011 23:34:49 +0700, Tom Hoar > > > > <[email protected]> wrote: > >> I built translation model several months ago with moses svn rev 3675 > >> (~ Oct 2010). I'm trying to run under a newer moses build (svn 4011). > >> I re-binarized the kenlm language model to the newer supported > >> version (4). Running moses ended in a Segmentation fault. So, I > >> re-binarized the phrase table (this translation model does not have a > >> reordering table). Running still ends with a segmentation fault. The > >> Moses output running in a terminal stdin/stdout is below. > >> > >> Other translation models that I've trained with the new (svn 4011) > >> build work fine. Have there been any changes that require that I > >> re-train/re-tune this model? > >> > >> Thanks, > >> Tom > >> > >> user@host:~$ moses -f > >> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_ > >>all-T=kenlm-n=5.conf -v 4 > >> Defined parameters (per moses.ini or switch): > >> config: > >> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_ > >>all-T=kenlm-n=5.conf > >> > >> distortion-limit: 0 > >> input-factors: 0 > >> lmodel-file: 9 0 5 > >> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm > >> mapping: 0 T 0 > >> threads: 4 > >> ttable-file: 1 0 0 5 > >> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/ph > >>rase-table ttable-limit: 20 > >> v: 0 > >> verbose: 4 > >> weight-d: 1.71428e-06 > >> weight-l: 0.198294 > >> weight-t: 0.224902 0.00343444 0.1214 6.07841e-07 -0.276618 > >> weight-w: -0.175349 > >> xml-input: exclusive > >> Added ScoreProducer(0 Distortion) index=0-0 > >> input type is: text input > >> Added ScoreProducer(1 WordPenalty) index=1-1 > >> Added ScoreProducer(2 !UnknownWordPenalty) index=2-2 > >> Loading lexical distortion models...have 0 models > >> Start loading LanguageModel > >> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm : > >> [0.000] seconds > >> Added ScoreProducer(3 LM_5gram) index=3-3 > >> Finished loading LanguageModels : [0.000] seconds > >> Creating phrase table features > >> Start loading PhraseTable > >> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/ph > >>rase-table > >> > >> : [0.000] seconds > >> > >> filePath: > >> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/ph > >>rase-table DecodeFeature: input=FactorMask<0> output=FactorMask<0> > >> Added ScoreProducer(4 PhraseModel) index=4-8 > >> Finished loading phrase tables : [0.000] seconds > >> DecodeStep(): > >> outputFactors=FactorMask<0> > >> conflictFactors=FactorMask<> > >> newOutputFactors=FactorMask<0> > >> Adding decoder graph 0 to translation system default > >> Adding language model 0 to translation system default > >> IO from STDOUT/STDIN > >> Created input-output object : [0.000] seconds > >> The score component vector looks like this: > >> Distortion > >> WordPenalty > >> !UnknownWordPenalty > >> LM_5gram > >> PhraseModel_1 > >> PhraseModel_2 > >> PhraseModel_3 > >> PhraseModel_4 > >> PhraseModel_5 > >> > >> The global weight vector looks like this: 0.000 -0.175 1.000 0.198 > >> 0.225 0.003 0.121 0.000 -0.277 > >> 種 > >> Translating line 0 in thread id 140373814597376 > >> Translating: 種 > >> > >> reading bin ttable > >> size of OFF_T 8 > >> binary phrasefile loaded, default OFF_T: -1 > >> Segmentation fault > >> user@host:~$ > >> > >> ま り ヰ 洋 種 > >> > >> user@host:~$ moses -f > >> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_ > >>all-T=kenlm-n=5.conf -v 4 > >> Defined parameters (per moses.ini or switch): > >> config: > >> /usr/local/etc/domy/oct-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7-l=ils_ja_ > >>all-T=kenlm-n=5.conf > >> > >> distortion-limit: 0 > >> input-factors: 0 > >> lmodel-file: 9 0 5 > >> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm > >> mapping: 0 T 0 > >> threads: 4 > >> ttable-file: 1 0 0 5 > >> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/ph > >>rase-table ttable-limit: 20 > >> v: 0 > >> verbose: 4 > >> weight-d: 1.71428e-06 > >> weight-l: 0.198294 > >> weight-t: 0.224902 0.00343444 0.1214 6.07841e-07 -0.276618 > >> weight-w: -0.175349 > >> xml-input: exclusive > >> Added ScoreProducer(0 Distortion) index=0-0 > >> input type is: text input > >> Added ScoreProducer(1 WordPenalty) index=1-1 > >> Added ScoreProducer(2 !UnknownWordPenalty) index=2-2 > >> Loading lexical distortion models...have 0 models > >> Start loading LanguageModel > >> /opt/ils/ENGINES/lms/lm-t=ja-l=ils_ja_all-T=kenlm-n=5/kenlm.ja.mm : > >> [0.000] seconds > >> Added ScoreProducer(3 LM_5gram) index=3-3 > >> Finished loading LanguageModels : [0.000] seconds > >> Creating phrase table features > >> Start loading PhraseTable > >> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/ph > >>rase-table > >> > >> : [0.000] seconds > >> > >> filePath: > >> /opt/ils/ENGINES/tables/tables-s=ja_fr10-t=ja-p=ils_ja_all-a=giza-g=7/ph > >>rase-table DecodeFeature: input=FactorMask<0> output=FactorMask<0> > >> Added ScoreProducer(4 PhraseModel) index=4-8 > >> Finished loading phrase tables : [0.000] seconds > >> DecodeStep(): > >> outputFactors=FactorMask<0> > >> conflictFactors=FactorMask<> > >> newOutputFactors=FactorMask<0> > >> Adding decoder graph 0 to translation system default > >> Adding language model 0 to translation system default > >> IO from STDOUT/STDIN > >> Created input-output object : [0.000] seconds > >> The score component vector looks like this: > >> Distortion > >> WordPenalty > >> !UnknownWordPenalty > >> LM_5gram > >> PhraseModel_1 > >> PhraseModel_2 > >> PhraseModel_3 > >> PhraseModel_4 > >> PhraseModel_5 > >> > >> The global weight vector looks like this: 0.000 -0.175 1.000 0.198 > >> 0.225 0.003 0.121 0.000 -0.277 > >> 種 > >> Translating line 0 in thread id 140373814597376 > >> Translating: 種 > >> > >> reading bin ttable > >> size of OFF_T 8 > >> binary phrasefile loaded, default OFF_T: -1 > >> Segmentation fault > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
