Hi, >From the toy phrase based tutorial at [1], I'm using a modified phrase table (I replaced the German with phrases in a different source language, the target language is still english). And I'm finding that a certain source phrase causes the decoder to seg fault.
Digging deeper into the verbose output I see that after the translation option collection it never enters the hypothesis stage even though there is at least one path available (see the verbose output below). The source phrase has 8 tokens and the translation "this is a" covers [0-5][6-7]. What would prevent the decoder from entering the hypothesis generation stage? I did notice that the last 3 lines ([1 ; 1-1], etc) are not present in successful source translation output. Thank you for your time. [1] http://www.statmt.org/moses/?n=Moses.Tutorial cheers kwame --- Defined parameters (per moses.ini or switch): beam-threshold: 0 config: phrase-model/moses.ini feature: KENLM name=LM factor=0 order=3 num-features=1 path=/home/kwame/Moses/moses-linux-64bit/sample-models/lm/europarl.srilm.gz WordPenalty PhraseDictionaryMemory input-factor=0 output-factor=0 path=/home/kwame/Moses/moses-linux-64bit/sample-models/phrase-model/fixed.sorted.bit num-features=1 table-limit=100 input-factors: 0 mapping: T 0 report-segmentation: stack: 1000 verbose: 3 2 weight: WordPenalty0= 0 LM= 1 PhraseDictionaryMemory0= 1 input type is: text input line=KENLM name=LM factor=0 order=3 num-features=1 path=/home/kwame/Moses/moses-linux-64bit/sample-models/lm/europarl.srilm.gz FeatureFunction: LM start: 0 end: 0 Loading the LM will be faster if you build a binary file. Reading /home/kwame/Moses/moses-linux-64bit/sample-models/lm/europarl.srilm.gz ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 **The ARPA file is missing <unk>. Substituting log10 probability -100.000. ************************************************************************************************** line=WordPenalty FeatureFunction: WordPenalty0 start: 1 end: 1 line=PhraseDictionaryMemory input-factor=0 output-factor=0 path=/home/kwame/Moses/moses-linux-64bit/sample-models/phrase-model/fixed.sorted.bit num-features=1 table-limit=100 FeatureFunction: PhraseDictionaryMemory0 start: 2 end: 2 DecodeFeature: Loading LM Loading WordPenalty0 Loading PhraseDictionaryMemory0 Start loading text phrase table. Moses format : [0.824] seconds Reading /home/kwame/Moses/moses-linux-64bit/sample-models/phrase-model/fixed.sorted.bit ----5---10---15---20---25---30---35---40---45---50---55---60---65---70---75---80---85---90---95--100 **************************************************************************************************** DecodeStep(): outputFactors=FactorMask<0> conflictFactors=FactorMask<> newOutputFactors=FactorMask<0> Created input-output object : [0.825] seconds The global weight vector looks like this: core=(1.000,0.000,1.000) Translating line 0 in thread id 140062576498432 Translating: 0 1 1 0 1 0 0 0 InitializeForInput( LM ) = 0.000 InitializeForInput( WordPenalty0 ) = 0.000 InitializeForInput( PhraseDictionaryMemory0 ) = 0.000 Line 0: Initialize search took 0.001 seconds total ... Translation Option Collection a :: term=: nonterm=: c=-5.515 core=(0.000,-1.000,0.000) c=-5.515 [[0..0]]core=(0.000,-1.000,0.000) the :: term=: nonterm=: c=-5.789 core=(0.000,-1.000,-0.916) c=-5.789 [[0..2]]core=(0.000,-1.000,-0.916) this :: term=: nonterm=: c=-8.002 core=(0.000,-1.000,-2.303) c=-8.002 [[0..3]]core=(0.000,-1.000,-2.303) this is :: term=: nonterm=: c=-10.291 core=(0.000,-2.000,-0.223) c=-10.291 [[0..5]]core=(0.000,-2.000,-0.223) or :: term=: nonterm=: c=-7.720 core=(0.000,-1.000,-2.303) c=-7.720 [[1..2]]core=(0.000,-1.000,-2.303) is :: term=: nonterm=: c=-4.922 core=(0.000,-1.000,0.000) c=-4.922 [[2..3]]core=(0.000,-1.000,0.000) it is :: term=: nonterm=: c=-8.821 core=(0.000,-2.000,-0.223) c=-8.821 [[2..5]]core=(0.000,-2.000,-0.223) a :: term=: nonterm=: c=-5.515 core=(0.000,-1.000,0.000) c=-5.515 [[3..3]]core=(0.000,-1.000,0.000) is :: term=: nonterm=: c=-4.922 core=(0.000,-1.000,0.000) c=-4.922 [[4..5]]core=(0.000,-1.000,0.000) old :: term=: nonterm=: c=-8.945 core=(0.000,-1.000,-0.223) c=-8.945 [[4..6]]core=(0.000,-1.000,-0.223) a :: term=: nonterm=: c=-5.515 core=(0.000,-1.000,0.000) c=-5.515 [[5..5]]core=(0.000,-1.000,0.000) 's :: term=: nonterm=: c=-6.116 core=(0.000,-1.000,0.000) c=-6.116 [[5..6]]core=(0.000,-1.000,0.000) an :: term=: nonterm=: c=-6.413 core=(0.000,-1.000,0.000) c=-6.413 [[5..6]]core=(0.000,-1.000,0.000) a :: term=: nonterm=: c=-5.515 core=(0.000,-1.000,0.000) c=-5.515 [[6..6]]core=(0.000,-1.000,0.000) 's :: term=: nonterm=: c=-6.116 core=(0.000,-1.000,0.000) c=-6.116 [[6..7]]core=(0.000,-1.000,0.000) an :: term=: nonterm=: c=-6.413 core=(0.000,-1.000,0.000) c=-6.413 [[6..7]]core=(0.000,-1.000,0.000) a :: term=: nonterm=: c=-5.515 core=(0.000,-1.000,0.000) c=-5.515 [[7..7]]core=(0.000,-1.000,0.000) [1 ; 1-1] [1 ; 2-2] [1 ; 4-4] ---
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
