I don't find anything wrong of this sentence in the test set. Other candidates of this sentence is good in the same batch of output. This problem occurs randomly (random sentence and candidate) during tuning.
2017-04-12 21:48, Hieu Hoang: > It looks like there is a phrase that is length 0, hence ' = 1'. > > Check your data has been cleaned and encoded correctly > > * Looking for MT/NLP opportunities * > Hieu Hoang > http://moses-smt.org/ > > > On 12 April 2017 at 13:36, Dingyuan Wang <[email protected] > <mailto:[email protected]>> wrote: > > Dear all, > > I come across the exactly same problem a year ago (follow the thread): > > https://www.mail-archive.com/[email protected]/msg13673.html > <https://www.mail-archive.com/[email protected]/msg13673.html> > > The moses constantly and randomly output corrupted best100 out that > crashes further kbmira tuning. Like: > > 45 ||| “ 愿 以 车 马 衣 裘 等 皆 与 朋 友 共 分 共 ， 则 皆 敝 之 亦 > 无 所 恨 > 。 ” ||| LexicalReordering0= -6.1176 0 0 -6.58298 0 0 Distortion0= 0 > LM0= -115.094 TWI_，= 0 SWD_OTHER= 2 WT_都~皆= 2 WT_OTHER~OTHER= 12 > WT_也~OTHER= 1 WT_把~以= 1 WT_，~，= 1 WT_等~OTHER= 1 WT_OTHER~所= 1 > WT_OTHER~无= 0 WT_OTHER~以= 1 WT_没有~无= 1 WT_了~之= 1 WT_”~”= 1 > WT_。~。= 1 > WT_也~而= 0 = 1 WT_OTHER~则= 1 WT_和~与= 1 WT_了~OTHER= 0 PL_t2= 5 > PL_s2= 4 > PL_1,2= 2 PL_3,4= 0 PL_s3= 1 WordPenalty0= -26 PhrasePenalty0= 21 > TranslationModel0= -66.0904 -70.4587 -24.5341 -28.4086 ||| -15.012 > > There is an error in "WT_也~而= 0 = 1". Then kbmira: > > kbmira with c=0.01 decay=0.999 no_shuffle=0 > Initialising random seed from system clock > terminate called after throwing an instance of > 'MosesTuning::FileFormatException' > what(): Error in line "-6.1176 0 0 -6.58298 0 0 0 -115.094 1 -26 21 > -66.0904 -70.4587 -24.5341 -28.4086 SWD_OTHER=2 WT_，~，=1 > WT_OTHER~OTHER=12 PL_t2=5 PL_s3=1 PL_s2=4 PL_1,2=2 WT_”~”=1 WT_。~。=1 > WT_没有~无=1 WT_了~之=1 WT_OTHER~以=1 WT_都~皆=2 WT_OTHER~所=1 > WT_OTHER~则=1 > WT_把~以=1 WT_和~与=1 WT_等~OTHER=1 WT_也~OTHER=1 " of run1.features.dat > Aborted (core dumped) > > System is Debian 9 (stretch/testing) with GCC 6.3.0, moses latest git > checkout. > > -- > Dingyuan Wang > _______________________________________________ > Moses-support mailing list > [email protected] <mailto:[email protected]> > http://mailman.mit.edu/mailman/listinfo/moses-support > <http://mailman.mit.edu/mailman/listinfo/moses-support> > > -- Dingyuan Wang _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
