Thank you. In our tests it seems that with the OnDisk table the quality is basically the same between the two versions of Moses (average 0.2 difference in score Bleu) but for the CompactPhraseTable the difference is larger (2 points Bleu loss in average after re-tuning with the new version of Moses, and more than 5 points Bleu in average without re-tuning). Do you think a better quality would be obtained by running a complete re-training of the model with the new version of Moses?
Best regards, Vito 2015-11-24 16:31 GMT+01:00 Hieu Hoang <[email protected]>: > There was a change in the underlying datastructure for stacks, it changed > from std::set (ordered) to boost::unordered_set. > > https://github.com/moses-smt/mosesdecoder/commit/6b182ee5e987a5b2823aea7eaaa7ef0457c6a30d > This got some speed gains > > 1 5 10 15 20 25 30 35 > 56 real 4m57.795s real 1m19.005s real 0m51.636s real 0m49.624s > real 0m49.869s real 0m52.475s real 0m53.806s real 0m54.957s 13/10 > baseline user 4m41.255s user 5m45.086s user 6m34.053s user > 8m12.430s user 8m10.667s user 8m16.486s user 8m10.592s user > 8m13.859s > sys 0m16.514s sys 0m35.494s sys 0m54.513s sys 1m10.643s > sys 1m18.449s sys 1m21.738s sys 1m23.133s sys 1m25.048s > > > > > > > > > > 57 real 4m41.148s real 1m16.002s real 0m50.747s real 0m48.711s > real 0m49.130s real 0m51.473s real 0m53.141s real 0m54.513s (56) > + unordered set stack user 4m23.968s user 5m30.356s user > 6m26.167s user 7m39.286s user 7m56.229s user 7m52.669s user > 7m56.978s user 7m56.216s > sys 0m17.231s sys 0m35.063s sys 0m54.081s sys 1m10.137s > sys 1m17.194s sys 1m22.912s sys 1m25.948s sys 1m26.247s > However, the hypotheses are now added to the stack in a different order so > there will be slight differences in results > > > On 24/11/2015 13:53, Vito Mandorino wrote: > > Hi, > > in some of our tests a recent version of Moses (pulled from github last > week) and an older one do not give the same translations on the same source > segment (with the same moses.ini). > Here is the 5-best list for the translation of 'test' with the last week > version: > > 0 ||| test ||| LexicalReordering0= -1.1969 0 0 0 0 0 Distortion0= 0 LM0= > -51.1788 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -3.03811 -2.5834 -2.08503 -1.83075 ||| -1.27754 > 0 ||| testing ||| LexicalReordering0= 0 0 0 0 0 0 Distortion0= 0 LM0= > -35.1495 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -5.21045 -5.04877 -4.71131 -4.66382 ||| -1.70337 > 0 ||| funds ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0= > -11.3753 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -10.8209 -10.6835 -5.14555 -5.73388 ||| -1.77009 > 0 ||| known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 > LM0= -58.8877 WordPenalty0= -3 PhrasePenalty0= 1 > PhraseDictionaryMultiModel0= -4.42285 -11.9339 -5.14555 -18.0392 ||| > -1.89152 > 0 ||| as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0= > -35.5353 WordPenalty0= -2 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -9.34698 -11.9339 -5.14555 -9.14874 ||| -1.89159 > > and with the older version of Moses: > > 0 ||| funds ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0= > -11.3753 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -2.52548 -2.52544 -2.45544 -2.48609 ||| -0.815668 > 0 ||| as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0= > -35.5353 WordPenalty0= -2 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -2.52464 -2.52565 -2.45544 -2.5244 ||| -0.953799 > 0 ||| as ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 LM0= > -34.1633 WordPenalty0= -1 PhrasePenalty0= 1 PhraseDictionaryMultiModel0= > -2.5256 -2.52565 -2.45544 -2.48609 ||| -1.07254 > 0 ||| known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 Distortion0= 0 > LM0= -58.8877 WordPenalty0= -3 PhrasePenalty0= 1 > PhraseDictionaryMultiModel0= -2.38597 -2.52565 -2.45544 -2.52573 ||| > -1.07536 > 0 ||| is known as a ||| LexicalReordering0= -3.1355 0 0 0 0 0 > Distortion0= 0 LM0= -80.8518 WordPenalty0= -4 PhrasePenalty0= 1 > PhraseDictionaryMultiModel0= -2.37158 -2.52565 -2.45544 -2.52573 ||| > -1.18753 > > This looks very strange. The only difference is in the phrase-table > scores. Do you have any idea of what is going on? The only possibility > which come to mind is maybe a different handling of the > PhraseDictionaryMultiModel feature. > The moses.ini is in attachment. > > Best regards, > > Vito > > > > -- > *M**. Vito MANDORINO -- Chief Scientist* > > > [image: Description : Description : lingua_custodia_final full logo] > > *The Translation Trustee* > > *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux* > > *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89 > <%2B33%206%2084%2065%2068%2089>* > > *Email :* * > <[email protected]>[email protected] > <[email protected]>* > > *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> - > www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>* > > > _______________________________________________ > Moses-support mailing > [email protected]http://mailman.mit.edu/mailman/listinfo/moses-support > > > -- > Hieu Hoanghttp://www.hoang.co.uk/hieu > > -- *M**. Vito MANDORINO -- Chief Scientist* [image: Description : Description : lingua_custodia_final full logo] *The Translation Trustee* *1, Place Charles de Gaulle, **78180 Montigny-le-Bretonneux* *Tel : +33 1 30 44 04 23 Mobile : +33 6 84 65 68 89 <%2B33%206%2084%2065%2068%2089>* *Email :* *[email protected] <[email protected]>* *Website :* *www.linguacustodia.com <http://www.linguacustodia.com/> - www.thetranslationtrustee.com <http://www.thetranslationtrustee.com/>*
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
