Dear All, I have a question and perhaps want to draw your attention to the following fact. The Moses2 webpage http://www.statmt.org/moses/?n=Site.Moses2 claims that Moses2 has better multi-threading support than Moses and scales better on multi-core machines. As can be extrapolated from the performance picture on that webpage the performance of Moses and Moses2 on a single core are therefore almost the same.
We develop our own free and open source distributed SMT infrastructure and our empirical comparison of the translation tools shows that things are not quite like that. Actually Moses2 is about 2 times faster than Moses on a single thread and with adding more threads on a multi-core machine this speed difference is only reducing ... We actually observe that Moses2 scales worse in the number of threads than Moses and that the performance benefits of Moses2 seem to be solely due to faster single sentence decoding. So I am curious why do we have so much different results from the official ones? Could some one please give me a hint on that? Is the information on the Moses2 webpage outdated? The results of our empirical evaluation can be found by the following link: https://github.com/ivan-zapreev/Basic-Translation- Infrastructure#translation-server-evaluation The experimental setup is thoroughly described in: 1. https://github.com/ivan-zapreev/Basic-Translation-Infrastructure#test-set-up-1 2. https://github.com/ivan-zapreev/Basic-Translation-Infrastructure#test-server 3. https://github.com/ivan-zapreev/Basic-Translation-Infrastructure#mosesmoses2 Clearly, the models were located on a local disk and there were no processes affecting the timing results. The experiments were repeated multiple times and average values with standard deviations were obtained. We can not make the models available just like that, as for one they are very big, see the experimental setup description: - *Language Model* - 48.9 Gb (5-gram model); - *Translation Model* - 1.3 Gb (5 features model); - *Reordering Model* - 9.7 Gb (8 features model); Moreover, they are obtained on a Chinese to English OpenMT MT-04 data set so they can be reproduced from there. Thank you in advance! Kind regards, Dr. Ivan S. Zapreev
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
