* Looking for MT/NLP opportunities * Hieu Hoang http://moses-smt.org/
On 16 March 2017 at 12:25, Ivan Zapreev <[email protected]> wrote: > Dear All, > > I have a question and perhaps want to draw your attention to the following > fact. The Moses2 webpage http://www.statmt.org/moses/?n=Site.Moses2 > claims that Moses2 has better multi-threading support than Moses and scales > better on multi-core machines. As can be extrapolated from the performance > picture on that webpage the performance of Moses and Moses2 on a single > core are therefore almost the same. > > We develop our own free and open source distributed SMT infrastructure and > our empirical comparison of the translation tools shows that things are not > quite like that. Actually Moses2 is about 2 times faster than Moses on a > single thread and with adding more threads on a multi-core machine this > speed difference is only reducing ... We actually observe that Moses2 > scales worse in the number of threads than Moses and that the performance > benefits of Moses2 seem to be solely due to faster single sentence > decoding. So I am curious why do we have so much different results from the > official ones? Could some one please give me a hint on that? Is the > information on the Moses2 webpage outdated? > I'm curious too. The webpage is up to date, the info should be all correct > > The results of our empirical evaluation can be found by the following link: > > https://github.com/ivan-zapreev/Basic-Translation-Infrastruc > ture#translation-server-evaluation > > The experimental setup is thoroughly described in: > > 1. https://github.com/ivan-zapreev/Basic-Translation- > Infrastructure#test-set-up-1 > 2. https://github.com/ivan-zapreev/Basic-Translation- > Infrastructure#test-server > 3. https://github.com/ivan-zapreev/Basic-Translation- > Infrastructure#mosesmoses2 > > Some clarifications: So your server has 20 cores (40 hyperthreads) and 16GB RAM? If that's correct, then the RAM size would be a problem - you need as much RAM as the total size of your models, plus more for working memory and the OS. Do you run Moses command line, or the server? My timings are based on the command line, the server is a little slower. Do you run Moses directly, or is another evaluation process running it? Are you sure that evaluation process is working as it should? Do you minimise the effect of disk read by pre-loading the models into filesystem cache? This is usually done by running this before running the decoder cat [binary model files] > /dev/null > Clearly, the models were located on a local disk and there were no > processes affecting the timing results. The experiments were repeated > multiple times and average values with standard deviations were obtained. > > We can not make the models available just like that, as for one they are > very big, see the experimental setup description: > it may take a while, but I can't replicate your results without it. Alternatively, I can provide you with my models so you can try & replicate my results. > > - *Language Model* - 48.9 Gb (5-gram model); > - *Translation Model* - 1.3 Gb (5 features model); > - *Reordering Model* - 9.7 Gb (8 features model); > > Moreover, they are obtained on a Chinese to English OpenMT MT-04 data set > so they can be reproduced from there. > > Thank you in advance! > > Kind regards, > > Dr. Ivan S. Zapreev > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > >
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
