The score program should be a good candidate for parallelisation. It take an unnecessarily long time, especially for hiero grammar. Some diddly stuff you have to do for good-Turing discount
Adam's suufix array has also been partially integrated into Moses, it needs to be finished. Disk space and io are also issues for hiero grammar Hieu Sent from my flying horse On 18 Apr 2012, at 07:01 AM, Philipp Koehn <[email protected]> wrote: > Hi everybody, > > the trend in computing is towards machines with more and more CPUs, > and Moses can already take advantage of this during decoder. Just > specify "-threads 10" and decoding is ten times faster (well, almost, > and you actually have to have 10 CPUs on you machine). > > The training pipeline, however, is mostly single-threaded. > > Maybe you want to change that and learn something about threads > in C++ in the process. > > We added some documentation about multi-threading in Moses: > http://www.statmt.org/moses/?n=Moses.CodeGuideThreading > which should get you started. > > There are several instant candidates: the phrase and rule extractors > (only the hierarchical extract-rules is multi-threaded), rule scoring, > word alignment symmetrization, etc. Maybe even some code that > is currently in Perl or Python, such as the rule filtering, tokenization > or truecasing. > > -phi > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
