Thanks, Barry, your answer is perfectly helpful. I see where I was mistaken. I thought threading in moses meant multi-threaded decoding (search), but it turns out it distributes sentences over n decoders each running in separate thread. This explains why I got high CPU utilization when running moses in standalone mode. Indeed, I'm building a scalable production translation system here, and this question arose while optimizing the utilization of a 16-core machine.
However, if I understand it correctly, the current approach to multi-threading in moses means that there is some idle time while waiting for the sentences to be translated in the same order as in input (both input and output are still serial). Are there any plans to make the decoding itself multi-threaded? Regards, Karlis > Hi Karlis > > In both moses and mosesserver, the unit of work is the sentence. So > moses > processes each sentence in its own thread, and does not split it up as > you > suggest. > > I think the difference in semantics is reasonable, as multi-threading > has > different aims in moses and mosesserver. In moses, multi-threading > allows the > decoder to split a file into parts, and process each part in a separate > thread, but sharing language and translation models. > > However in mosesserver, multithreading allows several clients to > connect > simultaneously, wirhout blocking each other. The moses server API only > accepts one sentence at a time, so it doesn't have the opportunity to > split > the input into sentences. If you're building an online translation > system, > then you probably have another component sitting in front of > mosesserver > which handles tokenisation, casing and splitting, which can take care > of > parallelisation. > > hope that helps - best regards - Barry _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
