barry's right. Binarize the phrase table and the LM with irstlm or kenlm. Then just look at the file sizes & add a few 100mb and that's your memory requirement for adequate speed.
You can run the phrase-based decoder in about 300mb if everything is binarized. I run it on the iphone for fun :) (the chart decoder needs 1-2gb) On 26/08/2011 15:07, Barry Haddow wrote: >> Ok, >> i discovered that probably we can have a 64gb ram 8/12 cores >> machine. >> The requirements for translation are the same for the >> training? >> >> I prepared two language models in binary format. And i >> noticed that when the server is loading/translating it takes 89/90% of >> ram (actually the test environment has 4gb of RAM), and 10% of cpu. >> But >> when there aren't pending translation the memory used is 0%. >> So for >> translation machine i still need a 8/12 cores, or i can have a >> "smaller" machine? >> For translation what is important? Memory or CPU? >> >> And for example with 64gb ram, approximatively how many models can i >> load on the same machine (suppose we have models with ~400'000/800'000 >> sentences)? >> > > Hi Ivan > > As far as ram is concerned, you need enough to load your model, any more won't > make much difference, and any less then it will run impossibly slow due to > swapping. > > If your data is processed in batches then you can benefit from having more > CPUs and running multi-threaded decoding. > > I'm afraid I've no figures mapping training sentences to model size. I'd > suggest that you run some experiments in your setup. > > cheers - Barry > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
