John, LM portions are loaded from disk on demand and memory is releases after decoding each sentence. However, be sure that the disk does not become a bottleneck. If you run moses-parallel on many machines, you could better make copies of the LM on all machines, in the same directory e.g. /tmp/lm.mm ..
Marcello FBK-irst ________________________________________ From: [EMAIL PROTECTED] [EMAIL PROTECTED] On Behalf Of John D. Burger [EMAIL PROTECTED] Sent: Tuesday, August 12, 2008 10:02 PM To: [email protected] Subject: Re: [Moses-support] R: Binary IRSTLM models Marcello Federico wrote: > (no MAP) means that no memory > mapping is active. You should try > with the suffix .mm for the lm file. Aha! Now it says "mapping 70326518 5-grams" etc., instead of "loading ..." It seems slightly faster, too. If I understand correctly, this should run with a smaller memory footprint, and multiple moses executables running at the same time can share some of their segments, right? I imagine this will combine nicely with moses-parallel, yes? Thanks! - John Burger MITRE _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
