Hi Guchun,
May I ask what motivates you to work with SGE? Barry Haddow posted this comment on the list in January this year. We have similar experiences. Tom -------- Original Message -------- Subject: Re: [Moses-support] running moses on a cluster with sge Date: Mon, 31 Jan 2011 09:52:11 +0000 From: Barry Haddow To: [email protected] Cc: Tom Hoar , "Noubours, Sandra" Hi Sandra The short answer is that it really depends how big your models are. Running on a cluster helps speed up tuning because most of the time in tuning is spent decoding, which can be easily parallelised by splitting up the file into chunks. So each of the individual machines should be capable of loading your models and running a decoder. The problem with using a cluster (as opposed to multicore) is that each machine has to have its own ram, and if you want to load large models then you need a lot of ram. Whereas with multicore, each thread can access the same model. Sure, binarising saves a lot on ram usage, but it slows you down and puts a lot of load on the filesystem which can cause problems on clusters. Our group's machines are a mixture of 8 and 16 core Xeon 2.67GHz, with 36-72G ram, no sge. We also have access to the university cluster, but since the most ram you can get is 16G and sge hold jobs don't work at the moment we don't really use it for moses any more, hope that helps - regards - Barry On Tue, 15 Nov 2011 22:02:48 +0000, Guchun Zhang wrote: Sorry, forgot to attach the mert.out file. Guchun
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
