Hi Tom, Thanks for Barry's post back in January. I wasn't a member at that time, so didn't receive such a post. My motives will be mostly because of cost. With SGE, I can reuse some of the old but still usable machines and expand the network when necessary. Plus, it is quite unlikely I will be able to get hold of machines as powerful as yours. So if I want to have a relatively faster Moses system, SGE is a pretty good option for me.
Hope this makes sense. Cheers, Guchun On 16 November 2011 12:27, Tom Hoar <[email protected]>wrote: > Hi Guchun, > > May I ask what motivates you to work with SGE? Barry Haddow posted this > comment on the list in January this year. We have similar experiences. > > Tom > > > > -------- Original Message -------- > Subject: Re: [Moses-support] running moses on a cluster with sge > Date: Mon, 31 Jan 2011 09:52:11 +0000 > From: Barry Haddow <[email protected]> > To: [email protected] > Cc: Tom Hoar <[email protected]>, "Noubours, Sandra" < > [email protected]> > > Hi Sandra > > The short answer is that it really depends how big your models are. > Running on a cluster helps speed up tuning because most of the time in > tuning is spent decoding, which can be easily parallelised by splitting up > the file into chunks. So each of the individual machines should be capable > of loading your models and running a decoder. > > The problem with using a cluster (as opposed to multicore) is that each > machine has to have its own ram, and if you want to load large models then > you need a lot of ram. Whereas with multicore, each thread can access the > same model. Sure, binarising saves a lot on ram usage, but it slows you > down and puts a lot of load on the filesystem which can cause problems on > clusters. > > Our group's machines are a mixture of 8 and 16 core Xeon 2.67GHz, with > 36-72G ram, no sge. We also have access to the university cluster, but > since the most ram you can get is 16G and sge hold jobs don't work at the > moment we don't really use it for moses any more, > > hope that helps - regards - Barry > > > > On Tue, 15 Nov 2011 22:02:48 +0000, Guchun Zhang <[email protected]> > wrote: > > Sorry, forgot to attach the mert.out file. > Guchun > > -- *Guchun Zhang* Localization Engineer Alpha CRC Ltd | Cambridge, UK Direct: +44 1223 431035 [email protected] <[email protected]>
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
