I definitely make use of SGE when I run my experiments. I had forgotten Barry's post. I haven't updated my code in a while, but if SGE is currently broken, I'll definitely want to help fix it. I won't be able to look at things until at least December, though, and more likely January.
Lane On Wed, Nov 16, 2011 at 10:08 AM, Guchun Zhang <[email protected]> wrote: > Hi Tom, > Thanks for Barry's post back in January. I wasn't a member at that time, so > didn't receive such a post. My motives will be mostly because of cost. With > SGE, I can reuse some of the old but still usable machines and expand the > network when necessary. Plus, it is quite unlikely I will be able to get > hold of machines as powerful as yours. So if I want to have a relatively > faster Moses system, SGE is a pretty good option for me. > Hope this makes sense. > Cheers, > Guchun > > On 16 November 2011 12:27, Tom Hoar <[email protected]> > wrote: >> >> Hi Guchun, >> >> May I ask what motivates you to work with SGE? Barry Haddow posted this >> comment on the list in January this year. We have similar experiences. >> >> Tom >> >> -------- Original Message -------- >> Subject: Re: [Moses-support] running moses on a cluster with sge >> Date: Mon, 31 Jan 2011 09:52:11 +0000 >> From: Barry Haddow <[email protected]> >> To: [email protected] >> Cc: Tom Hoar <[email protected]>, "Noubours, Sandra" >> <[email protected]> >> >> Hi Sandra >> >> The short answer is that it really depends how big your models are. >> Running on a cluster helps speed up tuning because most of the time in >> tuning is spent decoding, which can be easily parallelised by splitting up >> the file into chunks. So each of the individual machines should be capable >> of loading your models and running a decoder. >> >> The problem with using a cluster (as opposed to multicore) is that each >> machine has to have its own ram, and if you want to load large models then >> you need a lot of ram. Whereas with multicore, each thread can access the >> same model. Sure, binarising saves a lot on ram usage, but it slows you down >> and puts a lot of load on the filesystem which can cause problems on >> clusters. >> >> Our group's machines are a mixture of 8 and 16 core Xeon 2.67GHz, with >> 36-72G ram, no sge. We also have access to the university cluster, but since >> the most ram you can get is 16G and sge hold jobs don't work at the moment >> we don't really use it for moses any more, >> >> hope that helps - regards - Barry >> >> >> >> On Tue, 15 Nov 2011 22:02:48 +0000, Guchun Zhang <[email protected]> >> wrote: >> >> Sorry, forgot to attach the mert.out file. >> >> Guchun > > > -- > > Guchun Zhang > > Localization Engineer > Alpha CRC Ltd | Cambridge, UK > Direct: +44 1223 431035 > [email protected] > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
