Thanks, Lane. On 16 November 2011 15:11, Lane Schwartz <[email protected]> wrote:
> I definitely make use of SGE when I run my experiments. I had > forgotten Barry's post. I haven't updated my code in a while, but if > SGE is currently broken, I'll definitely want to help fix it. I won't > be able to look at things until at least December, though, and more > likely January. > > Lane > > > On Wed, Nov 16, 2011 at 10:08 AM, Guchun Zhang <[email protected]> > wrote: > > Hi Tom, > > Thanks for Barry's post back in January. I wasn't a member at that time, > so > > didn't receive such a post. My motives will be mostly because of cost. > With > > SGE, I can reuse some of the old but still usable machines and expand the > > network when necessary. Plus, it is quite unlikely I will be able to get > > hold of machines as powerful as yours. So if I want to have a relatively > > faster Moses system, SGE is a pretty good option for me. > > Hope this makes sense. > > Cheers, > > Guchun > > > > On 16 November 2011 12:27, Tom Hoar < > [email protected]> > > wrote: > >> > >> Hi Guchun, > >> > >> May I ask what motivates you to work with SGE? Barry Haddow posted this > >> comment on the list in January this year. We have similar experiences. > >> > >> Tom > >> > >> -------- Original Message -------- > >> Subject: Re: [Moses-support] running moses on a cluster with sge > >> Date: Mon, 31 Jan 2011 09:52:11 +0000 > >> From: Barry Haddow <[email protected]> > >> To: [email protected] > >> Cc: Tom Hoar <[email protected]>, "Noubours, Sandra" > >> <[email protected]> > >> > >> Hi Sandra > >> > >> The short answer is that it really depends how big your models are. > >> Running on a cluster helps speed up tuning because most of the time in > >> tuning is spent decoding, which can be easily parallelised by splitting > up > >> the file into chunks. So each of the individual machines should be > capable > >> of loading your models and running a decoder. > >> > >> The problem with using a cluster (as opposed to multicore) is that each > >> machine has to have its own ram, and if you want to load large models > then > >> you need a lot of ram. Whereas with multicore, each thread can access > the > >> same model. Sure, binarising saves a lot on ram usage, but it slows you > down > >> and puts a lot of load on the filesystem which can cause problems on > >> clusters. > >> > >> Our group's machines are a mixture of 8 and 16 core Xeon 2.67GHz, with > >> 36-72G ram, no sge. We also have access to the university cluster, but > since > >> the most ram you can get is 16G and sge hold jobs don't work at the > moment > >> we don't really use it for moses any more, > >> > >> hope that helps - regards - Barry > >> > >> > >> > >> On Tue, 15 Nov 2011 22:02:48 +0000, Guchun Zhang <[email protected]> > >> wrote: > >> > >> Sorry, forgot to attach the mert.out file. > >> > >> Guchun > > > > > > -- > > > > Guchun Zhang > > > > Localization Engineer > > Alpha CRC Ltd | Cambridge, UK > > Direct: +44 1223 431035 > > [email protected] > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > > > > > -- *Guchun Zhang* Localization Engineer Alpha CRC Ltd | Cambridge, UK Direct: +44 1223 431035 [email protected] <[email protected]>
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
