Hi Lane As far as I know, SGE is not broken in moses. It's just that hold jobs don't work on our cluster, so moses SGE doesn't work there,
cheers - Barry On Wednesday 16 November 2011 15:11:29 Lane Schwartz wrote: > I definitely make use of SGE when I run my experiments. I had > forgotten Barry's post. I haven't updated my code in a while, but if > SGE is currently broken, I'll definitely want to help fix it. I won't > be able to look at things until at least December, though, and more > likely January. > > Lane > > On Wed, Nov 16, 2011 at 10:08 AM, Guchun Zhang <[email protected]> wrote: > > Hi Tom, > > Thanks for Barry's post back in January. I wasn't a member at that time, > > so didn't receive such a post. My motives will be mostly because of cost. > > With SGE, I can reuse some of the old but still usable machines and > > expand the network when necessary. Plus, it is quite unlikely I will be > > able to get hold of machines as powerful as yours. So if I want to have a > > relatively faster Moses system, SGE is a pretty good option for me. > > Hope this makes sense. > > Cheers, > > Guchun > > > > On 16 November 2011 12:27, Tom Hoar > > <[email protected]> > > > > wrote: > >> Hi Guchun, > >> > >> May I ask what motivates you to work with SGE? Barry Haddow posted this > >> comment on the list in January this year. We have similar experiences. > >> > >> Tom > >> > >> -------- Original Message -------- > >> Subject: Re: [Moses-support] running moses on a cluster with sge > >> Date: Mon, 31 Jan 2011 09:52:11 +0000 > >> From: Barry Haddow <[email protected]> > >> To: [email protected] > >> Cc: Tom Hoar <[email protected]>, "Noubours, Sandra" > >> <[email protected]> > >> > >> Hi Sandra > >> > >> The short answer is that it really depends how big your models are. > >> Running on a cluster helps speed up tuning because most of the time in > >> tuning is spent decoding, which can be easily parallelised by splitting > >> up the file into chunks. So each of the individual machines should be > >> capable of loading your models and running a decoder. > >> > >> The problem with using a cluster (as opposed to multicore) is that each > >> machine has to have its own ram, and if you want to load large models > >> then you need a lot of ram. Whereas with multicore, each thread can > >> access the same model. Sure, binarising saves a lot on ram usage, but it > >> slows you down and puts a lot of load on the filesystem which can cause > >> problems on clusters. > >> > >> Our group's machines are a mixture of 8 and 16 core Xeon 2.67GHz, with > >> 36-72G ram, no sge. We also have access to the university cluster, but > >> since the most ram you can get is 16G and sge hold jobs don't work at > >> the moment we don't really use it for moses any more, > >> > >> hope that helps - regards - Barry > >> > >> > >> > >> On Tue, 15 Nov 2011 22:02:48 +0000, Guchun Zhang <[email protected]> > >> wrote: > >> > >> Sorry, forgot to attach the mert.out file. > >> > >> Guchun > > > > -- > > > > Guchun Zhang > > > > Localization Engineer > > Alpha CRC Ltd | Cambridge, UK > > Direct: +44 1223 431035 > > [email protected] > > > > _______________________________________________ > > Moses-support mailing list > > [email protected] > > http://mailman.mit.edu/mailman/listinfo/moses-support > > _______________________________________________ > Moses-support mailing list > [email protected] > http://mailman.mit.edu/mailman/listinfo/moses-support > -- Barry Haddow University of Edinburgh +44 (0) 131 651 3173 -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. _______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
