You mentioned DrQueue, that batch processing app fulfills many of your requirements. I wrote an ebuild for it too which is on bugs.gentoo
Hanni On 11/01/2008, Jos Houtman <[EMAIL PROTECTED]> wrote: > > Thanx for the replies, > > I will certainly try the beowulf mailinglist next week, > The suggestions are great gearman seems to fit except for being perl > oriented (atleast that's what I read sofar). > Torque/pbs, don't know about that yet. > > Thanx again, > > Jos Houtman > System administrator Hyves.nl > email: [EMAIL PROTECTED] > > -----Original Message----- > From: Robin H. Johnson [mailto:[EMAIL PROTECTED] > Sent: donderdag 10 januari 2008 15:30 > To: gentoo-cluster@lists.gentoo.org > Subject: Re: [gentoo-cluster] cluster or distributed queue, general > question > > On Thu, Jan 10, 2008 at 02:59:27PM +0100, Jos Houtman wrote: > > For my master thesis I took up a project that requires mapping of a > number of statically defined parallel jobs into a more dynamic > environment that allows better scaling. > > The situation as described below let me to believe a cluster or > distributed queue (DrQueue?) solution is necessary. For the situation > see [situation] at the end of this email. > Off the top of my head, many of your requirements are available in two > totally different apps: > - Gearman, written by Brad Fitzpatrick @ LiveJournal. Perl mainly, I > think there are other interfaces as well to it. > - Torque/PBS - somewhat less of a fit, I'm not certain about running > perpetual jobs. > > You may also need some degree of STONITH for the job running only once > during node failure case. (Say the job manager crashes, the job is still > running, but you have no control of it. You need to zap it hard). > > -- > Robin Hugh Johnson > Gentoo Linux Developer & Infra Guy > E-Mail : [EMAIL PROTECTED] > GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 > -- > gentoo-cluster@lists.gentoo.org mailing list > > -- E-mail: [EMAIL PROTECTED] Mobile: +44 (0) 7985580147 My Blog: http://ainkaboot.co.uk/blogs/hanni/ Website: http://ainkaboot.co.uk http://drqueue.org
