Richard Chycoski <[email protected]> writes:
> Edward Ned Harvey wrote:
>>
>> By default, it will distribute jobs every 15 sec, but it can be configured
>> down to 1sec. So if you have jobs of 2sec, the efficiency might not be
>> great.
>>
>> You can have job dependencies, but I'm not sure how much knowledge it has
>> that's relevant to your situation. If you submit a job, let's say jobid is
>> 500, and you submit another job, let's say 501, you can make job 501 depend
>> on 500. There isn't any conditional detection of "pass/fail" on 500
>> ... just a scheduling delay to ensure 501 runs after 500.
*nod* That is actually a pretty big strike against SGE compared to Condor,
which at least offers "success before the next job" level dependencies without
an additional intelligent scheduler we would have to write.
I think most of our jobs would be pretty independent, but at least a couple of
the current uses do have dependencies internal to them.
[...]
> The comment above about 'seconds' and 'distribute' is important. A typical
> load sharing system is working hard to spread work across all of the
> machines that it 'owns'. Typical uses are software builds. For the second
> comment - builds have few dependencies - these are usually resolved in a
> 'make' or similar process.
Mmmmm. I think our use case is more firmly in the "load sharing" camp, which
is terminology that I had not run into before this. Thank you for the pointer.
[...]
> If you need load sharing, it would seem that systems like SGE and LSF are
> what you are looking for. If you need batch processing, Autosys, BMC, Orsyp,
> Tidal, and Tivoli are the places to look for answers. These are the
> commercial (or near-commercial solutions), I haven't worked with the Open
> Source alternatives in this field - which for batch processing is common
> since most companies want some company 'on the hook' if their batch
> processing system (usually inextricably linked to their 'bread-and-butter')
> fails. For load sharing, I'd certainly look over open systems like SGE.
Well, this work is to a reasonable degree our "bread and butter", but not at
the level where it would be easy to get much funding behind this. So, open
source alternatives it probably is.
Regards,
Daniel
--
✣ Daniel Pittman ✉ [email protected] ☎ +61 401 155 707
♽ made with 100 percent post-consumer electrons
_______________________________________________
Tech mailing list
[email protected]
http://lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
