G'day.
I am interested in soliciting experiences deploying, using and maintaining the
Condor batch processing system, especially under Linux / Debian.
Our use would predominantly be many small jobs, rather than a few large jobs,
with runtimes measured in a few hours. Probably only a handful of nodes, on
the order of half a dozen, in total.[1]
My key concerns are:
1. How stable is Condor on Linux, and especially Debian?
2. Is it reasonably easy to manage over time, especially when software
upgrades are required to Condor, or to the underlying platform?
3. We presently have Etch on most of our systems, and are looking to migrate
to Lenny in the future; is mixing underlying distributions going to give
Condor heartburn?
4. How efficient is it when used for jobs with processing requirements ranging
from a few seconds[2] through to a few hours? Are we going to need to put
in substantial work to avoid running tiny jobs through Condor, and only
push the big ones out?
5. How do you find Condor for dependent jobs, both from the point of view of
one job spawning multiple subsequent (or sub) jobs, and from the point of
view of "dependency based" solutions: deliver result X, where X requires Y
first, and Y requires A and B, etc.
I am also interested in experiences with torque, on the same sort of criteria,
and any suggestions y'all might have for other tools that merit consideration.
Oh, the jobs are very mixed: from "mail merge" type applications, through to
sequential and branching data processing for reporting, through to data
validation.
We do have a mixture of tools, languages and data I/O requirements for the
applications, so something that requires rebuilding these into something other
than Perl is unlikely to happen, and being tied to Perl is a negative.
Regards,
Daniel
--
✣ Daniel Pittman ✉ [email protected] ☎ +61 401 155 707
♽ made with 100 percent post-consumer electrons
_______________________________________________
Tech mailing list
[email protected]
http://lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
