Check for Marathon On 07 Oct 2015, at 09:56, Brian Candler <b.cand...@pobox.com<mailto:b.cand...@pobox.com>> wrote:
Are there any open-source job queue/batch systems which run under Mesos? I am thinking of things like HTCondor, Torque etc. The requirement is to be able to: - define an overall job as a set of sub-tasks (could be many thousands) - put sub-tasks into a queue; execute tasks from the queue - dependencies: don't add a sub-task into the queue until its precursors have completed successfully - restart: after an error, be able to restart the job but skipping those sub-tasks which completed successfully - preferably handle short-lived tasks efficiently (of order of 10 seconds duration) Clearly it's possible to write a framework to do this, but I don't want to re-invent the wheel if it has been done already. Thanks, Brian. P.S. I found Chronos, but it doesn't seem a good match. As far as I can see, it's intended for applications where you pre-define a bunch of tasks (via GUI? via REST?) and then trigger them periodically. Nikolaos Ballas | Software Development Manager Technology Nexus S.a.r.l. 2-4 Rue Eugene Rupert 2453 Luxembourg Delivery address: 2-3 Rue Eugene Rupert,Vertigo Polaris Building Tel: + 3522619113580 cont...@nexusgroup.com<mailto:contact...@nexusgroup.com> | nexusgroup.com<http://www.nexusgroup.com/> LinkedIn.com<http://www.linkedin.com/company/nexus-technology> | Twitter<http://www.twitter.com/technologynexus> | Facebook.com<https://www.facebook.com/pages/Technology-Nexus/133756470003189> [cid:87987ACD-6CF7-41BE-9517-E612DBF86ABA@pwcacc.com] \
