It varies, some inhouse apps, some commerical software. Although we're wrapping everything in a docker container for consistency.
There are usually multiple steps for each task, and these steps can often be subdivided into multiple parallel processes. Different steps require different executables though. Aaron ________________________________ From: Tim Chen [[email protected]] Sent: 13 May 2015 19:01 To: [email protected] Subject: Re: Batch Scheduler with dependency support How are you running your batch jobs? Is the batch job script/executable an in-house app? Tim On Wed, May 13, 2015 at 9:46 AM, Andras Kerekes <[email protected]<mailto:[email protected]>> wrote: You might want to have a look at stolos too: https://github.com/sailthru/stolos Andras From: Aaron Carey [mailto:[email protected]<mailto:[email protected]>] Sent: Wednesday, May 13, 2015 11:54 AM To: [email protected]<mailto:[email protected]> Subject: RE: Batch Scheduler with dependency support Thanks! I hadn't come across that one before :) ________________________________ From: [email protected]<mailto:[email protected]> [[email protected]<mailto:[email protected]>] on behalf of Jeff Schroeder [[email protected]<mailto:[email protected]>] Sent: 13 May 2015 16:39 To: [email protected]<mailto:[email protected]> Subject: Re: Batch Scheduler with dependency support Lookup Hubspot's Singularity On Wednesday, May 13, 2015, Aaron Carey <[email protected]<mailto:[email protected]>> wrote: Thanks Jeff, Any other options around as well? ________________________________ From: [email protected]<http://UrlBlockedError.aspx> [[email protected]<http://UrlBlockedError.aspx>] on behalf of Jeff Schroeder [[email protected]<http://UrlBlockedError.aspx>] Sent: 13 May 2015 14:12 To: [email protected]<http://UrlBlockedError.aspx> Subject: Batch Scheduler with dependency support It does both just as well, along with cron-like functionality. It is harder to install and takes a bit more understanding however. The official tutorial is a process that loops 100 times and then exits. http://aurora.apache.org/documentation/latest/tutorial/#the-script Aurora is pretty much a superset of most other generic frameworks sans maybe hubspot's singularity. On Wednesday, May 13, 2015, Aaron Carey <[email protected]<http://UrlBlockedError.aspx>> wrote: I was under the impression Aurora was for long running services? Is it suitable for scheduling one of batch processes too? thanks, Aaron ________________________________ From: [email protected]<mailto:[email protected]> [[email protected]<mailto:[email protected]>] on behalf of Jeff Schroeder [[email protected]<mailto:[email protected]>] Sent: 13 May 2015 13:12 To: [email protected]<mailto:[email protected]> Subject: Re: Batch Scheduler with dependency support Apache Aurora does this and you can be explicit about the ordering On Wednesday, May 13, 2015, Aaron Carey <[email protected]<mailto:[email protected]>> wrote: Hi All, I was just wondering if anyone out there knew of a good mesos batch scheduler which supports dependencies between tasks? (ie Task B cannot run until Task A is complete) Thanks, Aaron -- Text by Jeff, typos by iPhone -- Text by Jeff, typos by iPhone -- Text by Jeff, typos by iPhone
