We need to implement hybrid workflows, including batch processing (Spark). Many of the jobs run unique docker images with very different dependencies and resources so we can’t use the Process level ordering instead of Job ordering.
I’ve seen the resolution of https://issues.apache.org/jira/browse/AURORA-735 <https://issues.apache.org/jira/browse/AURORA-735> is Later :) > On 02 Feb 2016, at 21:44, Bill Farner <[email protected]> wrote: > > In general, i've assumed that job dependencies create more problems than they > solve (e.g. scheduling behavior when a parent job is removed, parent/child > relationships that span auth groups, etc). Dependencies seem handy for > setting up and tearing down groups of jobs for things like development > environments, but that should be easily replaceable by a small script. Is > this contrary to your experience? Through API calls? > > On Tue, Feb 2, 2016 at 12:34 PM, Krisztian Szucs <[email protected] > <mailto:[email protected]>> wrote: > Hi Everyone! > > We’d like to migrate our jobs from Chronos to Aurora. > AFAIK Aurora doesn’t support dependant jobs. > Could You recommend any tools or a workaround to specify e.g. parent jobs? > > - Krisztian > >
