I don't think that we have a suggested way. If I would have the requirement, I would look into Oozie. I think its quite easy to add additional services (=Flink) into Oozie. In addition, it seems to have a REST interface and some other stuff.
If you want, you could also implement one yourself and contribute it back to Flink. On Thu, Sep 18, 2014 at 10:11 AM, Flavio Pompermaier <[email protected]> wrote: > Yes I was referring exactly to that, I was also involved in the Dopa > project :) > So, at the moment what is the suggested way to schedule jobs with Flink? > > > On Thu, Sep 18, 2014 at 9:48 AM, Robert Metzger <[email protected]> > wrote: > >> Are you referring to this project? >> https://github.com/TU-Berlin/dopa-scheduler >> Its not an official repository of the Flink (Stratosphere) project. I >> think a PhD student at TU Berlin created the code there. >> >> >> >> On Thu, Sep 11, 2014 at 4:29 PM, Flavio Pompermaier <[email protected] >> > wrote: >> >>> Of course with Flink I could in principle execute almost everything with >>> a single Job but, in general, I could write 2 different jobs and decide >>> from time to time when the second should be run. >>> That's why also metheor scripts are very useful :) >>> From what I know there was a scheduler in Stratosphere that was using >>> RabbitMQ, right? >>> >>> I would like to avoid to run linux commands and instead use some REST >>> interface to trigger or schedule jobs. >>> >>> Best, >>> Flavio >>> >>> >>> On Thu, Sep 11, 2014 at 4:07 PM, Fabian Hueske <[email protected]> >>> wrote: >>> >>>> Hi Flavio, >>>> >>>> what exactly do you mean by scheduling? >>>> Do you want to run a job in regular intervals or execute a complex >>>> workflow? >>>> >>>> Oozie is primarily used to orchestrate the execution of MapReduce >>>> workflows. Since, MR is a rather inflexible programming model, complex >>>> tasks need to split up into multiple dependent jobs that are executed once >>>> their predecessors have finished. Oozie orchestrates this execution. >>>> In Flink, you can build a complex analysis flow as a single program and >>>> execute it. Hence, there is no need for a workflow scheduler such as Oozie. >>>> >>>> If you want to run a job in regular intervals, you can configure a cron >>>> job, that starts executes the CLI client or implement a Java or Scala >>>> program that submits jobs a certain points in time. >>>> >>>> Best, Fabian >>>> >>>> 2014-09-11 15:36 GMT+02:00 Flavio Pompermaier <[email protected]>: >>>> >>>>> Hi to all, >>>>> >>>>> I'd like to know if there's an example of how to schedule a Job in >>>>> Flink. >>>>> Do we still need something like Oozie or Quartz or we can avoid them? >>>>> >>>>> Best, >>>>> Flavio >>>>> >>>> >>> >
