Re: Spark Job trigger in production

I assume that you would like to trigger Spark batch jobs, and not streaming jobs. For production jobs, I recommend avoiding scheduling batch jobs directly with cron or cron services like Chronos. Sometimes, jobs will fail, either due to missing input data, or due to execution problems. When it hap

Re: Spark Job trigger in production

If you are using Mesos, then u can use Chronos or Marathon On Wed, Jul 20, 2016 at 6:08 AM Rabin Banerjee wrote: > ++ crontab :) > > On Wed, Jul 20, 2016 at 9:07 AM, Andrew Ehrlich > wrote: > >> Another option is Oozie with the spark action: >> https://oozie.apache.org/docs/4.2.0/DG_SparkActionE

Re: Spark Job trigger in production

++ crontab :) On Wed, Jul 20, 2016 at 9:07 AM, Andrew Ehrlich wrote: > Another option is Oozie with the spark action: > https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html > > On Jul 18, 2016, at 12:15 AM, Jagat Singh wrote: > > You can use following options > > * spark-submit from

Re: Spark Job trigger in production

Another option is Oozie with the spark action: https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html > On Jul 18, 2016, at 12:15 AM, Jagat Singh wrote: > > You can use following options > > * spark-submit from shel

Re: Spark Job trigger in production

You can use following options * spark-submit from shell * some kind of job server. See spark-jobserver for details * some notebook environment See Zeppelin for example On 18 July 2016 at 17:13, manish jaiswal wrote: > Hi, > > > What is the best approach to trigger spark job in production cl