Re: Idempotent Job Submission

2017-04-28 Thread Till Rohrmann
(probably next week) I will spend some time refining these bigger steps and creating JIRA issues for them. Cheers, Till ​ On Thu, Apr 20, 2017 at 6:20 PM, James Bucher <jbuc...@expedia.com> wrote: > Hey all, > > I have been doing some digging to see if there is a good way to do an &

Idempotent Job Submission

2017-04-20 Thread James Bucher
Hey all, I have been doing some digging to see if there is a good way to do an idempotent job submission. I was hoping to write a job submission agent that does the following: 1. Checks to see if the cluster is running yet (can contact a JobManager) 2. Checks to see if the job