Re: Support deadline for tasks

Hi, Benjamin, Usually for us if tasks run longer than a certain period of time it means that something has gone wrong and we should just abort/try again. David (also at Yelp) On Fri, Mar 23, 2018 at 7:14 PM, Benjamin Mahler wrote: > Ah, I was more curious about why they

Re: Support deadline for tasks

Ah, I was more curious about why they need to be killed after a timeout. E.g. After a particular deadline the work is useless (in Zhitao's case). On Fri, Mar 23, 2018 at 6:22 PM Sagar Sadashiv Patwardhan wrote: > Hi Benjamin, > We have a few tasks that

Re: Support deadline for tasks

Sagar, could you share your use case? Or is it exactly the same as Zhitao's? On Fri, Mar 23, 2018 at 3:15 PM, Sagar Sadashiv Patwardhan wrote: > +1 > > This will be useful for us(Yelp) as well. > > On Fri, Mar 23, 2018 at 1:31 PM, Benjamin Mahler > wrote: >

Re: Support deadline for tasks

Also, it's advantageous for mesos to be aware of a hard deadline when it comes to resource allocation. We know that some resources will free up and can make better decisions when it comes to pre-emption, for example. Currently, mesos doesn't know if a task will run forever or will run to

Re: Support deadline for tasks

> On Mar 23, 2018, at 9:57 AM, Renan DelValle wrote: > > Hi Zhitao, > > Since this is something that could potentially be handled by the executor > and/or framework, I was wondering if you could speak to the advantages of > making this a TaskInfo primitive vs

Re: Support deadline for tasks

Hi Zhitao, Since this is something that could potentially be handled by the executor and/or framework, I was wondering if you could speak to the advantages of making this a TaskInfo primitive vs having the executor (or even the framework) handle it. -Renan On Fri, Mar 23, 2018 at 9:19 AM,

Re: Support deadline for tasks

Thanks James. I'll update the JIRA with our names and start with some prototype. On Thu, Mar 22, 2018 at 9:07 PM, James Peach wrote: > > > > On Mar 22, 2018, at 10:06 AM, Zhitao Li wrote: > > > > In our environment, we run a lot of batch jobs, some of

Re: Support deadline for tasks

> On Mar 22, 2018, at 10:06 AM, Zhitao Li wrote: > > In our environment, we run a lot of batch jobs, some of which have tight > timeline. If any tasks in the job runs longer than x hours, it does not make > sense to run it anymore. > > For instance, a team would

Support deadline for tasks

In our environment, we run a lot of batch jobs, some of which have tight timeline. If any tasks in the job runs longer than x hours, it does not make sense to run it anymore. For instance, a team would submit a job which builds a weekly index and repeats every Monday. If the job does not finish