Hi, My question is conceptual about what is scheduler in a cluster-wide context and what counts as scheduler responsibility. The actual implementation is not that important but rather where you put that dependency logic, who decides if dependencies are satisfied. It is not about PODs because dependent services doesn’t have to run in the same pod.
It seems to me that Mesos scheduler is defined like that it decides if and where a given task should be executed based on offered resources. In many application offered resources are possibly not enough to decide if to run a task. E.g. state of dependent services, licenses etc. I’m asking if there is any discussion, guidelines, best practices where these decisions belong when designing a framework. Can it still be a scheduler responsibility? Should it be in some other module, e.g. policy engine. I just try to understand where scheduler responsibilities start and end. If I would define scheduler as an entity deciding if and where (and how?) to execute a task based on state of resources and once decided the actual execution is delegated to an executor and I would like to consider all that logic above as a scheduler responsibility, then I have to either extend my definition of a scheduler or I have to be able to map everything as a resource. After all set of running services could be considered as a resource. REST is as well trying to represent everything as a resource. I’m probably trying to think too much, Petr From: haosdent [mailto:[email protected]] Sent: 14. prosince 2016 5:14 To: user Subject: Re: Can I consider other framework tasks as a resource? Does it make sense? Hi, @Petr. > Like if I want to run my task collocated with some other tasks on the same > node I have to make this decision somewhere. Do you mean "POD" here? For my cases, if there are some dependencies between my tasks, I use database, message queue or zookeeper to implement my requirement. On Wed, Dec 14, 2016 at 3:09 AM, Petr Novak <[email protected]> wrote: Hello, I want to execute tasks which requires some other tasks from other framework(s) already running. I’m thinking where such logic/strategy/policy belongs in principle. I understand scheduling as a process to decide where to execute task according to some resources availability, typically CPU, mem, net, hdd etc. If my task require other tasks running could I generalize and consider that those tasks from other frameworks are kind of required resources and put this logic/strategy decisions into scheduler? Like if I want to run my task collocated with some other tasks on the same node I have to make this decision somewhere. Does it make any sense? I’m asking because I have never thought about other frameworks/tasks as “resources” so that I could put them into scheduler to satisfy my understanding of a scheduler. Or it rather belongs higher like to a framework, or lower to an executor? Should scheduler be dedicated to decisions about resources which are offered and am I mixing concepts? Or I just should keep distinction between resources and requirements/policies or whatever but anyway does this kind of logic still belongs to scheduler or it should be somewhere else? I’m trying to understand which logic should be in scheduler and what should go somewhere else. Many thanks, Petr -- Best Regards, Haosdent Huang
