[
https://issues.apache.org/jira/browse/MESOS-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15145515#comment-15145515
]
Anindya Sinha edited comment on MESOS-4666 at 2/16/16 3:57 AM:
---------------------------------------------------------------
btw, in total resources, the proposal is to send what the mesos-slave started
off with (either {{--resources}} flag or derived from the system).
Say a slave starts with {noformat} cpus(*): 8;mem(*):16384;disk(*): 163840
{noformat} we send this info in {{Offer::total_resources}} to denote that this
is the max capability for this slave in addition to available resources for
this specific framework.
Another example is if a slave starts with {noformat} cpus(*): 6;cpus(role1):
2;mem(*):8192;mem(role1): 8192;disk(*): 102400;disk(role1): 61440 {noformat},
we send the Offer::total_resources as a summation of all resource types across
roles, ie. {noformat} cpus(*): 8;mem(*):16384;disk(*): 163840 {noformat}.
So we want to send the total resources the slave started off with when it
registered. I do not think it exposes what other frameworks have
launched/reserved. So I am not sure if this would still qualify as a security
issue.
My use case is that we have 2 different frameworks in the same role to launch
these related tasks on the same slave. Also, we do not want any other tasks to
be running on this slave. We do not use static reservations but reserve
resources dynamically.
So the idea is that when the 1st framework comes up, we reserve all the
resources on the slave if the {{Offer::total_resources}} ==
{{Offer::resources}} (which suggests that the slave is not running any tasks
across any frameworks). Once the reservation is successful, we launch task1
from framework1. Then framework2 comes up and uses the reserved resources to
launch task2.
was (Author: anindya.sinha):
btw, in total resources, the proposal is to send what the mesos-slave started
off with (either --resources flag or derived from the system).
Say a slave starts with {noformat} cpus(*): 8;mem(*):16384;disk(*): 163840
{noformat}, we send this info in Offer::total_resources to denote that this is
the max capability for this slave in addition to available resources for this
specific framework.
Another example is if a slave starts with {noformat} cpus(*): 6;cpus(role1):
2;mem(*):8192;mem(role1): 8192;disk(*): 102400;disk(role1): 61440 {noformat},
we send the Offer::total_resources as a summation of all resource types across
roles, ie. {noformat} cpus(*): 8;mem(*):16384;disk(*): 163840 {noformat}.
So we want to send the total resources the slave started off with when it
registered. I do not think it exposes what other frameworks have
launched/reserved. So I am not sure if this would still qualify as a security
issue.
My use case is that we have 2 different frameworks in the same role to launch
these related tasks on the same slave. Also, we do not want any other tasks to
be running on this slave. We do not use static reservations but reserve
resources dynamically.
So the idea is that when the 1st framework comes up, we reserve all the
resources on the slave if the Offer::total_resources == Offer::resources (which
suggests that the slave is not running any tasks across any frameworks). Once
the reservation is successful, we launch task1 from framework1. Then framework2
comes up and uses the reserved resources to launch task2.
> Expose total resources of a slave in offer for scheduling decisions
> -------------------------------------------------------------------
>
> Key: MESOS-4666
> URL: https://issues.apache.org/jira/browse/MESOS-4666
> Project: Mesos
> Issue Type: Improvement
> Components: general
> Affects Versions: 0.25.0
> Reporter: Anindya Sinha
> Assignee: Anindya Sinha
> Priority: Minor
>
> To effectively schedule certain class of tasks, the scheduler might need to
> know not only the available resources (as exposed currently) but also the
> maximum resources available on that slave. This is specifically true for
> clusters having different configurations of the slave nodes in terms of
> resources such as cpu, memory, disk, etc.
> Certain class of tasks might have a need to be scheduled on the same slave
> (esp needing shared persistent volumes, MESOS-3421). Instead of dedicating a
> slave to a framework, the framework can make a very good determination if it
> had exposure to both available as well as total resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
