[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17020530#comment-17020530 ] Benjamin Mahler commented on MESOS-1807: [~Charle] the command executor is a special case, it's implicitly generated and we oversubscribe a little bit to make room for it: https://github.com/apache/mesos/blob/1.9.0/src/slave/slave.cpp#L6663-L6676 I think the expectation for CUSTOM or the new DEFAULT executors are that they specify their resource requirements. Since it didn't break any backwards compatibility, we enforce it for the new DEFAULT case: https://github.com/apache/mesos/blob/1.9.0/src/master/validation.cpp#L1842-L1859 [~greggomann] is also working on cpu/mem requests vs limits (see MESOS-10001), so that may provide you with the flexibility you desire depending on what you're looking to do. > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Priority: Major > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should the cpu limits be for > that executor when there are no tasks running on it? If no cpu limits are set > then it might starve other executors/tasks on the slave violating isolation > guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. > According to a source code [TODO | > https://github.com/apache/mesos/blob/0226620747e1769434a1a83da547bfc3470a9549/src/master/validation.cpp#L400] > this should also include checking whether requested resources are greater > than MIN_CPUS/MIN_BYTES. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019740#comment-17019740 ] Charles commented on MESOS-1807: BTW, how come this problem doesn't affect the command executor? > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone >Priority: Major > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should the cpu limits be for > that executor when there are no tasks running on it? If no cpu limits are set > then it might starve other executors/tasks on the slave violating isolation > guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. > According to a source code [TODO | > https://github.com/apache/mesos/blob/0226620747e1769434a1a83da547bfc3470a9549/src/master/validation.cpp#L400] > this should also include checking whether requested resources are greater > than MIN_CPUS/MIN_BYTES. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[
https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17019733#comment-17019733
]
Charles commented on MESOS-1807:
Is there any way I could help this move forward?
I just got bitten by this where my custom executor would lead to random errors
described as [~vinodkone] "when the last task on the executor finishes and
Containerizer::update() is called with 0 cpus or 0 mem.". See for example
https://github.com/mesos/chronos/issues/428
{noformat}
ec2-__-___-___-___.compute-1.amazonaws.com E0414 00:41:50.864876 29069
slave.cpp:2344] Failed to update resources for container
867bfec1-ac28-4a4f-8904-3404e6d1e3e9 of executor shell-wrapper-executor running
task ct:1428972109061:0:my-chronos-job on status update for terminal task,
destroying container: Collect failed: No cpus resource given
{noformat}
In the mean time what's the proper workaround? Always define CPU and memory
resources for the executor? It's a bit annoying because it effectively means
arbitrarily limiting the CPU usage of the task (e.g. if there's 1 core and we
allocate 0.01 CPU to the executor, we only have 0.99 left for the task), but I
guess there's no really any way around that. Maybe [~bmahler] has an idea?
> Disallow executors with cpu only or memory only resources
> -
>
> Key: MESOS-1807
> URL: https://issues.apache.org/jira/browse/MESOS-1807
> Project: Mesos
> Issue Type: Improvement
>Reporter: Vinod Kone
>Priority: Major
> Attachments: Screenshot 2015-07-28 14.40.35.png
>
>
> Currently master allows executors to be launched with either only cpus or
> only memory but we shouldn't allow that.
> This is because executor is an actual unix process that is launched by the
> slave. If an executor doesn't specify cpus, what should the cpu limits be for
> that executor when there are no tasks running on it? If no cpu limits are set
> then it might starve other executors/tasks on the slave violating isolation
> guarantees. Same goes with memory. Moreover, the current
> containerizer/isolator code will throw failures when using such an executor,
> e.g., when the last task on the executor finishes and Containerizer::update()
> is called with 0 cpus or 0 mem.
> According to a source code [TODO |
> https://github.com/apache/mesos/blob/0226620747e1769434a1a83da547bfc3470a9549/src/master/validation.cpp#L400]
> this should also include checking whether requested resources are greater
> than MIN_CPUS/MIN_BYTES.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15843110#comment-15843110 ] James Peach commented on MESOS-1807: FWIW, if you try to run a task with 0 disk and actually enforce that, the task will fail. See MESOS-5393. > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should the cpu limits be for > that executor when there are no tasks running on it? If no cpu limits are set > then it might starve other executors/tasks on the slave violating isolation > guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. > According to a source code [TODO | > https://github.com/apache/mesos/blob/0226620747e1769434a1a83da547bfc3470a9549/src/master/validation.cpp#L400] > this should also include checking whether requested resources are greater > than MIN_CPUS/MIN_BYTES. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15219249#comment-15219249 ] chenqiang commented on MESOS-1807: -- there's any progress? > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should the cpu limits be for > that executor when there are no tasks running on it? If no cpu limits are set > then it might starve other executors/tasks on the slave violating isolation > guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. > According to a source code [TODO | > https://github.com/apache/mesos/blob/0226620747e1769434a1a83da547bfc3470a9549/src/master/validation.cpp#L400] > this should also include checking whether requested resources are greater > than MIN_CPUS/MIN_BYTES. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042590#comment-15042590 ] Avinash Sridharan commented on MESOS-1807: -- Hi Vinod, Is this ticket still a newbie ticket or we are waiting for MESOS-1187 and MESOS-1718 to be fixed before we look at this. Was thinking of taking up a newbie issue to start off. So wanted to check if we can make progress on this? > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Labels: newbie > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should do the cpu limits be > for that executor when there are no tasks running on it? If no cpu limits are > set then it might starve other executors/tasks on the slave violating > isolation guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982172#comment-14982172 ] Klaus Ma commented on MESOS-1807: - [~vi...@twitter.com]/[~bmahler], I'd like to contribute to this ticket as it seems related with MESOS-2845. But there're some question to this ticket: - how do we define "0"? there're several tickets about checking scalar against "0", e.g. MESOS-1187. I'd suggest to define the min cpu/mem, and check executor's resources against mins. - for the executor, it seems we're going to cover two cases: 1. executor & task runner is different: so we need assign resources to both; if it's true, maybe we need to assign resources to command executor in MESOS-2845; we need to evaluate the resource consumption of command line executor. 2. executor & task runner is the same one; we plus the resources of them > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Labels: newbie > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should do the cpu limits be > for that executor when there are no tasks running on it? If no cpu limits are > set then it might starve other executors/tasks on the slave violating > isolation guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982827#comment-14982827 ] Vinod Kone commented on MESOS-1807: --- I think we need to first fix MESOS-1187 and MESOS-1718 before we tackle this ticket. I'll take a look at the latest discussion and add comments on the respective tickets. > Disallow executors with cpu only or memory only resources > - > > Key: MESOS-1807 > URL: https://issues.apache.org/jira/browse/MESOS-1807 > Project: Mesos > Issue Type: Improvement >Reporter: Vinod Kone > Labels: newbie > Attachments: Screenshot 2015-07-28 14.40.35.png > > > Currently master allows executors to be launched with either only cpus or > only memory but we shouldn't allow that. > This is because executor is an actual unix process that is launched by the > slave. If an executor doesn't specify cpus, what should do the cpu limits be > for that executor when there are no tasks running on it? If no cpu limits are > set then it might starve other executors/tasks on the slave violating > isolation guarantees. Same goes with memory. Moreover, the current > containerizer/isolator code will throw failures when using such an executor, > e.g., when the last task on the executor finishes and Containerizer::update() > is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644867#comment-14644867 ] Erik Weathers commented on MESOS-1807: -- [~vi...@twitter.com]: without the web UI fix then to me its unacceptable to *use* fractional CPUs at all. So for storm-on-mesos I'm using 0 as the executor CPUs (because the mesos-supervisor does almost nothing, 0 is not entirely unreasonable). When the behavior change requested in this ticket is enacted (changing from warning to error) then I will either be stuck on the previous version of mesos, or I will be forced to have an ugly UI. Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Attachments: Screenshot 2015-07-28 14.40.35.png Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644829#comment-14644829 ] Erik Weathers commented on MESOS-1807: -- In mesos-0.20.1 the mesos UI got completely broken if you had fractional CPUs assigned to executors/tasks: * https://issues.apache.org/jira/secure/attachment/12747600/Screenshot%202015-07-28%2014.40.35.png) I would consider that bug a must fix before disallowing 0 CPUs for executors. Maybe it's already fixed? Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Attachments: Screenshot 2015-07-28 14.40.35.png Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14644845#comment-14644845 ] Vinod Kone commented on MESOS-1807: --- Sorry I don't follow. What is the relation between this ticket and webui fix? Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Attachments: Screenshot 2015-07-28 14.40.35.png Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14599058#comment-14599058 ] Adam B commented on MESOS-1807: --- Thanks for the explanation, Vinod. I'm targeting this out of 0.23 since nobody is actively working on it. Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596851#comment-14596851 ] Vinod Kone commented on MESOS-1807: --- There seems to be some confusion, so let me clairfy. Mesos currently allows executors that use either 0 cpus or 0 memory. Since 0.21.0, it emits a warning, but continues to allow them. Note that this ticket is not resolved yet. The goal of this ticket is to disallow executors with 0 cpus or 0 memory. Any issues that marathon or chronos are seeing is the long standing behavior of Mesos and precisely why this ticket was created. This deprecation was announced in the CHANGELOG for 0.21.0. Not sure if there was a specific email sent to the dev list though. Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14596807#comment-14596807 ] Elizabeth Lingg commented on MESOS-1807: For me, the issue is that Chronos and Marathon, for example, currently launch custom executors with 0 cpu AND 0 memory. While this needs to be fixed in Chronos and Marathon, a full announcement, warning, deprecation cycle would be appropriate in my view. I do agree that custom executors need to specify both CPU and Memory. Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14594522#comment-14594522 ] Adam B commented on MESOS-1807: --- I'm confused.. are Spark/Marathon/Chronos failing already due to executors with 0 cpu AND 0 memory, and this ticket wants to also fail for 0 cpu OR 0 memory? Do we need to through an announcement/warning/deprecation cycle first? Or is this issue to remove/relax the previous constraint so that frameworks will fail only with executors with 0 cpu XOR 0 memory? Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14496876#comment-14496876 ] Elizabeth Lingg commented on MESOS-1807: +1, [~tstclair] This has not only caused Spark to fail, but also it has caused Marathon and Chronos to as well. See https://github.com/mesos/chronos/issues/428. Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314609#comment-14314609 ] Niklas Quarfot Nielsen commented on MESOS-1807: --- Hey [~vinodkone] - Do we have anyone working on this? Or else, should we bump to 0.23.0? Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Assignee: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196173#comment-14196173 ] Timothy St. Clair commented on MESOS-1807: -- Was the original goal of cpu+memory only offers to enable resizing? The behavior change had caused other frameworks to fail, namely Spark. Ideally I would love to have some integration testing on mods like this. [~tnachen] ^ Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Assignee: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MESOS-1807) Disallow executors with cpu only or memory only resources
[ https://issues.apache.org/jira/browse/MESOS-1807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14196174#comment-14196174 ] Timothy St. Clair commented on MESOS-1807: -- Was the original goal of cpu+memory only offers to enable resizing? The behavior change had caused other frameworks to fail, namely Spark. Ideally I would love to have some integration testing on mods like this. [~tnachen] ^ Disallow executors with cpu only or memory only resources - Key: MESOS-1807 URL: https://issues.apache.org/jira/browse/MESOS-1807 Project: Mesos Issue Type: Improvement Reporter: Vinod Kone Assignee: Vinod Kone Labels: newbie Currently master allows executors to be launched with either only cpus or only memory but we shouldn't allow that. This is because executor is an actual unix process that is launched by the slave. If an executor doesn't specify cpus, what should do the cpu limits be for that executor when there are no tasks running on it? If no cpu limits are set then it might starve other executors/tasks on the slave violating isolation guarantees. Same goes with memory. Moreover, the current containerizer/isolator code will throw failures when using such an executor, e.g., when the last task on the executor finishes and Containerizer::update() is called with 0 cpus or 0 mem. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
