Re: Number of tasks per executor and resource limits
Perfect, thanks! On Fri, 21 Feb 2020, 00:55 Qian Zhang, wrote: > if we wanted to support multiple tasks per executor, could we leverage the >> mesos containeriser to easily put each task in its own cgroup? > > > If what your custom executor does is similar with default executor (i.e., > launch each task in a task group as a nested container by calling Mesos > agent API `LAUNCH_CONTAINER`), then the answer is yes since we are going to > support nested cgroups for nested containers (i.e., each nested container > can have its own cgroups) in Mesos containerizer soon. > > > Regards, > Qian Zhang > > > On Fri, Feb 21, 2020 at 4:46 AM Vinod Kone wrote: > >> Andrei, Qian: Can one of you answer the above question? >> >> On Thu, Feb 20, 2020 at 7:15 PM Charles-François Natali < >> cf.nat...@gmail.com> wrote: >> >>> Thanks for the quick reply! >>> >>> I think we're going to go for one executor per task for now, that's much >>> simpler. >>> >>> Otherwise I was wondering - if we wanted to support multiple tasks per >>> executor, could we leverage the mesos containeriser to easily put each task >>> in its own cgroup? >>> >>> >>> >>> Is it just a matter of setting 'execu >>> On Thu, 20 Feb 2020, 17:40 Vinod Kone, wrote: >>> Hi Charles, We are actually working on a new feature that puts each of the tasks (of the default executor) in its own cgroup so that they can be individually limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For custom executor, you would be on your own to implement the same. Also, your custom executor / scheduler should be able to limit one task per executor if you so desire. On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali < cf.nat...@gmail.com> wrote: > Hi, > > Is there a way to force Mesos to use one executor per task? > The reason I would want to do that is for resources limits: for > example, when using cgroup to limit CPU and memory, IIUC the containeriser > sets limits corresponding to the sum of the resources allocated to the > tasks managed by the underlying executor. > > Which means that for example if a task is allocated 1GB and another > 2GB, if they are started by the same executor, the collarbone > containeriser > will limit the total memory for the two tasks (plus the executor) to 3GB. > But, unless I'm missing something, nothing prevents a process (assuming > one > process per task with e.g. the command executor or a custom executor) from > using more than its limit. > > Obviously it would be possible for the executor to enforce the > individual limits itself using cgroup - does the command/default executor > do that? - but in our case where we use a custom executor it would be > quite > painful. > > Unless I'm missing something? > > Thanks in advance for suggestions, > > Charles >
Re: Number of tasks per executor and resource limits
> > if we wanted to support multiple tasks per executor, could we leverage the > mesos containeriser to easily put each task in its own cgroup? If what your custom executor does is similar with default executor (i.e., launch each task in a task group as a nested container by calling Mesos agent API `LAUNCH_CONTAINER`), then the answer is yes since we are going to support nested cgroups for nested containers (i.e., each nested container can have its own cgroups) in Mesos containerizer soon. Regards, Qian Zhang On Fri, Feb 21, 2020 at 4:46 AM Vinod Kone wrote: > Andrei, Qian: Can one of you answer the above question? > > On Thu, Feb 20, 2020 at 7:15 PM Charles-François Natali < > cf.nat...@gmail.com> wrote: > >> Thanks for the quick reply! >> >> I think we're going to go for one executor per task for now, that's much >> simpler. >> >> Otherwise I was wondering - if we wanted to support multiple tasks per >> executor, could we leverage the mesos containeriser to easily put each task >> in its own cgroup? >> >> >> >> Is it just a matter of setting 'execu >> On Thu, 20 Feb 2020, 17:40 Vinod Kone, wrote: >> >>> Hi Charles, >>> >>> We are actually working on a new feature that puts each of the tasks (of >>> the default executor) in its own cgroup so that they can be individually >>> limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For >>> custom executor, you would be on your own to implement the same. Also, your >>> custom executor / scheduler should be able to limit one task per executor >>> if you so desire. >>> >>> On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali < >>> cf.nat...@gmail.com> wrote: >>> Hi, Is there a way to force Mesos to use one executor per task? The reason I would want to do that is for resources limits: for example, when using cgroup to limit CPU and memory, IIUC the containeriser sets limits corresponding to the sum of the resources allocated to the tasks managed by the underlying executor. Which means that for example if a task is allocated 1GB and another 2GB, if they are started by the same executor, the collarbone containeriser will limit the total memory for the two tasks (plus the executor) to 3GB. But, unless I'm missing something, nothing prevents a process (assuming one process per task with e.g. the command executor or a custom executor) from using more than its limit. Obviously it would be possible for the executor to enforce the individual limits itself using cgroup - does the command/default executor do that? - but in our case where we use a custom executor it would be quite painful. Unless I'm missing something? Thanks in advance for suggestions, Charles >>>
Re: Number of tasks per executor and resource limits
Andrei, Qian: Can one of you answer the above question? On Thu, Feb 20, 2020 at 7:15 PM Charles-François Natali wrote: > Thanks for the quick reply! > > I think we're going to go for one executor per task for now, that's much > simpler. > > Otherwise I was wondering - if we wanted to support multiple tasks per > executor, could we leverage the mesos containeriser to easily put each task > in its own cgroup? > > > > Is it just a matter of setting 'execu > On Thu, 20 Feb 2020, 17:40 Vinod Kone, wrote: > >> Hi Charles, >> >> We are actually working on a new feature that puts each of the tasks (of >> the default executor) in its own cgroup so that they can be individually >> limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For >> custom executor, you would be on your own to implement the same. Also, your >> custom executor / scheduler should be able to limit one task per executor >> if you so desire. >> >> On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali < >> cf.nat...@gmail.com> wrote: >> >>> Hi, >>> >>> Is there a way to force Mesos to use one executor per task? >>> The reason I would want to do that is for resources limits: for example, >>> when using cgroup to limit CPU and memory, IIUC the containeriser sets >>> limits corresponding to the sum of the resources allocated to the tasks >>> managed by the underlying executor. >>> >>> Which means that for example if a task is allocated 1GB and another 2GB, >>> if they are started by the same executor, the collarbone containeriser will >>> limit the total memory for the two tasks (plus the executor) to 3GB. But, >>> unless I'm missing something, nothing prevents a process (assuming one >>> process per task with e.g. the command executor or a custom executor) from >>> using more than its limit. >>> >>> Obviously it would be possible for the executor to enforce the >>> individual limits itself using cgroup - does the command/default executor >>> do that? - but in our case where we use a custom executor it would be quite >>> painful. >>> >>> Unless I'm missing something? >>> >>> Thanks in advance for suggestions, >>> >>> Charles >>> >>
Re: Number of tasks per executor and resource limits
Thanks for the quick reply! I think we're going to go for one executor per task for now, that's much simpler. Otherwise I was wondering - if we wanted to support multiple tasks per executor, could we leverage the mesos containeriser to easily put each task in its own cgroup? Is it just a matter of setting 'execu On Thu, 20 Feb 2020, 17:40 Vinod Kone, wrote: > Hi Charles, > > We are actually working on a new feature that puts each of the tasks (of > the default executor) in its own cgroup so that they can be individually > limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For > custom executor, you would be on your own to implement the same. Also, your > custom executor / scheduler should be able to limit one task per executor > if you so desire. > > On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali < > cf.nat...@gmail.com> wrote: > >> Hi, >> >> Is there a way to force Mesos to use one executor per task? >> The reason I would want to do that is for resources limits: for example, >> when using cgroup to limit CPU and memory, IIUC the containeriser sets >> limits corresponding to the sum of the resources allocated to the tasks >> managed by the underlying executor. >> >> Which means that for example if a task is allocated 1GB and another 2GB, >> if they are started by the same executor, the collarbone containeriser will >> limit the total memory for the two tasks (plus the executor) to 3GB. But, >> unless I'm missing something, nothing prevents a process (assuming one >> process per task with e.g. the command executor or a custom executor) from >> using more than its limit. >> >> Obviously it would be possible for the executor to enforce the individual >> limits itself using cgroup - does the command/default executor do that? - >> but in our case where we use a custom executor it would be quite painful. >> >> Unless I'm missing something? >> >> Thanks in advance for suggestions, >> >> Charles >> >
Re: Number of tasks per executor and resource limits
Hi Charles, We are actually working on a new feature that puts each of the tasks (of the default executor) in its own cgroup so that they can be individually limited. See https://issues.apache.org/jira/browse/MESOS-9916 . For custom executor, you would be on your own to implement the same. Also, your custom executor / scheduler should be able to limit one task per executor if you so desire. On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali wrote: > Hi, > > Is there a way to force Mesos to use one executor per task? > The reason I would want to do that is for resources limits: for example, > when using cgroup to limit CPU and memory, IIUC the containeriser sets > limits corresponding to the sum of the resources allocated to the tasks > managed by the underlying executor. > > Which means that for example if a task is allocated 1GB and another 2GB, > if they are started by the same executor, the collarbone containeriser will > limit the total memory for the two tasks (plus the executor) to 3GB. But, > unless I'm missing something, nothing prevents a process (assuming one > process per task with e.g. the command executor or a custom executor) from > using more than its limit. > > Obviously it would be possible for the executor to enforce the individual > limits itself using cgroup - does the command/default executor do that? - > but in our case where we use a custom executor it would be quite painful. > > Unless I'm missing something? > > Thanks in advance for suggestions, > > Charles >
