Perfect, thanks!
On Fri, 21 Feb 2020, 00:55 Qian Zhang, <zhq527...@gmail.com> wrote: > if we wanted to support multiple tasks per executor, could we leverage the >> mesos containeriser to easily put each task in its own cgroup? > > > If what your custom executor does is similar with default executor (i.e., > launch each task in a task group as a nested container by calling Mesos > agent API `LAUNCH_CONTAINER`), then the answer is yes since we are going to > support nested cgroups for nested containers (i.e., each nested container > can have its own cgroups) in Mesos containerizer soon. > > > Regards, > Qian Zhang > > > On Fri, Feb 21, 2020 at 4:46 AM Vinod Kone <vinodk...@apache.org> wrote: > >> Andrei, Qian: Can one of you answer the above question? >> >> On Thu, Feb 20, 2020 at 7:15 PM Charles-François Natali < >> cf.nat...@gmail.com> wrote: >> >>> Thanks for the quick reply! >>> >>> I think we're going to go for one executor per task for now, that's much >>> simpler. >>> >>> Otherwise I was wondering - if we wanted to support multiple tasks per >>> executor, could we leverage the mesos containeriser to easily put each task >>> in its own cgroup? >>> >>> >>> >>> Is it just a matter of setting 'execu >>> On Thu, 20 Feb 2020, 17:40 Vinod Kone, <vinodk...@apache.org> wrote: >>> >>>> Hi Charles, >>>> >>>> We are actually working on a new feature that puts each of the tasks >>>> (of the default executor) in its own cgroup so that they can be >>>> individually limited. See >>>> https://issues.apache.org/jira/browse/MESOS-9916 . For custom >>>> executor, you would be on your own to implement the same. Also, your custom >>>> executor / scheduler should be able to limit one task per executor if you >>>> so desire. >>>> >>>> On Thu, Feb 20, 2020 at 6:34 PM Charles-François Natali < >>>> cf.nat...@gmail.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> Is there a way to force Mesos to use one executor per task? >>>>> The reason I would want to do that is for resources limits: for >>>>> example, when using cgroup to limit CPU and memory, IIUC the containeriser >>>>> sets limits corresponding to the sum of the resources allocated to the >>>>> tasks managed by the underlying executor. >>>>> >>>>> Which means that for example if a task is allocated 1GB and another >>>>> 2GB, if they are started by the same executor, the collarbone >>>>> containeriser >>>>> will limit the total memory for the two tasks (plus the executor) to 3GB. >>>>> But, unless I'm missing something, nothing prevents a process (assuming >>>>> one >>>>> process per task with e.g. the command executor or a custom executor) from >>>>> using more than its limit. >>>>> >>>>> Obviously it would be possible for the executor to enforce the >>>>> individual limits itself using cgroup - does the command/default executor >>>>> do that? - but in our case where we use a custom executor it would be >>>>> quite >>>>> painful. >>>>> >>>>> Unless I'm missing something? >>>>> >>>>> Thanks in advance for suggestions, >>>>> >>>>> Charles >>>>> >>>>
