[jira] [Commented] (MESOS-7990) Support systemd named hierarchy (name=systemd) for Mesos Containerizer.

[Jie Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359169#comment-16359169
 ] 

Jie Yu commented on MESOS-7990:
---

commit a86ff8c36532f97b6eb6b44c6f871de24afbcc4d (HEAD -> master, origin/master, 
origin/HEAD)
Author: Jie Yu 
Date: Thu Oct 5 21:11:41 2017 -0700

Created cgroups under systemd hierarchy in LinuxLauncher.

This patch added the support for systemd hierarchy in LinuxLauncher.
 It created the same cgroup layout under the systemd hierarchy (if
 systemd is enabled) as that in the freezer hierarchy.

This can give us a bunch of benefits:
 1) systemd-cgls can list mesos container processes.
 2) systemd-cgtop can show stats for mesos containers.
 3) Avoid the pid migration issue described in MESOS-3352.

For example:

```
 [jie@core-dev ~]$ systemd-cgls
 |-1 /usr/lib/systemd/systemd --system --deserialize 20
 |-mesos
 | |-8282b91a-5724-4964-a623-7c6bd68ff4ad
 | |-31737 /usr/libexec/mesos/mesos-containerizer launch
 | |-31739 mesos-default-executor --launcher_dir=/usr/libexec/mesos
 | |-mesos
 | |-8555f4af-fa4f-4c9c-aeb3-0c9f72e6a2de
 | |-31791 /usr/libexec/mesos/mesos-containerizer launch
 | |-31793 sleep 1000
 ```

Review: https://reviews.apache.org/r/62800

commit 05a2909508df56253372b4fe36330339b5de00b1
Author: Jie Yu 
Date: Thu Oct 5 20:52:38 2017 -0700

Fixed an issue for the I/O switchboard process lifetime.

We expect the I/O switchboard process to last across agent restarts
 (similar to log rotate process or executor processes). Therefore, we
 should put it into 'mesos_executor.slice' like others.

Review: https://reviews.apache.org/r/62799

commit 2ece53c39a3e791f6892c4a734b6d3187f184190
Author: Jie Yu 
Date: Thu Oct 5 20:49:20 2017 -0700

Added named cgroup hierarchy support.

This patch add a helper to get the cgroup associated with the given
 pid for a named cgroup hierarchy.

Review: https://reviews.apache.org/r/62798

> Support systemd named hierarchy (name=systemd) for Mesos Containerizer.
> ---
>
> Key: MESOS-7990
> URL: https://issues.apache.org/jira/browse/MESOS-7990
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Jie Yu
>Priority: Major
>
> Similar to docker's cgroupfs cgroup driver, we should create cgroups under 
> /sys/fs/cgroup/systemd (if it exists), and move container pid into the 
> corresponding cgroup ( /sys/fs/cgroup/systemd/mesos/).
> This can give us a bunch of benefits:
> 1) systemd-cgls can list mesos containers
> 2) systemd-cgtop can show stats for mesos containers
> ...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (MESOS-7990) Support systemd named hierarchy (name=systemd) for Mesos Containerizer.

[Jie Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195300#comment-16195300
 ] 

Jie Yu commented on MESOS-7990:
---

Some sample output:
{noformat}
[jie@core-dev ~]$ systemd-cgls
|-1 /usr/lib/systemd/systemd --system --deserialize 20
|-mesos
|  |-8282b91a-5724-4964-a623-7c6bd68ff4ad
|  |-31737 /usr/libexec/mesos/mesos-containerizer launch
|  |-31739 mesos-default-executor --launcher_dir=/usr/libexec/mesos
|  |-mesos
| |-8555f4af-fa4f-4c9c-aeb3-0c9f72e6a2de
|   |-31791 /usr/libexec/mesos/mesos-containerizer launch
|   |-31793 sleep 1000
{noformat}

> Support systemd named hierarchy (name=systemd) for Mesos Containerizer.
> ---
>
> Key: MESOS-7990
> URL: https://issues.apache.org/jira/browse/MESOS-7990
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: Jie Yu
>Assignee: Jie Yu
>
> Similar to docker's cgroupfs cgroup driver, we should create cgroups under 
> /sys/fs/cgroup/systemd (if it exists), and move container pid into the 
> corresponding cgroup ( /sys/fs/cgroup/systemd/mesos/).
> This can give us a bunch of benefits:
> 1) systemd-cgls can list mesos containers
> 2) systemd-cgtop can show stats for mesos containers
> ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7990) Support systemd named hierarchy (name=systemd) for Mesos Containerizer.

[Jie Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194123#comment-16194123
 ] 

Jie Yu commented on MESOS-7990:
---

https://reviews.apache.org/r/62798/
https://reviews.apache.org/r/62800/

> Support systemd named hierarchy (name=systemd) for Mesos Containerizer.
> ---
>
> Key: MESOS-7990
> URL: https://issues.apache.org/jira/browse/MESOS-7990
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: Jie Yu
>
> Similar to docker's cgroupfs cgroup driver, we should create cgroups under 
> /sys/fs/cgroup/systemd (if it exists), and move container pid into the 
> corresponding cgroup ( /sys/fs/cgroup/systemd/mesos/).
> This can give us a bunch of benefits:
> 1) systemd-cgls can list mesos containers
> 2) systemd-cgtop can show stats for mesos containers
> ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7990) Support systemd named hierarchy (name=systemd) for Mesos Containerizer.

[Jie Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175460#comment-16175460
 ] 

Jie Yu commented on MESOS-7990:
---

[~jasonlai] Yes, i am aware of that (the naming convention). In fact, we're not 
supposed to touch named systemd cgroup hierarchy manually. However, major 
container orchestrators (docker, k8s) all manipulate systemd cgroup hierarchy 
directly. They all have an alternative mode that supports systemd more natively 
(using machined or system slice for containers, like rkt does).

We want to support both too. The native systemd support will be added later. 
This ticket is for the cgroupfs support.

> Support systemd named hierarchy (name=systemd) for Mesos Containerizer.
> ---
>
> Key: MESOS-7990
> URL: https://issues.apache.org/jira/browse/MESOS-7990
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: Jie Yu
>
> Similar to docker's cgroupfs cgroup driver, we should create cgroups under 
> /sys/fs/cgroup/systemd (if it exists), and move container pid into the 
> corresponding cgroup ( /sys/fs/cgroup/systemd/mesos/).
> This can give us a bunch of benefits:
> 1) systemd-cgls can list mesos containers
> 2) systemd-cgtop can show stats for mesos containers
> ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (MESOS-7990) Support systemd named hierarchy (name=systemd) for Mesos Containerizer.

[Jason Lai (JIRA)]

[ 
https://issues.apache.org/jira/browse/MESOS-7990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16175448#comment-16175448
 ] 

Jason Lai commented on MESOS-7990:
--

I'm all for the systemd support. But it doesn't go that easily as 
{{/systemd/mesos/}}, as Systemd has imposed some conventions on tasks' 
cgroup names. There are some references 
[here|https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/resource_management_guide/sec-default_cgroup_hierarchies].
 AFAIK, rkt has aligned pretty well with the systemd conventions for their 
containers. Would be worth looking at what they're doing

> Support systemd named hierarchy (name=systemd) for Mesos Containerizer.
> ---
>
> Key: MESOS-7990
> URL: https://issues.apache.org/jira/browse/MESOS-7990
> Project: Mesos
>  Issue Type: Improvement
>  Components: containerization
>Reporter: Jie Yu
>
> Similar to docker's cgroupfs cgroup driver, we should create cgroups under 
> /sys/fs/cgroup/systemd (if it exists), and move container pid into the 
> corresponding cgroup ( /sys/fs/cgroup/systemd/mesos/).
> This can give us a bunch of benefits:
> 1) systemd-cgls can list mesos containers
> 2) systemd-cgtop can show stats for mesos containers
> ...



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)