Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-05 Thread Michael Kerrisk (man-pages) 
Hello Tejun,

On 5 December 2017 at 17:00, Tejun Heo  wrote:
> Hello, Michael.
>
> On Tue, Dec 05, 2017 at 08:45:19AM +0100, Michael Kerrisk (man-pages) wrote:
>> b) The reason for my initial problems was this test in
>> the kernel in cpu_cgroup_can_attach():
>>
>> #ifdef CONFIG_RT_GROUP_SCHED
>> if (!sched_rt_can_attach(css_tg(css), task))
>> return -EINVAL;
>> #else
>> /* We don't support RT-tasks being in separate groups */
>> if (task->sched_class != _sched_class)
>> return -EINVAL;
>> #endif
>>
>> I don't have CONFIG_RT_GROUP_SCHED, and the second 'if' was yielding
>> false because of some SCHED_RR processes that are in some of the nonroot
>> cgroups created by systemd, namely:
>
> I see.  Thanks for tracking it down.  Yeah, the RT side of things
> isn't too well thought out yet.
>
>> # ps ax -L -o 'pid tid cls rtprio comm'|grep RR
>>   685   723  RR 99 rtkit-daemon
>>   972   979  RR  5 alsa-sink-ALC26
>>   972   982  RR  5 alsa-source-ALC
>>  1594  1597  RR  5 alsa-sink-ALC26
>>  1594  1600  RR  5 alsa-source-ALC
>>
>> So, one solution is to move those processes to the root cgroup,
>> and then it's possible to write '+pids' to cgroup.subtree_control.
>
> You mean '+cpu", right?

D'oh! Yes. I was also doing some tests with the 'pids' controller, and
mistyped. (I see I even manage to mistitle this mail thread :-(.)

>> Is enabling CONFIG_RT_GROUP_SCHED also a solution? (I have
>> not had a chance to test that yet.)
>
> We aren't yet sure about how we should handle RT and haven't enabled
> RT part on cgroup2 yet.  You can test the same scenario in cgroup1 tho
> and would have to configure RT shares all along the hierarchy to the
> leaf cgroup.

Okay. Thanks for the confirmation that this is at least surprising behavior.

>> Anyway, it seems like this should be documented somewhere in the
>> kernel Documentation files, since it may be that others will run
>> into this as well. I'm not quite sure what should be added to the
>> documentation. Do you have some idea?
>
> I think the only thing we can say right now is that RT processes
> should be in the root cgroup.  :(

Okay. (Will you add that to the v2 doc file?)

Cheers,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-05 Thread Michael Kerrisk (man-pages) 
Hello Tejun,

On 5 December 2017 at 17:00, Tejun Heo  wrote:
> Hello, Michael.
>
> On Tue, Dec 05, 2017 at 08:45:19AM +0100, Michael Kerrisk (man-pages) wrote:
>> b) The reason for my initial problems was this test in
>> the kernel in cpu_cgroup_can_attach():
>>
>> #ifdef CONFIG_RT_GROUP_SCHED
>> if (!sched_rt_can_attach(css_tg(css), task))
>> return -EINVAL;
>> #else
>> /* We don't support RT-tasks being in separate groups */
>> if (task->sched_class != _sched_class)
>> return -EINVAL;
>> #endif
>>
>> I don't have CONFIG_RT_GROUP_SCHED, and the second 'if' was yielding
>> false because of some SCHED_RR processes that are in some of the nonroot
>> cgroups created by systemd, namely:
>
> I see.  Thanks for tracking it down.  Yeah, the RT side of things
> isn't too well thought out yet.
>
>> # ps ax -L -o 'pid tid cls rtprio comm'|grep RR
>>   685   723  RR 99 rtkit-daemon
>>   972   979  RR  5 alsa-sink-ALC26
>>   972   982  RR  5 alsa-source-ALC
>>  1594  1597  RR  5 alsa-sink-ALC26
>>  1594  1600  RR  5 alsa-source-ALC
>>
>> So, one solution is to move those processes to the root cgroup,
>> and then it's possible to write '+pids' to cgroup.subtree_control.
>
> You mean '+cpu", right?

D'oh! Yes. I was also doing some tests with the 'pids' controller, and
mistyped. (I see I even manage to mistitle this mail thread :-(.)

>> Is enabling CONFIG_RT_GROUP_SCHED also a solution? (I have
>> not had a chance to test that yet.)
>
> We aren't yet sure about how we should handle RT and haven't enabled
> RT part on cgroup2 yet.  You can test the same scenario in cgroup1 tho
> and would have to configure RT shares all along the hierarchy to the
> leaf cgroup.

Okay. Thanks for the confirmation that this is at least surprising behavior.

>> Anyway, it seems like this should be documented somewhere in the
>> kernel Documentation files, since it may be that others will run
>> into this as well. I'm not quite sure what should be added to the
>> documentation. Do you have some idea?
>
> I think the only thing we can say right now is that RT processes
> should be in the root cgroup.  :(

Okay. (Will you add that to the v2 doc file?)

Cheers,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-05 Thread Tejun Heo 
Hello, Michael.

On Tue, Dec 05, 2017 at 08:45:19AM +0100, Michael Kerrisk (man-pages) wrote:
> b) The reason for my initial problems was this test in
> the kernel in cpu_cgroup_can_attach():
> 
> #ifdef CONFIG_RT_GROUP_SCHED
> if (!sched_rt_can_attach(css_tg(css), task))
> return -EINVAL;
> #else
> /* We don't support RT-tasks being in separate groups */
> if (task->sched_class != _sched_class)
> return -EINVAL;
> #endif
> 
> I don't have CONFIG_RT_GROUP_SCHED, and the second 'if' was yielding
> false because of some SCHED_RR processes that are in some of the nonroot
> cgroups created by systemd, namely:

I see.  Thanks for tracking it down.  Yeah, the RT side of things
isn't too well thought out yet.

> # ps ax -L -o 'pid tid cls rtprio comm'|grep RR
>   685   723  RR 99 rtkit-daemon
>   972   979  RR  5 alsa-sink-ALC26
>   972   982  RR  5 alsa-source-ALC
>  1594  1597  RR  5 alsa-sink-ALC26
>  1594  1600  RR  5 alsa-source-ALC
> 
> So, one solution is to move those processes to the root cgroup,
> and then it's possible to write '+pids' to cgroup.subtree_control.

You mean '+cpu", right?

> Is enabling CONFIG_RT_GROUP_SCHED also a solution? (I have
> not had a chance to test that yet.)

We aren't yet sure about how we should handle RT and haven't enabled
RT part on cgroup2 yet.  You can test the same scenario in cgroup1 tho
and would have to configure RT shares all along the hierarchy to the
leaf cgroup.

> Anyway, it seems like this should be documented somewhere in the
> kernel Documentation files, since it may be that others will run
> into this as well. I'm not quite sure what should be added to the
> documentation. Do you have some idea?

I think the only thing we can say right now is that RT processes
should be in the root cgroup.  :(

Thanks a lot.

-- 
tejun

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-05 Thread Tejun Heo 
Hello, Michael.

On Tue, Dec 05, 2017 at 08:45:19AM +0100, Michael Kerrisk (man-pages) wrote:
> b) The reason for my initial problems was this test in
> the kernel in cpu_cgroup_can_attach():
> 
> #ifdef CONFIG_RT_GROUP_SCHED
> if (!sched_rt_can_attach(css_tg(css), task))
> return -EINVAL;
> #else
> /* We don't support RT-tasks being in separate groups */
> if (task->sched_class != _sched_class)
> return -EINVAL;
> #endif
> 
> I don't have CONFIG_RT_GROUP_SCHED, and the second 'if' was yielding
> false because of some SCHED_RR processes that are in some of the nonroot
> cgroups created by systemd, namely:

I see.  Thanks for tracking it down.  Yeah, the RT side of things
isn't too well thought out yet.

> # ps ax -L -o 'pid tid cls rtprio comm'|grep RR
>   685   723  RR 99 rtkit-daemon
>   972   979  RR  5 alsa-sink-ALC26
>   972   982  RR  5 alsa-source-ALC
>  1594  1597  RR  5 alsa-sink-ALC26
>  1594  1600  RR  5 alsa-source-ALC
> 
> So, one solution is to move those processes to the root cgroup,
> and then it's possible to write '+pids' to cgroup.subtree_control.

You mean '+cpu", right?

> Is enabling CONFIG_RT_GROUP_SCHED also a solution? (I have
> not had a chance to test that yet.)

We aren't yet sure about how we should handle RT and haven't enabled
RT part on cgroup2 yet.  You can test the same scenario in cgroup1 tho
and would have to configure RT shares all along the hierarchy to the
leaf cgroup.

> Anyway, it seems like this should be documented somewhere in the
> kernel Documentation files, since it may be that others will run
> into this as well. I'm not quite sure what should be added to the
> documentation. Do you have some idea?

I think the only thing we can say right now is that RT processes
should be in the root cgroup.  :(

Thanks a lot.

-- 
tejun

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-04 Thread Michael Kerrisk (man-pages) 
[dropping Lennart into CC]

Hello Tejun,

On 12/04/2017 10:47 PM, Tejun Heo wrote:
> Hello, Michael.
> 
> On Mon, Dec 04, 2017 at 10:35:13PM +0100, Michael Kerrisk (man-pages) wrote:
>> I was trying to do some simple testing ot the CPU controller
>> that is merged into 4.15, and ran immediately into some confusion.
>> In the root cgroup on a freshly booted 4.150-rc1, I try the following:
>>
>> # pwd
>> /sys/fs/cgroup/unified
>> # echo '+cpu' > cgroup.subtree_control 
>> sh: echo: write error: Invalid argument
>>
>> What am I missing> I presume I'm missing something obvious, although
>> nothing jumped out at me as I read the cgroups-v2.txt file.
> 
> Checking whether I messed up something really basic... hmmm doesn't
> seem that way.  What do /sys/fs/cgroup/unified/cgroup.controllers and
> /proc/cgroups say?

Oh -- they're all sensible:

In the root cgroup:

# cat cgroup.controllers 
cpu io memory pids

$ cat /proc/cgroups 
#subsys_namehierarchy   num_cgroups enabled
cpuset  0   142 1
cpu 0   142 1
cpuacct 0   142 1
blkio   0   142 1
memory  0   142 1
devices 0   142 1
freezer 0   142 1
net_cls 0   142 1
perf_event  0   142 1
net_prio0   142 1
hugetlb 0   142 1
pids0   142 1

But, I through some trial and error and printk() I worked out

a) If I first move all tasks to the root cgroup, then I can
write '+cpu' to the cgroup.subtree_control file in the root
cgroup.

b) The reason for my initial problems was this test in
the kernel in cpu_cgroup_can_attach():

#ifdef CONFIG_RT_GROUP_SCHED
if (!sched_rt_can_attach(css_tg(css), task))
return -EINVAL;
#else
/* We don't support RT-tasks being in separate groups */
if (task->sched_class != _sched_class)
return -EINVAL;
#endif

I don't have CONFIG_RT_GROUP_SCHED, and the second 'if' was yielding
false because of some SCHED_RR processes that are in some of the nonroot
cgroups created by systemd, namely:

# ps ax -L -o 'pid tid cls rtprio comm'|grep RR
  685   723  RR 99 rtkit-daemon
  972   979  RR  5 alsa-sink-ALC26
  972   982  RR  5 alsa-source-ALC
 1594  1597  RR  5 alsa-sink-ALC26
 1594  1600  RR  5 alsa-source-ALC

So, one solution is to move those processes to the root cgroup,
and then it's possible to write '+pids' to cgroup.subtree_control.

Is enabling CONFIG_RT_GROUP_SCHED also a solution? (I have
not had a chance to test that yet.)

Anyway, it seems like this should be documented somewhere in the
kernel Documentation files, since it may be that others will run
into this as well. I'm not quite sure what should be added to the
documentation. Do you have some idea?

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-04 Thread Michael Kerrisk (man-pages) 
[dropping Lennart into CC]

Hello Tejun,

On 12/04/2017 10:47 PM, Tejun Heo wrote:
> Hello, Michael.
> 
> On Mon, Dec 04, 2017 at 10:35:13PM +0100, Michael Kerrisk (man-pages) wrote:
>> I was trying to do some simple testing ot the CPU controller
>> that is merged into 4.15, and ran immediately into some confusion.
>> In the root cgroup on a freshly booted 4.150-rc1, I try the following:
>>
>> # pwd
>> /sys/fs/cgroup/unified
>> # echo '+cpu' > cgroup.subtree_control 
>> sh: echo: write error: Invalid argument
>>
>> What am I missing> I presume I'm missing something obvious, although
>> nothing jumped out at me as I read the cgroups-v2.txt file.
> 
> Checking whether I messed up something really basic... hmmm doesn't
> seem that way.  What do /sys/fs/cgroup/unified/cgroup.controllers and
> /proc/cgroups say?

Oh -- they're all sensible:

In the root cgroup:

# cat cgroup.controllers 
cpu io memory pids

$ cat /proc/cgroups 
#subsys_namehierarchy   num_cgroups enabled
cpuset  0   142 1
cpu 0   142 1
cpuacct 0   142 1
blkio   0   142 1
memory  0   142 1
devices 0   142 1
freezer 0   142 1
net_cls 0   142 1
perf_event  0   142 1
net_prio0   142 1
hugetlb 0   142 1
pids0   142 1

But, I through some trial and error and printk() I worked out

a) If I first move all tasks to the root cgroup, then I can
write '+cpu' to the cgroup.subtree_control file in the root
cgroup.

b) The reason for my initial problems was this test in
the kernel in cpu_cgroup_can_attach():

#ifdef CONFIG_RT_GROUP_SCHED
if (!sched_rt_can_attach(css_tg(css), task))
return -EINVAL;
#else
/* We don't support RT-tasks being in separate groups */
if (task->sched_class != _sched_class)
return -EINVAL;
#endif

I don't have CONFIG_RT_GROUP_SCHED, and the second 'if' was yielding
false because of some SCHED_RR processes that are in some of the nonroot
cgroups created by systemd, namely:

# ps ax -L -o 'pid tid cls rtprio comm'|grep RR
  685   723  RR 99 rtkit-daemon
  972   979  RR  5 alsa-sink-ALC26
  972   982  RR  5 alsa-source-ALC
 1594  1597  RR  5 alsa-sink-ALC26
 1594  1600  RR  5 alsa-source-ALC

So, one solution is to move those processes to the root cgroup,
and then it's possible to write '+pids' to cgroup.subtree_control.

Is enabling CONFIG_RT_GROUP_SCHED also a solution? (I have
not had a chance to test that yet.)

Anyway, it seems like this should be documented somewhere in the
kernel Documentation files, since it may be that others will run
into this as well. I'm not quite sure what should be added to the
documentation. Do you have some idea?

Thanks,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-04 Thread Tejun Heo 
Hello, Michael.

On Mon, Dec 04, 2017 at 10:35:13PM +0100, Michael Kerrisk (man-pages) wrote:
> I was trying to do some simple testing ot the CPU controller
> that is merged into 4.15, and ran immediately into some confusion.
> In the root cgroup on a freshly booted 4.150-rc1, I try the following:
> 
> # pwd
> /sys/fs/cgroup/unified
> # echo '+cpu' > cgroup.subtree_control 
> sh: echo: write error: Invalid argument
>
> What am I missing> I presume I'm missing something obvious, although
> nothing jumped out at me as I read the cgroups-v2.txt file.

Checking whether I messed up something really basic... hmmm doesn't
seem that way.  What do /sys/fs/cgroup/unified/cgroup.controllers and
/proc/cgroups say?

Thanks.

-- 
tejun

Re: Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-04 Thread Tejun Heo 
Hello, Michael.

On Mon, Dec 04, 2017 at 10:35:13PM +0100, Michael Kerrisk (man-pages) wrote:
> I was trying to do some simple testing ot the CPU controller
> that is merged into 4.15, and ran immediately into some confusion.
> In the root cgroup on a freshly booted 4.150-rc1, I try the following:
> 
> # pwd
> /sys/fs/cgroup/unified
> # echo '+cpu' > cgroup.subtree_control 
> sh: echo: write error: Invalid argument
>
> What am I missing> I presume I'm missing something obvious, although
> nothing jumped out at me as I read the cgroups-v2.txt file.

Checking whether I messed up something really basic... hmmm doesn't
seem that way.  What do /sys/fs/cgroup/unified/cgroup.controllers and
/proc/cgroups say?

Thanks.

-- 
tejun

Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-04 Thread Michael Kerrisk (man-pages) 
Hello Tejun,

I was trying to do some simple testing ot the CPU controller
that is merged into 4.15, and ran immediately into some confusion.
In the root cgroup on a freshly booted 4.150-rc1, I try the following:

# pwd
/sys/fs/cgroup/unified
# echo '+cpu' > cgroup.subtree_control 
sh: echo: write error: Invalid argument

What am I missing> I presume I'm missing something obvious, although
nothing jumped out at me as I read the cgroups-v2.txt file.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Writing "+pids" to cgroup.subtree_control flie yields EINVAL

2017-12-04 Thread Michael Kerrisk (man-pages) 
Hello Tejun,

I was trying to do some simple testing ot the CPU controller
that is merged into 4.15, and ran immediately into some confusion.
In the root cgroup on a freshly booted 4.150-rc1, I try the following:

# pwd
/sys/fs/cgroup/unified
# echo '+cpu' > cgroup.subtree_control 
sh: echo: write error: Invalid argument

What am I missing> I presume I'm missing something obvious, although
nothing jumped out at me as I read the cgroups-v2.txt file.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/