On Mon, Jun 24, 2013 at 11:38 AM, Tejun Heo <t...@kernel.org> wrote: > Hello, > > On Mon, Jun 24, 2013 at 03:27:15PM +0200, Lennart Poettering wrote: >> On Sat, 22.06.13 15:19, Andy Lutomirski (l...@amacapital.net) wrote: >> >> > 1. I put all the entire world into a separate, highly constrained >> > cgroup. My real-time code runs outside that cgroup. This seems to >> > exactly what slices are for, but I need kernel threads to go in to >> > the constrained cgroup. Will systemd support this? >> >> I am not sure whether the ability to move kernel threads into cgroups >> will stay around at all, from the kernel side. Tejun, can you comment on >> this? > > Any kernel threads with PF_NO_SETAFFINITY set already can't be removed > from the root cgroup. In general, I don't think moving kernel threads > into !root cgroups is a good idea. They're in most cases shared > resources and userland doesn't really have much idea what they're > actually doing, which is the fundmental issue. > > Which kthreads are running on the kernel side and what they're doing > is strict implementation detail from the kernel side. There's no > effort from kernel side in keeping them stable and userland is likely > to get things completely wrong - e.g. many kernel threads named after > workqueues in any recent kernels don't actually do anything until the > system is under heavy memory pressure. Userland can't tell and has no > control over what's being executed where at all and that's the way it > should be. > > That said, there are cases where certain async executions are > concretely bound to userland processes - say, (planned) aio updates, > virt drivers and so on. Right now, virt implements something pretty > hacky but I think they'll have to be tied closer to the usual process > mechanism - ie. they should be saying that these kthreads are serving > this process and should be treated as such in terms of resource > control rather than the current "move this kthread to this set of > cgroups, don't ask why" thing. Another not-well-thought-out aspect of > the current cgroup. :( > > I have an idea where it should be headed in the long term but am not > sure about short-term solution. Given that the only sort wide-spread > use case is virt kthreads, maybe it just needs to be special cased for > now. Not sure.
I'll be okay (I think) if I can reliably set affinities of these threads. I'm currently doing it with cgroups. That being said, I don't like the direction that kernel thread magic affinity is going. It may be great for cache performance and reducing random bounding, but I have a scheduling-jitter-sensitive workload and I don't care about overall system throughput. I need the kernel to stay the f!&k off my important cpus, and arranging for this to happen is becoming increasingly complicated. > >> > 2. I manage services and tasks outside systemd (for one thing, I >> > currently use Ubuntu, but even if I were on Fedora, I have a bunch >> > of fine-grained things that figure out how they're supposed to >> > allocate resources, and porting them to systemd just to keep working >> > in the new world order would be a PITA [1]). >> > >> > (cgroups have the odd feature that they are per-task, not per thread >> > group, and the systemd proposal seems likely to break anything that >> > actually wants task granularity. I may actually want to use this, >> > even though it's a bit evil -- my real-time thread groups have >> > non-real-time threads.) >> >> Here too, Tejun is pretty keen on removing the ability of splitting up >> threads into cgroups from the kernel, and will only allow this >> per-process. Tejun, please comment! > > Yes, again, the biggest issue is how much of low-level cgroup details > become known to individual programs. Splitting threads into different > cgroup would in most cases mean that the binary itself would become > aware of cgroup and it's akin to burying sysctl knob tunings into > individual binaries. cgroup is not an interface for each individual > program to fiddle with. If certain thread-granular control is > absolutely necessary and justifiable, it's something to be added to > the existing thread API, not something to be bolted on using cgroups. > cgroups are most certainly something that a binary can be aware of. It's not like a sysctl knob at all -- it's per process. I have lots of binaries that have worked quite well for a couple years that move themselves into different cgroups. I have no problem with a unified hierarchy, but I need control of my little piece of the hierarchy. I don't care if the interface to do so changes, but the basic functionality is important. > So, I'm quite strongly against allowing allowing splitting threads of > the same process into different cgroups. I don't need that feature. (Which is not to say that no one else does.) --Andy _______________________________________________ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
