Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On Sat, 07.03.15 00:20, Alban Crequy (alban.cre...@gmail.com) wrote: I figure we could open a new mount namespace and mount the file system socket into the chroot, but not sure I like the idea... Maybe that's the way to do it... but where would you bind mount the socket file? in $CHROOT/tmp which should be writeable when PrivateTmp=true? Of course it will not work if the daemon is doing the chroot itself instead of relying on systemd's RootDirectory. /tmp is not a place for sockets. Whatever code sets up the execution environment which wants to allow notifications would have the responsibility to bind mount the notification socket... I mean, the notification socket isn't really too different from other sockets. If you run your php program in a chroot, and you want ot to access your mysql server via AF_UNIX, then you need to mount the socket over too, that's really the same story here... The same problem exists even without using PrivateNetwork/RootDirectory when the service starts a container with nspawn --private-network and the program inside the container wants to notify when it's ready. This has the same root cause: the service runs in a new mount/chroot and a new network namespace. This is not a problem. This is a feature. I mean, you asked for isolation, hence you got isolation... I am pretty sure that there should not be any way for container member processes to communication via assumed-to-be-local IPC to processes outside of the container, unless they do so with the container manager. In this case meaning: if you want notification like this, then your container manager needs to proxy that. There is also the additional problem that the program inside the container runs in a different cgroup (/system.slice/docker-... for docker containers, or /machine.slice... for nspawn containers). There is the tool sdnotify-proxy to proxy the notify socket from systemd to a socket file which can be bind mounted in the container. sdnotify-proxy works, but I would like to know if someone finds a better way for containers :) I am not sure I understand what sdnotify-proxy does and who needs it? Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On Thu, 05.03.15 12:16, Alban Crequy (alban.cre...@gmail.com) wrote: Hmm, but what would you do for a service that has both PrivateNetwork and chroot enabled? I am all open for shifting things around again, but I inda would prefer a solution that works universally in the end... Ideas? I figure we could open a new mount namespace and mount the file system socket into the chroot, but not sure I like the idea... I don't know what is the best solution but using a socket file seems better than using an abstract unix socket: processes in a systemd-nspawn container could discover the notify socket of the host in /proc/net/unix (if it does not use a new network namespace) and send garbage file descriptors with SCM_RIGHTS from the container to the host. Systemd on the host does the right thing: it closes the fds when the datagram was not sent by a managed unit. Well, this is only an issue if people do not use network namespacing. But if they don't use it they should not expect that much isolation. It's the deal they make... Does manager_get_unit_by_pid() matches the exact cgroup path only or does it also matches a prefix path? I wonder about nspawn containers started by a service unit on the host. No, it cares for subtree membership only. THat said, access can be restricted via NotifyAccess=. If people set it to NotifyAccess=all they of course need to know what they are doing... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On 9 December 2014 at 17:28, Lennart Poettering lenn...@poettering.net wrote: On Tue, 09.12.14 16:24, Krzysztof Kotlenga (k.kotle...@sims.pl) wrote: Hi. Currently notify socket is unavailable in chrooted services (again) unless you bind mount it there. Is there perhaps another, less cumbersome way? So far notify socket was: 1. abstract socket commit 8c47c7325fa1ab72febf807f8831ff24c75fbf45 notify: add minimal readiness/status protocol for spawned daemons 2. file-system socket commit 91b22f21f3824c1766d34f622c5bbb70cbe881a8 core: move abstract namespace sockets to /dev/.run Now that we have /dev/.run there's no need to use abstract namespace sockets. So, let's move things to /dev/.run, to make things more easily discoverable and improve compat with chroot() and fs namespacing. 3. abstract socket again commit 29252e9e5bad3b0bcfc45d9bc761aee4b0ece1da manager: turn notify socket into abstract namespace socket again sd_notify() should work for daemons that chroot() as part of their initilization, hence it's a good idea to use an abstract namespace socket which is not affected by chroot. 4. file-system socket again commit 7181dbdb2e3112858d62bdaea4f0ad2ed685ccba core: move notify sockets to /run and $XDG_RUNTIME_DIR A service with PrivateNetwork= cannot access abstract namespace sockets of the host anymore, hence let's better not use abstract namespace sockets for this, since we want to make sure that PrivateNetwork= is useful and doesn't break sd_notify(). So... would it be acceptable to have two notify sockets, one abstract and one normal, the latter only set for services with PrivateNetwork or - better maybe - explicitly selectable? Any other ideas? Hmm, but what would you do for a service that has both PrivateNetwork and chroot enabled? I am all open for shifting things around again, but I inda would prefer a solution that works universally in the end... Ideas? I figure we could open a new mount namespace and mount the file system socket into the chroot, but not sure I like the idea... Maybe that's the way to do it... but where would you bind mount the socket file? in $CHROOT/tmp which should be writeable when PrivateTmp=true? Of course it will not work if the daemon is doing the chroot itself instead of relying on systemd's RootDirectory. The same problem exists even without using PrivateNetwork/RootDirectory when the service starts a container with nspawn --private-network and the program inside the container wants to notify when it's ready. This has the same root cause: the service runs in a new mount/chroot and a new network namespace. There is also the additional problem that the program inside the container runs in a different cgroup (/system.slice/docker-... for docker containers, or /machine.slice... for nspawn containers). There is the tool sdnotify-proxy to proxy the notify socket from systemd to a socket file which can be bind mounted in the container. sdnotify-proxy works, but I would like to know if someone finds a better way for containers :) Alban ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On 9 December 2014 at 17:28, Lennart Poettering lenn...@poettering.net wrote: On Tue, 09.12.14 16:24, Krzysztof Kotlenga (k.kotle...@sims.pl) wrote: Hi. Currently notify socket is unavailable in chrooted services (again) unless you bind mount it there. Is there perhaps another, less cumbersome way? So far notify socket was: 1. abstract socket commit 8c47c7325fa1ab72febf807f8831ff24c75fbf45 notify: add minimal readiness/status protocol for spawned daemons 2. file-system socket commit 91b22f21f3824c1766d34f622c5bbb70cbe881a8 core: move abstract namespace sockets to /dev/.run Now that we have /dev/.run there's no need to use abstract namespace sockets. So, let's move things to /dev/.run, to make things more easily discoverable and improve compat with chroot() and fs namespacing. 3. abstract socket again commit 29252e9e5bad3b0bcfc45d9bc761aee4b0ece1da manager: turn notify socket into abstract namespace socket again sd_notify() should work for daemons that chroot() as part of their initilization, hence it's a good idea to use an abstract namespace socket which is not affected by chroot. 4. file-system socket again commit 7181dbdb2e3112858d62bdaea4f0ad2ed685ccba core: move notify sockets to /run and $XDG_RUNTIME_DIR A service with PrivateNetwork= cannot access abstract namespace sockets of the host anymore, hence let's better not use abstract namespace sockets for this, since we want to make sure that PrivateNetwork= is useful and doesn't break sd_notify(). So... would it be acceptable to have two notify sockets, one abstract and one normal, the latter only set for services with PrivateNetwork or - better maybe - explicitly selectable? Any other ideas? Hmm, but what would you do for a service that has both PrivateNetwork and chroot enabled? I am all open for shifting things around again, but I inda would prefer a solution that works universally in the end... Ideas? I figure we could open a new mount namespace and mount the file system socket into the chroot, but not sure I like the idea... I don't know what is the best solution but using a socket file seems better than using an abstract unix socket: processes in a systemd-nspawn container could discover the notify socket of the host in /proc/net/unix (if it does not use a new network namespace) and send garbage file descriptors with SCM_RIGHTS from the container to the host. Systemd on the host does the right thing: it closes the fds when the datagram was not sent by a managed unit. Does manager_get_unit_by_pid() matches the exact cgroup path only or does it also matches a prefix path? I wonder about nspawn containers started by a service unit on the host. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On Tue, 2014-12-09 at 17:28 +0100, Lennart Poettering wrote: On Tue, 09.12.14 16:24, Krzysztof Kotlenga (k.kotle...@sims.pl) wrote: Currently notify socket is unavailable in chrooted services (again) unless you bind mount it there. Is there perhaps another, less cumbersome way? So far notify socket was: [snip change history] So... would it be acceptable to have two notify sockets, one abstract and one normal, the latter only set for services with PrivateNetwork or - better maybe - explicitly selectable? Any other ideas? Hmm, but what would you do for a service that has both PrivateNetwork and chroot enabled? Well, PrivateNetwork is clearly asking for trouble... A socket doesn't really look like a right tool for the job in this case. I know Unix signals were previously discarded in https://bugzilla.redhat.com/show_bug.cgi?id=833105 Dunno what's left. Ideas? I figure we could open a new mount namespace and mount the file system socket into the chroot, but not sure I like the idea... I don't know what else is possible, sorry. Anyone? -- K. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On Wed, 10.12.14 16:33, Krzysztof Kotlenga (k.kotle...@sims.pl) wrote: On Tue, 2014-12-09 at 17:28 +0100, Lennart Poettering wrote: On Tue, 09.12.14 16:24, Krzysztof Kotlenga (k.kotle...@sims.pl) wrote: Currently notify socket is unavailable in chrooted services (again) unless you bind mount it there. Is there perhaps another, less cumbersome way? So far notify socket was: [snip change history] So... would it be acceptable to have two notify sockets, one abstract and one normal, the latter only set for services with PrivateNetwork or - better maybe - explicitly selectable? Any other ideas? Hmm, but what would you do for a service that has both PrivateNetwork and chroot enabled? Well, PrivateNetwork is clearly asking for trouble... A socket doesn't really look like a right tool for the job in this case. I know Unix signals were previously discarded in https://bugzilla.redhat.com/show_bug.cgi?id=833105 Dunno what's left. AF_UNIX sockets in the file system namespace are unaffected from PrivateNetwork= really. It's just the combination of file system rearrangements and PrivateNetwork= that is the problem... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Notification socket and chroot vs PrivateNetwork conflict (abstract vs file-system)
On Tue, 09.12.14 16:24, Krzysztof Kotlenga (k.kotle...@sims.pl) wrote: Hi. Currently notify socket is unavailable in chrooted services (again) unless you bind mount it there. Is there perhaps another, less cumbersome way? So far notify socket was: 1. abstract socket commit 8c47c7325fa1ab72febf807f8831ff24c75fbf45 notify: add minimal readiness/status protocol for spawned daemons 2. file-system socket commit 91b22f21f3824c1766d34f622c5bbb70cbe881a8 core: move abstract namespace sockets to /dev/.run Now that we have /dev/.run there's no need to use abstract namespace sockets. So, let's move things to /dev/.run, to make things more easily discoverable and improve compat with chroot() and fs namespacing. 3. abstract socket again commit 29252e9e5bad3b0bcfc45d9bc761aee4b0ece1da manager: turn notify socket into abstract namespace socket again sd_notify() should work for daemons that chroot() as part of their initilization, hence it's a good idea to use an abstract namespace socket which is not affected by chroot. 4. file-system socket again commit 7181dbdb2e3112858d62bdaea4f0ad2ed685ccba core: move notify sockets to /run and $XDG_RUNTIME_DIR A service with PrivateNetwork= cannot access abstract namespace sockets of the host anymore, hence let's better not use abstract namespace sockets for this, since we want to make sure that PrivateNetwork= is useful and doesn't break sd_notify(). So... would it be acceptable to have two notify sockets, one abstract and one normal, the latter only set for services with PrivateNetwork or - better maybe - explicitly selectable? Any other ideas? Hmm, but what would you do for a service that has both PrivateNetwork and chroot enabled? I am all open for shifting things around again, but I inda would prefer a solution that works universally in the end... Ideas? I figure we could open a new mount namespace and mount the file system socket into the chroot, but not sure I like the idea... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
