Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Sun, Dec 8, 2013 at 8:25 PM, Shawn Landden shawnland...@gmail.com wrote: This one doesn't work. I have have a somewhat-working next patch, but the way epoll_wait works it actually isn't lazy at all, and would require EPOLLET to even do one-spawn-per-connection (global connection), but we can't do that with the current event loop. I ended up getting side-tracked into adding mount-namespace to lsof, cause i noticed that feature lacking while working on this Anyways, a new series will come, and I am working on it. Why don't we just start with an eagerly spawned pool? We can always add lazy support later. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Mon, Nov 18, 2013 at 12:36:52AM +0100, Zbigniew Jędrzejewski-Szmek wrote: On Sun, Nov 17, 2013 at 02:57:56PM -0800, Shawn Landden wrote: On Sun, Nov 17, 2013 at 2:28 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Sun, Nov 17, 2013 at 08:46:46PM +0100, Lennart Poettering wrote: On Sat, 16.11.13 13:18, Shawn Landden (sh...@churchofgit.com) wrote: So, what's the status here? Are you sending a new series, or is this one supposed to be merged? Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Sun, Dec 8, 2013 at 7:50 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Mon, Nov 18, 2013 at 12:36:52AM +0100, Zbigniew Jędrzejewski-Szmek wrote: On Sun, Nov 17, 2013 at 02:57:56PM -0800, Shawn Landden wrote: On Sun, Nov 17, 2013 at 2:28 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Sun, Nov 17, 2013 at 08:46:46PM +0100, Lennart Poettering wrote: On Sat, 16.11.13 13:18, Shawn Landden (sh...@churchofgit.com) wrote: So, what's the status here? Are you sending a new series, or is this one supposed to be merged? This one doesn't work. I have have a somewhat-working next patch, but the way epoll_wait works it actually isn't lazy at all, and would require EPOLLET to even do one-spawn-per-connection (global connection), but we can't do that with the current event loop. I ended up getting side-tracked into adding mount-namespace to lsof, cause i noticed that feature lacking while working on this Anyways, a new series will come, and I am working on it. Zbyszek -- --- Shawn Landden +1 360 389 3001 (SMS preferred) ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Sat, 16.11.13 13:18, Shawn Landden (sh...@churchofgit.com) wrote: --- man/systemd.socket.xml| 9 + src/core/dbus-socket.c| 2 ++ src/core/load-fragment-gperf.gperf.m4 | 1 + src/core/socket.c | 7 ++- src/core/socket.h | 1 + 5 files changed, 19 insertions(+), 1 deletion(-) diff --git a/man/systemd.socket.xml b/man/systemd.socket.xml index 4a2189b..fd6ec69 100644 --- a/man/systemd.socket.xml +++ b/man/systemd.socket.xml @@ -530,6 +530,15 @@ /varlistentry varlistentry + termvarnameLazyDistribute=/varname/term +listitemparaTakes an boolean +value. If true, Distribute=n workers will not be spawned +simultameously, but one at a time while connections come it +until n are running. Because of the nature of SO_REUSEPORT workers are +spawned at an exponentially decreasing rate to the number of incoming connections./para/listitem +/varlistentry + +varlistentry Hmm, so I think support for SO_REUSEPORT should just be a special case of the Distribute= logic. i.e. When Distribute= is set but ReusePort is off, then we'd pass the same original listening socket to all services we spawn. If Distribute is set but ReusePort is on then we'd create a new socket when we want to spawn a new instance, and simply bind it to the same sockaddr as the original one. Thinking about this: I have the suspicion that LazyDistribute shouldn't be an option but the normal and only behaviour... Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Sun, Nov 17, 2013 at 08:46:46PM +0100, Lennart Poettering wrote: On Sat, 16.11.13 13:18, Shawn Landden (sh...@churchofgit.com) wrote: varlistentry + termvarnameLazyDistribute=/varname/term +listitemparaTakes an boolean +value. If true, Distribute=n workers will not be spawned +simultameously, but one at a time while connections come it +until n are running. Because of the nature of SO_REUSEPORT workers are +spawned at an exponentially decreasing rate to the number of incoming connections./para/listitem +/varlistentry + +varlistentry Hmm, so I think support for SO_REUSEPORT should just be a special case of the Distribute= logic. i.e. When Distribute= is set but ReusePort is off, then we'd pass the same original listening socket to all services we spawn. If Distribute is set but ReusePort is on then we'd create a new socket when we want to spawn a new instance, and simply bind it to the same sockaddr as the original one. Thinking about this: I have the suspicion that LazyDistribute shouldn't be an option but the normal and only behaviour... Agreed. Also, I think ReusePort=1 should be the default if Distribute is used, and the kernel supports it. I don't think there are any downsides. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Sun, Nov 17, 2013 at 2:28 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Sun, Nov 17, 2013 at 08:46:46PM +0100, Lennart Poettering wrote: On Sat, 16.11.13 13:18, Shawn Landden (sh...@churchofgit.com) wrote: varlistentry + termvarnameLazyDistribute=/varname/term +listitemparaTakes an boolean +value. If true, Distribute=n workers will not be spawned +simultameously, but one at a time while connections come it +until n are running. Because of the nature of SO_REUSEPORT workers are +spawned at an exponentially decreasing rate to the number of incoming connections./para/listitem +/varlistentry + +varlistentry Hmm, so I think support for SO_REUSEPORT should just be a special case of the Distribute= logic. i.e. When Distribute= is set but ReusePort is off, then we'd pass the same original listening socket to all services we spawn. If Distribute is set but ReusePort is on then we'd create a new socket when we want to spawn a new instance, and simply bind it to the same sockaddr as the original one. What types of sockets allow multiple bind()s/accept()s without SO_REUSEPORT? I don't see what the use is having a socket passed to children that can't be used. Thinking about this: I have the suspicion that LazyDistribute shouldn't be an option but the normal and only behaviour... Agreed. OK Also, I think ReusePort=1 should be the default if Distribute is used, and the kernel supports it. I don't think there are any downsides. ReusePort=1 makes it where systemd no longer sits on the port which prevents rogue applications (root for 1024) from stealing the port. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- --- Shawn Landden +1 360 389 3001 (SMS preferred) ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [PATCH 4/4] core: lazy distribute for Distribute pools
On Sun, Nov 17, 2013 at 02:57:56PM -0800, Shawn Landden wrote: On Sun, Nov 17, 2013 at 2:28 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: On Sun, Nov 17, 2013 at 08:46:46PM +0100, Lennart Poettering wrote: On Sat, 16.11.13 13:18, Shawn Landden (sh...@churchofgit.com) wrote: varlistentry + termvarnameLazyDistribute=/varname/term +listitemparaTakes an boolean +value. If true, Distribute=n workers will not be spawned +simultameously, but one at a time while connections come it +until n are running. Because of the nature of SO_REUSEPORT workers are +spawned at an exponentially decreasing rate to the number of incoming connections./para/listitem +/varlistentry + +varlistentry Hmm, so I think support for SO_REUSEPORT should just be a special case of the Distribute= logic. i.e. When Distribute= is set but ReusePort is off, then we'd pass the same original listening socket to all services we spawn. If Distribute is set but ReusePort is on then we'd create a new socket when we want to spawn a new instance, and simply bind it to the same sockaddr as the original one. What types of sockets allow multiple bind()s/accept()s without SO_REUSEPORT? I don't see what the use is having a socket passed to children that can't be used. Thinking about this: I have the suspicion that LazyDistribute shouldn't be an option but the normal and only behaviour... Agreed. OK Also, I think ReusePort=1 should be the default if Distribute is used, and the kernel supports it. I don't think there are any downsides. ReusePort=1 makes it where systemd no longer sits on the port which prevents rogue applications (root for 1024) from stealing the port. To prevent unwanted processes from hijacking a port that has already been bound by a server using SO_REUSEPORT, all of the servers that later bind to that port must have an effective user ID that matches the effective user ID used to perform the first bind on the socket. (https://lwn.net/Articles/542629/). So systemd still sits on the port, and only programs running as root can open the port. There is a corner case, where before a program running as root could not open it, but now it can, even though it might not have full root priviledges. I'd just document this special case. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
