Re: [systemd-devel] journal and /run/systemd/journal/syslog
On Fri, Jul 12, 2013 at 1:08 AM, Lennart Poettering lenn...@poettering.net wrote: On Wed, 10.07.13 12:30, Umut Tezduyar (u...@tezduyar.com) wrote: We though about just bumping this globally via sysctl, but we feared that might not sit well with some folks, as we shouldn't change a global setting just because one user of it would benefit, especially given that we don't know the effect this might have on others... I want to go this route and I think it is not possible at the moment due to undetermined start order between syslog.socket and systemd-sysctl.service. Can we change that? Hmm, are you saying that changing the sysctl value too late will have no effect on the sockets that already exist at that time? That is correct. Limit is applied to the socket when it is created (sk-sk_max_ack_backlog = net-unx.sysctl_max_dgram_qlen;). You could do this on your system: # mkdir /etc/systemd/system/syslog.socket.d/ # echo [Unit] After=systemd-sysctl.service /etc/systemd/system/syslog.socket.d/after-sysctl.conf Then, make the sysctl setting, and things should work? Lennart I believe using dropin services should work as well as handling this in linuxrc or ramfs. I was more after an upstream solution then per user solution but since we are all on the same page regarding the kernel fix for per socket limit, I guess I need to solve it in my application. Thanks for the help. Umut -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journal and /run/systemd/journal/syslog
On Wed, 10.07.13 12:30, Umut Tezduyar (u...@tezduyar.com) wrote: We though about just bumping this globally via sysctl, but we feared that might not sit well with some folks, as we shouldn't change a global setting just because one user of it would benefit, especially given that we don't know the effect this might have on others... I want to go this route and I think it is not possible at the moment due to undetermined start order between syslog.socket and systemd-sysctl.service. Can we change that? Hmm, are you saying that changing the sysctl value too late will have no effect on the sockets that already exist at that time? You could do this on your system: # mkdir /etc/systemd/system/syslog.socket.d/ # echo [Unit] After=systemd-sysctl.service /etc/systemd/system/syslog.socket.d/after-sysctl.conf Then, make the sysctl setting, and things should work? Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journal and /run/systemd/journal/syslog
On Tue, Jul 9, 2013 at 1:58 AM, Lennart Poettering lenn...@poettering.net wrote: On Sun, 07.07.13 19:18, Umut Tezduyar (u...@tezduyar.com) wrote: Hi, I have noticed that if journal fails to send messages to /run/systemd/journal/syslog, it increments a counter and every 30 seconds it prints out a message on journal informing that forwarding x many messages has failed. If journal storage is set to none, there is no way of being informed about the missing messages. Should we not try to send a message to /run/systemd/journal/syslog if/when it has space again? Yes, sounds sensible. Added to the TODO list. syslog.socket sets ReceiveBuffer=8M. My understanding is intention of setting this value is specifying the maximum size of one datagram message that is allowed to send. socket.c is setting the SO_RCVBUF on the socket. According to man unix 7, SO_RCVBUF has no effect on the datagram but SO_SNDBUF does. Am I missing something? Oh, hm, this sucks. I wasn't aware of this. I always assumed SO_RCVBUF and SO_SNDBUF would both alter the same buffer, just from differente sides. Number of messages that can be queued on /run/systemd/journal/syslog is controlled by /proc/sys/net/unix/max_dgram_qlen and this value is by default 10 in most distributions. What this means is, after journal forwards 11 messages to the socket, it will not be able to do so until external logging daemon comes up and clears up /run/systemd/journal/syslog. If logging daemon is starting after basic.target, it is pretty certain that it will not be able to receive all the early boot messages. Increasing /proc/sys/net/unix/max_dgram_qlen is an option but is increasing a system wide limit right thing to do? Yeah, this is a big issue. For a long time there has been an item on the kernel wishlist to get an ioctl for setting the qlen per-socket. The code for that should actually be simple. Luckily kernel doesn't allocate a buffer for the socket (kernel:net/unix/af_unix.c) according to the value meaning setting it to 10 or 1000 doesn't matter in terms of pre-allocating memory. We though about just bumping this globally via sysctl, but we feared that might not sit well with some folks, as we shouldn't change a global setting just because one user of it would benefit, especially given that we don't know the effect this might have on others... I want to go this route and I think it is not possible at the moment due to undetermined start order between syslog.socket and systemd-sysctl.service. Can we change that? The better approach really appeared to us to be fixing the kernel properly instead, and get that new ioctl. Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journal and /run/systemd/journal/syslog
On Wed, Jul 10, 2013 at 12:30 PM, Umut Tezduyar u...@tezduyar.com wrote: On Tue, Jul 9, 2013 at 1:58 AM, Lennart Poettering lenn...@poettering.net wrote: We though about just bumping this globally via sysctl, but we feared that might not sit well with some folks, as we shouldn't change a global setting just because one user of it would benefit, especially given that we don't know the effect this might have on others... I want to go this route and I think it is not possible at the moment due to undetermined start order between syslog.socket and systemd-sysctl.service. Can we change that? It really makes more sense to add a socket option to the kernel to allow a privileged process to bump the limit for the one socket setsockopt() is called, and not change and rely on a global system-wide value affecting the entire system. Kay ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journal and /run/systemd/journal/syslog
On Wed, Jul 10, 2013 at 12:55 PM, Kay Sievers k...@vrfy.org wrote: On Wed, Jul 10, 2013 at 12:30 PM, Umut Tezduyar u...@tezduyar.com wrote: On Tue, Jul 9, 2013 at 1:58 AM, Lennart Poettering lenn...@poettering.net wrote: We though about just bumping this globally via sysctl, but we feared that might not sit well with some folks, as we shouldn't change a global setting just because one user of it would benefit, especially given that we don't know the effect this might have on others... I want to go this route and I think it is not possible at the moment due to undetermined start order between syslog.socket and systemd-sysctl.service. Can we change that? It really makes more sense to add a socket option to the kernel to allow a privileged process to bump the limit for the one socket setsockopt() is called, and not change and rely on a global system-wide value affecting the entire system. Kay We are totally on the same page on having a per-socket limit but I am guessing this change is not going to happen tomorrow. Even if it did, adopting the new kernel will also take some time. It would be nice to have a solution meanwhile. Umut ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] journal and /run/systemd/journal/syslog
On Sun, 07.07.13 19:18, Umut Tezduyar (u...@tezduyar.com) wrote: Hi, I have noticed that if journal fails to send messages to /run/systemd/journal/syslog, it increments a counter and every 30 seconds it prints out a message on journal informing that forwarding x many messages has failed. If journal storage is set to none, there is no way of being informed about the missing messages. Should we not try to send a message to /run/systemd/journal/syslog if/when it has space again? Yes, sounds sensible. Added to the TODO list. syslog.socket sets ReceiveBuffer=8M. My understanding is intention of setting this value is specifying the maximum size of one datagram message that is allowed to send. socket.c is setting the SO_RCVBUF on the socket. According to man unix 7, SO_RCVBUF has no effect on the datagram but SO_SNDBUF does. Am I missing something? Oh, hm, this sucks. I wasn't aware of this. I always assumed SO_RCVBUF and SO_SNDBUF would both alter the same buffer, just from differente sides. Number of messages that can be queued on /run/systemd/journal/syslog is controlled by /proc/sys/net/unix/max_dgram_qlen and this value is by default 10 in most distributions. What this means is, after journal forwards 11 messages to the socket, it will not be able to do so until external logging daemon comes up and clears up /run/systemd/journal/syslog. If logging daemon is starting after basic.target, it is pretty certain that it will not be able to receive all the early boot messages. Increasing /proc/sys/net/unix/max_dgram_qlen is an option but is increasing a system wide limit right thing to do? Yeah, this is a big issue. For a long time there has been an item on the kernel wishlist to get an ioctl for setting the qlen per-socket. The code for that should actually be simple. We though about just bumping this globally via sysctl, but we feared that might not sit well with some folks, as we shouldn't change a global setting just because one user of it would benefit, especially given that we don't know the effect this might have on others... The better approach really appeared to us to be fixing the kernel properly instead, and get that new ioctl. Lennart -- Lennart Poettering - Red Hat, Inc. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] journal and /run/systemd/journal/syslog
Hi, I have noticed that if journal fails to send messages to /run/systemd/journal/syslog, it increments a counter and every 30 seconds it prints out a message on journal informing that forwarding x many messages has failed. If journal storage is set to none, there is no way of being informed about the missing messages. Should we not try to send a message to /run/systemd/journal/syslog if/when it has space again? syslog.socket sets ReceiveBuffer=8M. My understanding is intention of setting this value is specifying the maximum size of one datagram message that is allowed to send. socket.c is setting the SO_RCVBUF on the socket. According to man unix 7, SO_RCVBUF has no effect on the datagram but SO_SNDBUF does. Am I missing something? Number of messages that can be queued on /run/systemd/journal/syslog is controlled by /proc/sys/net/unix/max_dgram_qlen and this value is by default 10 in most distributions. What this means is, after journal forwards 11 messages to the socket, it will not be able to do so until external logging daemon comes up and clears up /run/systemd/journal/syslog. If logging daemon is starting after basic.target, it is pretty certain that it will not be able to receive all the early boot messages. Increasing /proc/sys/net/unix/max_dgram_qlen is an option but is increasing a system wide limit right thing to do? Thanks ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
