Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > On Wed, 12.07.17 09:53, Florian Weimer (f...@deneb.enyo.de) wrote: > >> * Lennart Poettering: >> >> > On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote: >> > >> >> * Lennart Poettering: >> >> >> >> > This all stems from my experiences with PulseAudio back

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > On Wed, 12.07.17 09:51, Florian Weimer (f...@deneb.enyo.de) wrote: > >> * Lennart Poettering: >> >> > On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote: >> > >> >> * Lennart Poettering: >> >> >> >> > Apparently, this regressed between this version and >>

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

Am 12.07.2017 um 10:23 schrieb Lennart Poettering: On Wed, 12.07.17 09:51, Florian Weimer (f...@deneb.enyo.de) wrote: Fork detection using getpid is not reliable. It gives false negatives in the case of double-forks, where the process can be different but the PID is the same due to reuse.

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote: > >> * Lennart Poettering: >> >> > Apparently, this regressed between this version and >> > glibc-2.24-9.fc25.x86_64 hence. >> >> Yes, I backported the fork cache removal to Fedora 25. There is no >>

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Wed, 12 Jul 2017, Lennart Poettering wrote: On Wed, 12.07.17 09:53, Florian Weimer (f...@deneb.enyo.de) wrote: * Lennart Poettering: Where was this discussed in detail? Do you have any links about the discussions about this? It was on libc-alpha and the glibc bug tracker. Link? Lennart

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote: > * Lennart Poettering: > > > Apparently, this regressed between this version and > > glibc-2.24-9.fc25.x86_64 hence. > > Yes, I backported the fork cache removal to Fedora 25. There is no > longer a good way to main such a

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, Jul 11, 2017 at 7:06 PM, Tomasz Torcz wrote: > On Tue, Jul 11, 2017 at 05:20:10PM +0200, Lennart Poettering wrote: > > On Tue, 11.07.17 16:55, Tomasz Torcz (to...@pipebreaker.pl) wrote: > > > > Forgot to mention: > > > > > > > > $ rpm -qa glibc > > > >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Wed, 12.07.17 09:53, Florian Weimer (f...@deneb.enyo.de) wrote: > * Lennart Poettering: > > > On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote: > > > >> * Lennart Poettering: > >> > >> > This all stems from my experiences with PulseAudio back in the day: > >> > People do not

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Wed, 12.07.17 09:51, Florian Weimer (f...@deneb.enyo.de) wrote: > * Lennart Poettering: > > > On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote: > > > >> * Lennart Poettering: > >> > >> > Apparently, this regressed between this version and > >> > glibc-2.24-9.fc25.x86_64

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote: > >> * Lennart Poettering: >> >> > This all stems from my experiences with PulseAudio back in the day: >> > People do not grok the effect of fork(): it only duplicates the >> > invoking thread, not any

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote: > >> * Lennart Poettering: >> >> > Apparently, this regressed between this version and >> > glibc-2.24-9.fc25.x86_64 hence. >> >> Yes, I backported the fork cache removal to Fedora 25. There is no >>

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 21:26, Florian Weimer (f...@deneb.enyo.de) wrote: > * Lennart Poettering: > > > Apparently, this regressed between this version and > > glibc-2.24-9.fc25.x86_64 hence. > > Yes, I backported the fork cache removal to Fedora 25. There is no > longer a good way to main such a

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 21:24, Florian Weimer (f...@deneb.enyo.de) wrote: > * Lennart Poettering: > > > This all stems from my experiences with PulseAudio back in the day: > > People do not grok the effect of fork(): it only duplicates the > > invoking thread, not any other threads of the process,

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > Apparently, this regressed between this version and > glibc-2.24-9.fc25.x86_64 hence. Yes, I backported the fork cache removal to Fedora 25. There is no longer a good way to main such a cache in userspace because glibc cannot intercept anymore all the ways that can

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Lennart Poettering: > This all stems from my experiences with PulseAudio back in the day: > People do not grok the effect of fork(): it only duplicates the > invoking thread, not any other threads of the process, moreover all > data structures are copied as they are, and that's a time bomb

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

* Michael Chapman: > It's a pity glibc doesn't provide an equivalent for pthread_atfork() > outside of the pthread library. Having a notification that a fork has just > occurred would allow us to do the PID caching ourselves. In fact, it does, as a public symbol: __register_atfork. It's just

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, Jul 11, 2017 at 05:20:10PM +0200, Lennart Poettering wrote: > On Tue, 11.07.17 16:55, Tomasz Torcz (to...@pipebreaker.pl) wrote: > > > Forgot to mention: > > > > > > $ rpm -qa glibc > > > glibc-2.24-4.fc25.x86_64 > > > > > > Apparently, this regressed between this version and > > >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 16:55, Tomasz Torcz (to...@pipebreaker.pl) wrote: > On Tue, Jul 11, 2017 at 04:10:38PM +0200, Lennart Poettering wrote: > > On Tue, 11.07.17 16:07, Lennart Poettering (lenn...@poettering.net) wrote: > > > Hmm, so I run a slightly older glibc, as I haven#t updated my system > > >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, Jul 11, 2017 at 04:10:38PM +0200, Lennart Poettering wrote: > On Tue, 11.07.17 16:07, Lennart Poettering (lenn...@poettering.net) wrote: > > Hmm, so I run a slightly older glibc, as I haven#t updated my system > > in a while: > > > > $ strace -c journalctl --since -1hour 2>&1 >/dev/null |

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 16:07, Lennart Poettering (lenn...@poettering.net) wrote: > Hmm, so I run a slightly older glibc, as I haven#t updated my system > in a while: > > $ strace -c journalctl --since -1hour 2>&1 >/dev/null | head -10 > % time seconds usecs/call callserrors syscall >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 21:59, Michael Chapman (m...@very.puzzling.org) wrote: > On Tue, 11 Jul 2017, Lennart Poettering wrote: > > On Tue, 11.07.17 12:55, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: > > > > > On Tue, 2017-07-11 at 09:35 +0200, Lennart Poettering wrote: > > > > Normally it's dead

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, Jul 11, 2017 at 09:59:45PM +1000, Michael Chapman wrote: > On Tue, 11 Jul 2017, Lennart Poettering wrote: > >On Tue, 11.07.17 12:55, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: > > > >>On Tue, 2017-07-11 at 09:35 +0200, Lennart Poettering wrote: > >>>Normally it's dead cheap to check

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11 Jul 2017, Lennart Poettering wrote: On Tue, 11.07.17 12:55, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: On Tue, 2017-07-11 at 09:35 +0200, Lennart Poettering wrote: Normally it's dead cheap to check that, it's just reading and comparing one memory location. It's a pitty that this

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

Resend with correct list address On Tue, 2017-07-11 at 12:00 +0200, Lennart Poettering wrote: > On Tue, 11.07.17 12:55, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: > > Are you sure about those "Debian only" and "will be 'fixed'" parts? The > > Debian patch seems to be a cherry pick from upstream

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 2017-07-11 at 09:35 +0200, Lennart Poettering wrote: > Normally it's dead cheap to check that, it's just reading and > comparing one memory location. It's a pitty that this isn't the case > currently on Debian, but as it appears this is an oversight on their > side, and I am sure it will

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Tue, 11.07.17 12:55, Uoti Urpala (uoti.urp...@pp1.inet.fi) wrote: > On Tue, 2017-07-11 at 09:35 +0200, Lennart Poettering wrote: > > Normally it's dead cheap to check that, it's just reading and > > comparing one memory location. It's a pitty that this isn't the case > > currently on Debian,

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Mon, 10.07.17 14:04, vcap...@pengaru.com (vcap...@pengaru.com) wrote: > > > Except there's always a risk of these things regressing to normal > > > syscalls, > > > and one has to weigh the utility against that. It's unclear to me what > > > significant utility having the sd-journal API

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Sat, Jul 08, 2017 at 03:49:11AM +, Zbigniew Jędrzejewski-Szmek wrote: > On Fri, Jul 07, 2017 at 03:54:09PM -0700, vcap...@pengaru.com wrote: > > On Fri, Jul 07, 2017 at 10:34:22PM +, Zbigniew Jędrzejewski-Szmek wrote: > > > On Fri, Jul 07, 2017 at 02:35:16PM -0700, vcap...@pengaru.com

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Mon, 10 Jul 2017, Lennart Poettering wrote: On Mon, 10.07.17 21:51, Michael Chapman (m...@very.puzzling.org) wrote: This all stems from my experiences with PulseAudio back in the day: People do not grok the effect of fork(): it only duplicates the invoking thread, not any other threads of

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Mon, 10.07.17 21:51, Michael Chapman (m...@very.puzzling.org) wrote: > > This all stems from my experiences with PulseAudio back in the day: > > People do not grok the effect of fork(): it only duplicates the > > invoking thread, not any other threads of the process, moreover all > > data

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Mon, 10 Jul 2017, Lennart Poettering wrote: On Sat, 08.07.17 16:24, Michael Chapman (m...@very.puzzling.org) wrote: On Sat, 8 Jul 2017, vcap...@pengaru.com wrote: In doing some casual journalctl profiling and stracing, it became apparent that `journalctl -b --no-pager` runs across a

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Sat, 08.07.17 16:24, Michael Chapman (m...@very.puzzling.org) wrote: > On Sat, 8 Jul 2017, vcap...@pengaru.com wrote: > > In doing some casual journalctl profiling and stracing, it became apparent > > that `journalctl -b --no-pager` runs across a significant quantity of logs, > > ~10% of the

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Fri, 07.07.17 14:35, vcap...@pengaru.com (vcap...@pengaru.com) wrote: > On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote: > > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote: > > > Back when that commit was made, didn't glibc cache the getpid() result in > >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Sat, 8 Jul 2017, vcap...@pengaru.com wrote: In doing some casual journalctl profiling and stracing, it became apparent that `journalctl -b --no-pager` runs across a significant quantity of logs, ~10% of the time was thrown away on getpid() calls due to commmit a65f06b. As-is: # time

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Fri, Jul 07, 2017 at 03:54:09PM -0700, vcap...@pengaru.com wrote: > On Fri, Jul 07, 2017 at 10:34:22PM +, Zbigniew Jędrzejewski-Szmek wrote: > > On Fri, Jul 07, 2017 at 02:35:16PM -0700, vcap...@pengaru.com wrote: > > > On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote: >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Fri, Jul 07, 2017 at 10:34:22PM +, Zbigniew Jędrzejewski-Szmek wrote: > On Fri, Jul 07, 2017 at 02:35:16PM -0700, vcap...@pengaru.com wrote: > > On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote: > > > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote: > > >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Fri, Jul 07, 2017 at 02:35:16PM -0700, vcap...@pengaru.com wrote: > On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote: > > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote: > > > Back when that commit was made, didn't glibc cache the getpid() result in > > >

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote: > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote: > > Back when that commit was made, didn't glibc cache the getpid() result in > > userspace? That would explain why it was not noticed. > > Hmm, this crossed my

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

On Fri, Jul 07, 2017 at 01:49:54PM -0700, vcap...@pengaru.com wrote: > On Fri, Jul 07, 2017 at 08:37:08PM +, Mantas Mikulėnas wrote: > > Back when that commit was made, didn't glibc cache the getpid() result in > > userspace? That would explain why it was not noticed. > > Hmm, this crossed my

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

Back when that commit was made, didn't glibc cache the getpid() result in userspace? That would explain why it was not noticed. On Fri, Jul 7, 2017, 23:18 wrote: > In doing some casual journalctl profiling and stracing, it became apparent > that `journalctl -b --no-pager`

Re: [systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

Hmm, this crossed my mind, and come to think of it I did a dist-upgrade from Debian jessie to stretch overnight machine and haven't rebooted. Perhaps the vdso isn't working and the costly getpid() is a red herring, will reboot and retest to confirm. On Fri, Jul 07, 2017 at 08:37:08PM +,

[systemd-devel] Significant performance loss caused by commit a65f06b: journal: return -ECHILD after a fork

In doing some casual journalctl profiling and stracing, it became apparent that `journalctl -b --no-pager` runs across a significant quantity of logs, ~10% of the time was thrown away on getpid() calls due to commmit a65f06b. As-is: # time ./journalctl -b --no-pager > /dev/null real