Re: mod_http2 and Frequent wake-ups for mpm_event

With stock 2.4.25 + patch i'm getting this one again: (gdb) bt #0 0x00521dcd in h2_stream_out_prepare () #1 0x7fc1a2feca80 in ?? () #2 0x7fc1a2feca8c in ?? () #3 0x7fc1a2feca90 in ?? () #4 0x7fc1a057c0a0 in ?? () #5 0x7fc1a057cdd8 in ?? () #6 0x7fc1a2fecac0

Re: mod_http2 and Frequent wake-ups for mpm_event

I'm now testing stock 2.4.25 + patch. May this configure option have an influence? --enable-nonportable-atomics=yes Greets, Stefan Am 19.01.2017 um 15:35 schrieb Yann Ylavic: > Hi, > > On Thu, Jan 19, 2017 at 3:00 PM, Stefan Priebe - Profihost AG > wrote: >> >> @Yann:

Re: mod_http2 and Frequent wake-ups for mpm_event

This one is with mod_http2 v1.8.8 + http2 patch - no mpm patch. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f1aa4c45d63 in apr_pool_cleanup_kill () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 (gdb) bt #0 0x7f1aa4c45d63 in apr_pool_cleanup_kill () from

Re: mod_http2 and Frequent wake-ups for mpm_event

Am 19.01.2017 um 15:05 schrieb Stefan Eissing: > I prefer on top of v1.8.8, but it's your installation. Should also work on > older versions. got this one: (gdb) bt #0 0x7f4c424ec014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f4c4297f036 in ?? () from

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi, On Thu, Jan 19, 2017 at 3:00 PM, Stefan Priebe - Profihost AG wrote: > > @Yann: > should i use V7 or V6? I'd prefer you'd use none (such that we can verify the patch with stock 2.4.25, modulo mod_http2), but if it's easier for you to reproduce with an event's patch,

Re: mod_http2 and Frequent wake-ups for mpm_event

I prefer on top of v1.8.8, but it's your installation. Should also work on older versions. > Am 19.01.2017 um 15:00 schrieb Stefan Priebe - Profihost AG > : > > Hi, > > on top of v1.8.8? > > @Yann: > should i use V7 or V6? > > Stefan > > Am 19.01.2017 um 14:56

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi, on top of v1.8.8? @Yann: should i use V7 or V6? Stefan Am 19.01.2017 um 14:56 schrieb Stefan Eissing: > Stefan, > > could you check if the attached patch mitigates the problem at least to some > extend? Was suggested to me by Yann, all faults are mine. Thanks! > > Cheers, Stefan > > >

Re: mod_http2 and Frequent wake-ups for mpm_event

Stefan, could you check if the attached patch mitigates the problem at least to some extend? Was suggested to me by Yann, all faults are mine. Thanks! Cheers, Stefan h2_session_clean.diff Description: Binary data > Am 19.01.2017 um 12:45 schrieb Stefan Priebe - Profihost AG >

Re: mod_http2 and Frequent wake-ups for mpm_event

Am 19.01.2017 um 11:56 schrieb Stefan Eissing: > Stefan, > > yes, that is a known one that was addressed in v1.8.7. But I think it is > related to the others. There is some mistake in my thinking about session > pool cleanup that is more often uncovered by Yann's recent patches. I'll need >

Re: mod_http2 and Frequent wake-ups for mpm_event

Stefan, yes, that is a known one that was addressed in v1.8.7. But I think it is related to the others. There is some mistake in my thinking about session pool cleanup that is more often uncovered by Yann's recent patches. I'll need some deep dive into this one. For you, that means v1.8.8 is

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi, with V7 and mod_http2 core i'm always seeing exactly THIS trace and instead of every few minutes just once an hour: [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k

Re: mod_http2 and Frequent wake-ups for mpm_event

And another one: #0 0x7f7d81ba1f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 (gdb) bt #0 0x7f7d81ba1f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 #1 0x0052a8bc in h2_util_bb_avail () #2 0x7112ca10 in ?? ()

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi, Am 19.01.2017 um 09:11 schrieb Yann Ylavic: > On Thu, Jan 19, 2017 at 9:05 AM, Stefan Priebe - Profihost AG > wrote: >> With a vanilla apache 2.4.25 i got this one: >> >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal

Re: mod_http2 and Frequent wake-ups for mpm_event

On Thu, Jan 19, 2017 at 9:05 AM, Stefan Priebe - Profihost AG wrote: > With a vanilla apache 2.4.25 i got this one: > > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7fe2bb7b8add in

Re: mod_http2 and Frequent wake-ups for mpm_event

With a vanilla apache 2.4.25 i got this one: Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7fe2bb7b8add in read () from /lib/x86_64-linux-gnu/libpthread.so.0 (gdb) bt #0 0x7fe2bb7b8add in read () from

Re: mod_http2 and Frequent wake-ups for mpm_event

Am 19.01.2017 um 08:26 schrieb Stefan Eissing: > Will look into this. With which patch do you have it most frequently? It happens pretty fast while using: Apache 2.4.25 + mpm_event V6 + mod_http2 V1.8.8 Greets, Stefan > Cheers, Stefan > >> Am 19.01.2017 um 08:22 schrieb Stefan Priebe -

Re: mod_http2 and Frequent wake-ups for mpm_event

Will look into this. With which patch do you have it most frequently? Cheers, Stefan > Am 19.01.2017 um 08:22 schrieb Stefan Priebe - Profihost AG > : > > Hi Stefan, > > Apache 2.4.25 + mpm_event V7 and core mod_http2: > Core was generated by

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi Stefan, Apache 2.4.25 + mpm_event V7 and core mod_http2: Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00521d98 in h2_stream_out_prepare () (gdb) bt #0 0x00521d98 in h2_stream_out_prepare () #1

Re: mod_http2 and Frequent wake-ups for mpm_event

Dear Stefan, dear Yann, a longer test run shows it's also crashing without any mpm_event patch at all. So I'm really sorry. It just starts crashing faster with the mpm_event patches. I'll now retest regarding mod_http2 core, plain apache 2.4.25 and mod_http v1.8.8 may be it's a regression in

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 10:49 PM, Stefan Priebe - Profihost AG wrote: > > v5 does not apply to 2.4.25. If you can send me a v5 version that > applies to 2.4.25 i'll test. Here it is (slightly modified to fix possible deadlocks, per r1762742 and r1774538). Thanks! Index:

Re: mod_http2 and Frequent wake-ups for mpm_event

Am 18.01.2017 um 22:53 schrieb Yann Ylavic: > On Wed, Jan 18, 2017 at 10:50 PM, Yann Ylavic wrote: >> On Wed, Jan 18, 2017 at 10:44 PM, Yann Ylavic wrote: >>> On Wed, Jan 18, 2017 at 10:23 PM, Stefan Priebe - Profihost AG >>>

Re: mod_http2 and Frequent wake-ups for mpm_event

Am 18.01.2017 um 22:50 schrieb Yann Ylavic: > On Wed, Jan 18, 2017 at 10:44 PM, Yann Ylavic wrote: >> On Wed, Jan 18, 2017 at 10:23 PM, Stefan Priebe - Profihost AG >> wrote: >>> >> >> Also, do you confirm that: >> 1. it didn't crash with v5 +

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 10:50 PM, Yann Ylavic wrote: > On Wed, Jan 18, 2017 at 10:44 PM, Yann Ylavic wrote: >> On Wed, Jan 18, 2017 at 10:23 PM, Stefan Priebe - Profihost AG >> wrote: >>> >> >> Also, do you confirm that: >> 1.

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 10:44 PM, Yann Ylavic wrote: > On Wed, Jan 18, 2017 at 10:23 PM, Stefan Priebe - Profihost AG > wrote: >> > > Also, do you confirm that: > 1. it didn't crash with v5 + original (2.4.25's) mod_http2? Hmm, v5 was against 2.4.23,

Re: mod_http2 and Frequent wake-ups for mpm_event

Am 18.01.2017 um 22:44 schrieb Yann Ylavic: > On Wed, Jan 18, 2017 at 10:23 PM, Stefan Priebe - Profihost AG > wrote: >> >> sadly it does not solve the issue. > > OK, thanks, back to the paper. > > Any [error] log maybe? What kind of? server-error.log just says [Wed Jan

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 10:42 PM, Stefan Priebe - Profihost AG wrote: > I also tried to remove rev 1762701 from the v6 patchset. But it does not > match. That's more or less what v7 does..

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 10:23 PM, Stefan Priebe - Profihost AG wrote: > > sadly it does not solve the issue. OK, thanks, back to the paper. Any [error] log maybe? Also, do you confirm that: 1. it didn't crash with v5 + original (2.4.25's) mod_http2? 2. it started

Re: mod_http2 and Frequent wake-ups for mpm_event

I also tried to remove rev 1762701 from the v6 patchset. But it does not match. Stefan Am 18.01.2017 um 22:23 schrieb Stefan Priebe - Profihost AG: > Hi Yann, > > sadly it does not solve the issue. > > Trace looks still the same: > Core was generated by `/usr/local/apache2/bin/httpd -k start'.

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi Yann, sadly it does not solve the issue. Trace looks still the same: Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7fe0228a9014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt #0 0x7fe0228a9014

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 4:38 PM, Stefan Priebe - Profihost AG wrote: > Ok will wait whether you provide a fix or I should revert that part. Patch updated in PR 57399. Thanks, Yann.

Re: mod_http2 and Frequent wake-ups for mpm_event

Ok will wait whether you provide a fix or I should revert that part. Stefan Excuse my typo sent from my mobile phone. > Am 18.01.2017 um 16:32 schrieb Yann Ylavic : > > On Wed, Jan 18, 2017 at 4:17 PM, Stefan Priebe - Profihost AG > wrote: >> >>

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 4:17 PM, Stefan Priebe - Profihost AG wrote: > > is it enought to revert that commit? I think so (worth a try at least). I'm about to commit sort of this revert to trunk, with a possible explanation... Thanks Stefan for all these follow ups.

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi Yann, is it enought to revert that commit? http://svn.apache.org/r1762701 Stefan Am 18.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG: > Hi, > > site is mostly used with http2. So it may be totally unrelated to > mod_http2. Sorry for the noise than. I just thought it by digging >

Re: mod_http2 and Frequent wake-ups for mpm_event

> Am 18.01.2017 um 13:39 schrieb Stefan Eissing : > > There is currently no cleanup register for shutting down that way. If the > main connection pool gets cleaned before the special meta is destroyed, these > problem might happen. Not sure. I am babbling. Of

Re: mod_http2 and Frequent wake-ups for mpm_event

Yann, all these traces point to the connection shutdown case for mod_http2 which is triggered by writing a special meta bucket and cleaning up on its destruction. That's the trick to make sure that all data has been sent and similar to the EOR bucket. There is currently no cleanup register

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi, and a new one just some seconds ago: (gdb) bt #0 0x7f32a3747014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f32a3bda036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x7f32a3bda46f in apr_hash_set () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #3

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi, site is mostly used with http2. So it may be totally unrelated to mod_http2. Sorry for the noise than. I just thought it by digging through the traces. Stefan Am 18.01.2017 um 12:34 schrieb Yann Ylavic: > Hi Stefan, > > On Wed, Jan 18, 2017 at 11:33 AM, Stefan Priebe - Profihost AG >

Re: mod_http2 and Frequent wake-ups for mpm_event

Hi Stefan, On Wed, Jan 18, 2017 at 11:33 AM, Stefan Priebe - Profihost AG wrote: > > after applying the event patch to 2.4.25 from > https://bz.apache.org/bugzilla/show_bug.cgi?id=57399. > > I'm seeing segfaults in the mod_http2 code. I already bumped mod_http2 > to

Re: mod_http2 and Frequent wake-ups for mpm_event

On Wed, Jan 18, 2017 at 12:09 PM, Stefan Eissing wrote: > Can't have it all, Stefan! :) > > Seriously, will have a look at this. Thanks for the traces. I possibly messed up with event's locking in latest patch from PR 57399. Actually it removes locks around

Re: mod_http2 and Frequent wake-ups for mpm_event

Can't have it all, Stefan! Seriously, will have a look at this. Thanks for the traces. -Stefan > Am 18.01.2017 um 11:33 schrieb Stefan Priebe - Profihost AG > : > > Hi Stefan, > Hi Yann, > > after applying the event patch to 2.4.25 from >

<    1   2