Re: mod_http2 and Frequent wake-ups for mpm_event
> Am 20.02.2017 um 10:36 schrieb Stefan Priebe : > > Hi, > > still not a single segfault with mod_http2 1.9.0 - good work! Thanks! Our pleasure. Enjoy it until I break something in the next releases. > @Yann > Could you please tell me whether i should drop of your additional patches? > > Greets, > Stefan > > Am 09.02.2017 um 11:55 schrieb Stefan Priebe - Profihost AG: >> Hi Stefan, >> >> Am 09.02.2017 um 11:47 schrieb Stefan Eissing: >>> Stefan, >>> >>> at this point and after several efforts to write the right patch, I reached >>> the conclusion that I need to rethink the pool hierarchy and connection >>> shutdown strategy in mod_http2. The current, organically grown >>> implementation needs a refactor with the knowledge we have made so far. >> >> OK - thanks for your help and hard work. >> >>> So, a fix will not come today or tomorrow. But not too far away either. >>> From your comments I assume that these crashed happen not too frequently. >>> Hope this allows you to live with the current state for a while. >> >> Sure and yes it does not happen very often. >> >>> Btw. have the status 500 lines disappeared with the latest release? That >>> was one point still open on my list... >> >> Yes sorry i missed to report back. It's working fine now. >> >>> >>> Hope to give you something better to verify in your environment soon. >> >> Just tell me i'm ready to test. >> >> Greets, >> Stefan >> >>> Cheers, >>> >>> Stefan >>> Am 07.02.2017 um 20:56 schrieb Stefan Priebe - Profihost AG : Hi, got this one today with both patches applied: Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x0, allocator=0x7f350405e030) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x0, allocator=0x7f350405e030) at memory/unix/apr_pools.c:381 #1 apr_pool_clear (pool=0x7f3504060468) at memory/unix/apr_pools.c:793 #2 0x55d11256d688 in ap_push_pool (queue_info=0x7f35040604f8, pool_to_recycle=0x) at fdqueue.c:234 #3 0x55d11256899a in process_lingering_close (cs=0x7f3504060748, pfd=0x55d1143fd7f8) at event.c:1513 #4 0x55d11256c4d0 in listener_thread (thd=0x7f35040604f8, dummy=0x547f569acd4a6) at event.c:1837 #5 0x7f351757c0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f35172b162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Am 06.02.2017 um 22:24 schrieb Stefan Priebe - Profihost AG: > Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: >> Hi Stefan, >> >> this one does not apply on top of yann's >> mpm_event_listener_wakeup_bug57399_V7.patch patch. > > i've now used his patch to mpm and yours for mod_http2. > > Stefan > >> >> Stefan >> Am 06.02.2017 um 15:34 schrieb Stefan Eissing: >>> Yes, that was mixing the order. The attached v4 compiles and runs the >>> h2 tests for me without errors. >>> >>> >>> >>> >>> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing wrote: > Currently running some tests. Have crashes on the original patch in > my test suite. Fixed one, hunting for the next... I think it comes from my change that creates slave connections from master->pool (instead of mplx's), because now slave's pool is already destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() is called (hence the crash). I restored your original code in this new (attached) patch. @s.priebe, would you test this one please? >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>> >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>> Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
On Mon, Feb 20, 2017 at 10:36 AM, Stefan Priebe wrote: > > @Yann > Could you please tell me whether i should drop of your additional patches? Sorry Stefan, somehow I missed you message. I'll reply (inline) on the other thread.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, still not a single segfault with mod_http2 1.9.0 - good work! @Yann Could you please tell me whether i should drop of your additional patches? Greets, Stefan Am 09.02.2017 um 11:55 schrieb Stefan Priebe - Profihost AG: Hi Stefan, Am 09.02.2017 um 11:47 schrieb Stefan Eissing: Stefan, at this point and after several efforts to write the right patch, I reached the conclusion that I need to rethink the pool hierarchy and connection shutdown strategy in mod_http2. The current, organically grown implementation needs a refactor with the knowledge we have made so far. OK - thanks for your help and hard work. So, a fix will not come today or tomorrow. But not too far away either. From your comments I assume that these crashed happen not too frequently. Hope this allows you to live with the current state for a while. Sure and yes it does not happen very often. Btw. have the status 500 lines disappeared with the latest release? That was one point still open on my list... Yes sorry i missed to report back. It's working fine now. Hope to give you something better to verify in your environment soon. Just tell me i'm ready to test. Greets, Stefan Cheers, Stefan Am 07.02.2017 um 20:56 schrieb Stefan Priebe - Profihost AG : Hi, got this one today with both patches applied: Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x0, allocator=0x7f350405e030) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x0, allocator=0x7f350405e030) at memory/unix/apr_pools.c:381 #1 apr_pool_clear (pool=0x7f3504060468) at memory/unix/apr_pools.c:793 #2 0x55d11256d688 in ap_push_pool (queue_info=0x7f35040604f8, pool_to_recycle=0x) at fdqueue.c:234 #3 0x55d11256899a in process_lingering_close (cs=0x7f3504060748, pfd=0x55d1143fd7f8) at event.c:1513 #4 0x55d11256c4d0 in listener_thread (thd=0x7f35040604f8, dummy=0x547f569acd4a6) at event.c:1837 #5 0x7f351757c0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f35172b162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Am 06.02.2017 um 22:24 schrieb Stefan Priebe - Profihost AG: Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: Hi Stefan, this one does not apply on top of yann's mpm_event_listener_wakeup_bug57399_V7.patch patch. i've now used his patch to mpm and yours for mod_http2. Stefan Stefan Am 06.02.2017 um 15:34 schrieb Stefan Eissing: Yes, that was mixing the order. The attached v4 compiles and runs the h2 tests for me without errors. Am 06.02.2017 um 14:43 schrieb Yann Ylavic : On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing wrote: Currently running some tests. Have crashes on the original patch in my test suite. Fixed one, hunting for the next... I think it comes from my change that creates slave connections from master->pool (instead of mplx's), because now slave's pool is already destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() is called (hence the crash). I restored your original code in this new (attached) patch. @s.priebe, would you test this one please? Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan, Am 09.02.2017 um 11:47 schrieb Stefan Eissing: > Stefan, > > at this point and after several efforts to write the right patch, I reached > the conclusion that I need to rethink the pool hierarchy and connection > shutdown strategy in mod_http2. The current, organically grown implementation > needs a refactor with the knowledge we have made so far. OK - thanks for your help and hard work. > So, a fix will not come today or tomorrow. But not too far away either. From > your comments I assume that these crashed happen not too frequently. Hope > this allows you to live with the current state for a while. Sure and yes it does not happen very often. > Btw. have the status 500 lines disappeared with the latest release? That was > one point still open on my list... Yes sorry i missed to report back. It's working fine now. > > Hope to give you something better to verify in your environment soon. Just tell me i'm ready to test. Greets, Stefan > Cheers, > > Stefan > >> Am 07.02.2017 um 20:56 schrieb Stefan Priebe - Profihost AG >> : >> >> Hi, >> >> got this one today with both patches applied: >> >> Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 allocator_free (node=0x0, allocator=0x7f350405e030) >>at memory/unix/apr_pools.c:381 >> #0 allocator_free (node=0x0, allocator=0x7f350405e030) >>at memory/unix/apr_pools.c:381 >> #1 apr_pool_clear (pool=0x7f3504060468) at memory/unix/apr_pools.c:793 >> #2 0x55d11256d688 in ap_push_pool (queue_info=0x7f35040604f8, >>pool_to_recycle=0x) at fdqueue.c:234 >> #3 0x55d11256899a in process_lingering_close (cs=0x7f3504060748, >>pfd=0x55d1143fd7f8) at event.c:1513 >> #4 0x55d11256c4d0 in listener_thread (thd=0x7f35040604f8, >>dummy=0x547f569acd4a6) at event.c:1837 >> #5 0x7f351757c0a4 in start_thread () >> from /lib/x86_64-linux-gnu/libpthread.so.0 >> #6 0x7f35172b162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> Am 06.02.2017 um 22:24 schrieb Stefan Priebe - Profihost AG: >>> Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: Hi Stefan, this one does not apply on top of yann's mpm_event_listener_wakeup_bug57399_V7.patch patch. >>> >>> i've now used his patch to mpm and yours for mod_http2. >>> >>> Stefan >>> Stefan Am 06.02.2017 um 15:34 schrieb Stefan Eissing: > Yes, that was mixing the order. The attached v4 compiles and runs the h2 > tests for me without errors. > > > > > >> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : >> >> On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing >> wrote: >>> Currently running some tests. Have crashes on the original patch in my >>> test suite. Fixed one, hunting for the next... >> >> I think it comes from my change that creates slave connections from >> master->pool (instead of mplx's), because now slave's pool is already >> destroyed when >> h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() >> is called (hence the crash). >> >> I restored your original code in this new (attached) patch. >> >> @s.priebe, would you test this one please? >> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de > > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Stefan, at this point and after several efforts to write the right patch, I reached the conclusion that I need to rethink the pool hierarchy and connection shutdown strategy in mod_http2. The current, organically grown implementation needs a refactor with the knowledge we have made so far. So, a fix will not come today or tomorrow. But not too far away either. From your comments I assume that these crashed happen not too frequently. Hope this allows you to live with the current state for a while. Btw. have the status 500 lines disappeared with the latest release? That was one point still open on my list... Hope to give you something better to verify in your environment soon. Cheers, Stefan > Am 07.02.2017 um 20:56 schrieb Stefan Priebe - Profihost AG > : > > Hi, > > got this one today with both patches applied: > > Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 allocator_free (node=0x0, allocator=0x7f350405e030) >at memory/unix/apr_pools.c:381 > #0 allocator_free (node=0x0, allocator=0x7f350405e030) >at memory/unix/apr_pools.c:381 > #1 apr_pool_clear (pool=0x7f3504060468) at memory/unix/apr_pools.c:793 > #2 0x55d11256d688 in ap_push_pool (queue_info=0x7f35040604f8, >pool_to_recycle=0x) at fdqueue.c:234 > #3 0x55d11256899a in process_lingering_close (cs=0x7f3504060748, >pfd=0x55d1143fd7f8) at event.c:1513 > #4 0x55d11256c4d0 in listener_thread (thd=0x7f35040604f8, >dummy=0x547f569acd4a6) at event.c:1837 > #5 0x7f351757c0a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #6 0x7f35172b162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Am 06.02.2017 um 22:24 schrieb Stefan Priebe - Profihost AG: >> Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: >>> Hi Stefan, >>> >>> this one does not apply on top of yann's >>> mpm_event_listener_wakeup_bug57399_V7.patch patch. >> >> i've now used his patch to mpm and yours for mod_http2. >> >> Stefan >> >>> >>> Stefan >>> Am 06.02.2017 um 15:34 schrieb Stefan Eissing: Yes, that was mixing the order. The attached v4 compiles and runs the h2 tests for me without errors. > Am 06.02.2017 um 14:43 schrieb Yann Ylavic : > > On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing > wrote: >> Currently running some tests. Have crashes on the original patch in my >> test suite. Fixed one, hunting for the next... > > I think it comes from my change that creates slave connections from > master->pool (instead of mplx's), because now slave's pool is already > destroyed when > h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() > is called (hence the crash). > > I restored your original code in this new (attached) patch. > > @s.priebe, would you test this one please? > Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, got this one today with both patches applied: Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x0, allocator=0x7f350405e030) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x0, allocator=0x7f350405e030) at memory/unix/apr_pools.c:381 #1 apr_pool_clear (pool=0x7f3504060468) at memory/unix/apr_pools.c:793 #2 0x55d11256d688 in ap_push_pool (queue_info=0x7f35040604f8, pool_to_recycle=0x) at fdqueue.c:234 #3 0x55d11256899a in process_lingering_close (cs=0x7f3504060748, pfd=0x55d1143fd7f8) at event.c:1513 #4 0x55d11256c4d0 in listener_thread (thd=0x7f35040604f8, dummy=0x547f569acd4a6) at event.c:1837 #5 0x7f351757c0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f35172b162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Am 06.02.2017 um 22:24 schrieb Stefan Priebe - Profihost AG: > Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: >> Hi Stefan, >> >> this one does not apply on top of yann's >> mpm_event_listener_wakeup_bug57399_V7.patch patch. > > i've now used his patch to mpm and yours for mod_http2. > > Stefan > >> >> Stefan >> Am 06.02.2017 um 15:34 schrieb Stefan Eissing: >>> Yes, that was mixing the order. The attached v4 compiles and runs the h2 >>> tests for me without errors. >>> >>> >>> >>> >>> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing wrote: > Currently running some tests. Have crashes on the original patch in my > test suite. Fixed one, hunting for the next... I think it comes from my change that creates slave connections from master->pool (instead of mplx's), because now slave's pool is already destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() is called (hence the crash). I restored your original code in this new (attached) patch. @s.priebe, would you test this one please? >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>>
Re: mod_http2 and Frequent wake-ups for mpm_event
Ok, thanks. I did not have the other one in place. > Am 06.02.2017 um 22:24 schrieb Stefan Priebe - Profihost AG > : > >> Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: >> Hi Stefan, >> >> this one does not apply on top of yann's >> mpm_event_listener_wakeup_bug57399_V7.patch patch. > > i've now used his patch to mpm and yours for mod_http2. > > Stefan > >> >> Stefan >>> Am 06.02.2017 um 15:34 schrieb Stefan Eissing: >>> Yes, that was mixing the order. The attached v4 compiles and runs the h2 >>> tests for me without errors. >>> >>> >>> >>> >>> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing wrote: > Currently running some tests. Have crashes on the original patch in my > test suite. Fixed one, hunting for the next... I think it comes from my change that creates slave connections from master->pool (instead of mplx's), because now slave's pool is already destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() is called (hence the crash). I restored your original code in this new (attached) patch. @s.priebe, would you test this one please? >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>>
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 06.02.2017 um 22:22 schrieb Stefan Priebe - Profihost AG: > Hi Stefan, > > this one does not apply on top of yann's > mpm_event_listener_wakeup_bug57399_V7.patch patch. i've now used his patch to mpm and yours for mod_http2. Stefan > > Stefan > Am 06.02.2017 um 15:34 schrieb Stefan Eissing: >> Yes, that was mixing the order. The attached v4 compiles and runs the h2 >> tests for me without errors. >> >> >> >> >> >>> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : >>> >>> On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing >>> wrote: Currently running some tests. Have crashes on the original patch in my test suite. Fixed one, hunting for the next... >>> >>> I think it comes from my change that creates slave connections from >>> master->pool (instead of mplx's), because now slave's pool is already >>> destroyed when >>> h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() >>> is called (hence the crash). >>> >>> I restored your original code in this new (attached) patch. >>> >>> @s.priebe, would you test this one please? >>> >> >> Stefan Eissing >> >> bytes GmbH >> Hafenstrasse 16 >> 48155 Münster >> www.greenbytes.de >>
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan, this one does not apply on top of yann's mpm_event_listener_wakeup_bug57399_V7.patch patch. Stefan Am 06.02.2017 um 15:34 schrieb Stefan Eissing: > Yes, that was mixing the order. The attached v4 compiles and runs the h2 > tests for me without errors. > > > > > >> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : >> >> On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing >> wrote: >>> Currently running some tests. Have crashes on the original patch in my test >>> suite. Fixed one, hunting for the next... >> >> I think it comes from my change that creates slave connections from >> master->pool (instead of mplx's), because now slave's pool is already >> destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() >> is called (hence the crash). >> >> I restored your original code in this new (attached) patch. >> >> @s.priebe, would you test this one please? >> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 06.02.2017 um 16:06 schrieb Stefan Eissing: > It does, at the end. Traversed the directories in different order it seems. *arg* sorry > >> Am 06.02.2017 um 16:05 schrieb Stefan Priebe - Profihost AG >> : >> >> Am 06.02.2017 um 16:03 schrieb Stefan Eissing: >>> I took Yann's v3 patch and changed a tiny bit in the h2. I just added a var >>> declaration in event.c without which it did not compile for me. >> >> But your patch does not contain the changes to the event mpm. That's why >> i ask. >> >> Stefan >> >>> -Stefan >>> Am 06.02.2017 um 15:41 schrieb Stefan Priebe - Profihost AG : Hi, so i should test the mpm event part from Yann + your http2 part? Stefan Am 06.02.2017 um 15:34 schrieb Stefan Eissing: > Yes, that was mixing the order. The attached v4 compiles and runs the h2 > tests for me without errors. > > > > > >> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : >> >> On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing >> wrote: >>> Currently running some tests. Have crashes on the original patch in my >>> test suite. Fixed one, hunting for the next... >> >> I think it comes from my change that creates slave connections from >> master->pool (instead of mplx's), because now slave's pool is already >> destroyed when >> h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() >> is called (hence the crash). >> >> I restored your original code in this new (attached) patch. >> >> @s.priebe, would you test this one please? >> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de > >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
It does, at the end. Traversed the directories in different order it seems. > Am 06.02.2017 um 16:05 schrieb Stefan Priebe - Profihost AG > : > > Am 06.02.2017 um 16:03 schrieb Stefan Eissing: >> I took Yann's v3 patch and changed a tiny bit in the h2. I just added a var >> declaration in event.c without which it did not compile for me. > > But your patch does not contain the changes to the event mpm. That's why > i ask. > > Stefan > >> -Stefan >> >>> Am 06.02.2017 um 15:41 schrieb Stefan Priebe - Profihost AG >>> : >>> >>> Hi, >>> >>> so i should test the mpm event part from Yann + your http2 part? >>> >>> Stefan >>> >>> Am 06.02.2017 um 15:34 schrieb Stefan Eissing: Yes, that was mixing the order. The attached v4 compiles and runs the h2 tests for me without errors. > Am 06.02.2017 um 14:43 schrieb Yann Ylavic : > > On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing > wrote: >> Currently running some tests. Have crashes on the original patch in my >> test suite. Fixed one, hunting for the next... > > I think it comes from my change that creates slave connections from > master->pool (instead of mplx's), because now slave's pool is already > destroyed when > h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() > is called (hence the crash). > > I restored your original code in this new (attached) patch. > > @s.priebe, would you test this one please? > Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de >> >> Stefan Eissing >> >> bytes GmbH >> Hafenstrasse 16 >> 48155 Münster >> www.greenbytes.de >> Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 06.02.2017 um 16:03 schrieb Stefan Eissing: > I took Yann's v3 patch and changed a tiny bit in the h2. I just added a var > declaration in event.c without which it did not compile for me. But your patch does not contain the changes to the event mpm. That's why i ask. Stefan > -Stefan > >> Am 06.02.2017 um 15:41 schrieb Stefan Priebe - Profihost AG >> : >> >> Hi, >> >> so i should test the mpm event part from Yann + your http2 part? >> >> Stefan >> >> Am 06.02.2017 um 15:34 schrieb Stefan Eissing: >>> Yes, that was mixing the order. The attached v4 compiles and runs the h2 >>> tests for me without errors. >>> >>> >>> >>> >>> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing wrote: > Currently running some tests. Have crashes on the original patch in my > test suite. Fixed one, hunting for the next... I think it comes from my change that creates slave connections from master->pool (instead of mplx's), because now slave's pool is already destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() is called (hence the crash). I restored your original code in this new (attached) patch. @s.priebe, would you test this one please? >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
I took Yann's v3 patch and changed a tiny bit in the h2. I just added a var declaration in event.c without which it did not compile for me. -Stefan > Am 06.02.2017 um 15:41 schrieb Stefan Priebe - Profihost AG > : > > Hi, > > so i should test the mpm event part from Yann + your http2 part? > > Stefan > > Am 06.02.2017 um 15:34 schrieb Stefan Eissing: >> Yes, that was mixing the order. The attached v4 compiles and runs the h2 >> tests for me without errors. >> >> >> >> >> >>> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : >>> >>> On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing >>> wrote: Currently running some tests. Have crashes on the original patch in my test suite. Fixed one, hunting for the next... >>> >>> I think it comes from my change that creates slave connections from >>> master->pool (instead of mplx's), because now slave's pool is already >>> destroyed when >>> h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() >>> is called (hence the crash). >>> >>> I restored your original code in this new (attached) patch. >>> >>> @s.priebe, would you test this one please? >>> >> >> Stefan Eissing >> >> bytes GmbH >> Hafenstrasse 16 >> 48155 Münster >> www.greenbytes.de >> Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, so i should test the mpm event part from Yann + your http2 part? Stefan Am 06.02.2017 um 15:34 schrieb Stefan Eissing: > Yes, that was mixing the order. The attached v4 compiles and runs the h2 > tests for me without errors. > > > > > >> Am 06.02.2017 um 14:43 schrieb Yann Ylavic : >> >> On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing >> wrote: >>> Currently running some tests. Have crashes on the original patch in my test >>> suite. Fixed one, hunting for the next... >> >> I think it comes from my change that creates slave connections from >> master->pool (instead of mplx's), because now slave's pool is already >> destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() >> is called (hence the crash). >> >> I restored your original code in this new (attached) patch. >> >> @s.priebe, would you test this one please? >> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Yes, that was mixing the order. The attached v4 compiles and runs the h2 tests for me without errors. ptrans_and_slaves_allocator-v4.patch Description: Binary data > Am 06.02.2017 um 14:43 schrieb Yann Ylavic : > > On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing > wrote: >> Currently running some tests. Have crashes on the original patch in my test >> suite. Fixed one, hunting for the next... > > I think it comes from my change that creates slave connections from > master->pool (instead of mplx's), because now slave's pool is already > destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy() > is called (hence the crash). > > I restored your original code in this new (attached) patch. > > @s.priebe, would you test this one please? > Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
On Mon, Feb 6, 2017 at 2:23 PM, Ruediger Pluem wrote: > > The question how much cycles this spends in GLIBC / kernel. I don't > know. So maybe its not worth the effort. But if its not worth the > effort it is worth documenting why :-) Sure ;)
Re: mod_http2 and Frequent wake-ups for mpm_event
On Mon, Feb 6, 2017 at 2:31 PM, Stefan Eissing
wrote:
> Currently running some tests. Have crashes on the original patch in my test
> suite. Fixed one, hunting for the next...
I think it comes from my change that creates slave connections from
master->pool (instead of mplx's), because now slave's pool is already
destroyed when h2_mplx_release_and_join()->task_destroy()->h2_slave_destroy()
is called (hence the crash).
I restored your original code in this new (attached) patch.
@s.priebe, would you test this one please?
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c (revision 1781789)
+++ server/mpm/event/event.c (working copy)
@@ -1743,6 +1743,8 @@ static void * APR_THREAD_FUNC listener_thread(apr_
enable_listensocks(process_slot);
}
if (!listeners_disabled) {
+apr_thread_mutex_t *mutex;
+
lr = (ap_listen_rec *) pt->baton;
ap_pop_pool(&ptrans, worker_queue_info);
@@ -1751,19 +1753,24 @@ static void * APR_THREAD_FUNC listener_thread(apr_
apr_allocator_t *allocator;
apr_allocator_create(&allocator);
-apr_allocator_max_free_set(allocator,
- ap_max_mem_free);
-apr_pool_create_ex(&ptrans, pconf, NULL, allocator);
-apr_allocator_owner_set(allocator, ptrans);
-if (ptrans == NULL) {
+apr_allocator_max_free_set(allocator, ap_max_mem_free);
+rc = apr_pool_create_ex(&ptrans, pconf, NULL,
+allocator);
+if (rc != APR_SUCCESS) {
ap_log_error(APLOG_MARK, APLOG_CRIT, rc,
ap_server_conf, APLOGNO(03097)
"Failed to create transaction pool");
+apr_allocator_destroy(allocator);
signal_threads(ST_GRACEFUL);
return NULL;
}
+apr_allocator_owner_set(allocator, ptrans);
+apr_pool_tag(ptrans, "transaction");
}
-apr_pool_tag(ptrans, "transaction");
+apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT,
+ptrans);
+apr_allocator_mutex_set(apr_pool_allocator_get(ptrans),
+mutex);
get_worker(&have_idle_worker, 1, &workers_were_busy);
rc = lr->accept_func(&csd, lr, ptrans);
Index: modules/http2/h2_conn.c
===
--- modules/http2/h2_conn.c (revision 1781789)
+++ modules/http2/h2_conn.c (working copy)
@@ -26,6 +26,8 @@
#include
#include
+#include "mpm_common.h" /* for ap_max_mem_free */
+
#include "h2_private.h"
#include "h2.h"
#include "h2_config.h"
@@ -250,6 +252,7 @@ apr_status_t h2_conn_pre_close(struct h2_ctx *ctx,
conn_rec *h2_slave_create(conn_rec *master, int slave_id, apr_pool_t *parent)
{
apr_allocator_t *allocator;
+apr_thread_mutex_t *mutex;
apr_pool_t *pool;
conn_rec *c;
void *cfg;
@@ -264,14 +267,18 @@ conn_rec *h2_slave_create(conn_rec *master, int sl
* another thread.
*/
apr_allocator_create(&allocator);
+apr_allocator_max_free_set(allocator, ap_max_mem_free);
apr_pool_create_ex(&pool, parent, NULL, allocator);
+apr_allocator_owner_set(allocator, pool);
apr_pool_tag(pool, "h2_slave_conn");
-apr_allocator_owner_set(allocator, pool);
+apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, pool);
+apr_allocator_mutex_set(allocator, mutex);
c = (conn_rec *) apr_palloc(pool, sizeof(conn_rec));
if (c == NULL) {
ap_log_cerror(APLOG_MARK, APLOG_ERR, APR_ENOMEM, master,
APLOGNO(02913) "h2_task: creating conn");
+apr_pool_destroy(pool);
return NULL;
}
Index: modules/http2/h2_mplx.c
===
--- modules/http2/h2_mplx.c (revision 1781789)
+++ modules/http2/h2_mplx.c (working copy)
@@ -33,6 +33,7 @@
#include "h2_private.h"
#include "h2_bucket_beam.h"
#include "h2_config.h"
+#include "h2_session.h"
#include "h2_conn.h"
#include "h2_ctx.h"
#include "h2_h2.h"
@@ -259,32 +260,27 @@ static void h2_mplx_destroy(h2_mplx *m)
* their HTTP/1 cousins, the separate allocator seems to work better
* than protecting a shared h2_session one with an own lock.
*/
-h2_mplx *h2_mplx_create(conn_rec *c, apr_pool_t *parent,
-const h2_config *conf,
-
Re: mod_http2 and Frequent wake-ups for mpm_event
Currently running some tests. Have crashes on the original patch in my test suite. Fixed one, hunting for the next... > Am 06.02.2017 um 14:23 schrieb Ruediger Pluem : > > > > On 02/06/2017 01:51 PM, Yann Ylavic wrote: >> On Mon, Feb 6, 2017 at 1:42 PM, Yann Ylavic wrote: >>> On Mon, Feb 6, 2017 at 1:29 PM, Ruediger Pluem wrote: On 02/06/2017 11:56 AM, Yann Ylavic wrote: > Hi Stefan, > > On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG > wrote: >> >> your last patch results in multiple crashes every second: > > Sorry about that, the changes in mpm_event were incorrect (the mutex > was cleared with the pool when recycled, hence its pointer was > dangling). > > New patch attached, this time tested with the httpd framework (where > the previous patch segfaulted too). > > Thanks, > Yann. > Hmm, does it make sense performance wise to create the mutex over and over again? Or is this planned to be improved once it is known to fix the crash issue? >>> >>> Yes, I'm thinking of it, but it's not easy because we need a pool to >>> create the mutex. >>> Using ptrans makes it cleared on recycle (hence re-created), and using >>> the parent pool (pconf) requires synchronization. >>> >>> Possibly we could recycle both the pool (or the allocator) and its >>> mutex, but ap_push/pop_pool() wouldn't be lockless anymore... >> >> Not sure it's really worth it either because apr_thread_mutex_create() >> should boil down to "*mutex = PTHREAD_MUTEXT_INIT" on *nix, and >> probably something equivalent for InitializeCriticalSection() on >> windows... >> We probably not spend many cycles here (compared to more synchronization). > > The question how much cycles this spends in GLIBC / kernel. I don't know. So > maybe its not worth > the effort. But if its not worth the effort it is worth documenting why :-) > > > Regards > > Rüdiger Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
On 02/06/2017 01:51 PM, Yann Ylavic wrote: > On Mon, Feb 6, 2017 at 1:42 PM, Yann Ylavic wrote: >> On Mon, Feb 6, 2017 at 1:29 PM, Ruediger Pluem wrote: >>> >>> >>> On 02/06/2017 11:56 AM, Yann Ylavic wrote: Hi Stefan, On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG wrote: > > your last patch results in multiple crashes every second: Sorry about that, the changes in mpm_event were incorrect (the mutex was cleared with the pool when recycled, hence its pointer was dangling). New patch attached, this time tested with the httpd framework (where the previous patch segfaulted too). Thanks, Yann. >>> >>> Hmm, does it make sense performance wise to create the mutex over and over >>> again? >>> Or is this planned to be improved once it is known to fix the crash issue? >> >> Yes, I'm thinking of it, but it's not easy because we need a pool to >> create the mutex. >> Using ptrans makes it cleared on recycle (hence re-created), and using >> the parent pool (pconf) requires synchronization. >> >> Possibly we could recycle both the pool (or the allocator) and its >> mutex, but ap_push/pop_pool() wouldn't be lockless anymore... > > Not sure it's really worth it either because apr_thread_mutex_create() > should boil down to "*mutex = PTHREAD_MUTEXT_INIT" on *nix, and > probably something equivalent for InitializeCriticalSection() on > windows... > We probably not spend many cycles here (compared to more synchronization). The question how much cycles this spends in GLIBC / kernel. I don't know. So maybe its not worth the effort. But if its not worth the effort it is worth documenting why :-) Regards Rüdiger
Re: mod_http2 and Frequent wake-ups for mpm_event
On Mon, Feb 6, 2017 at 1:42 PM, Yann Ylavic wrote: > On Mon, Feb 6, 2017 at 1:29 PM, Ruediger Pluem wrote: >> >> >> On 02/06/2017 11:56 AM, Yann Ylavic wrote: >>> Hi Stefan, >>> >>> On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG >>> wrote: your last patch results in multiple crashes every second: >>> >>> Sorry about that, the changes in mpm_event were incorrect (the mutex >>> was cleared with the pool when recycled, hence its pointer was >>> dangling). >>> >>> New patch attached, this time tested with the httpd framework (where >>> the previous patch segfaulted too). >>> >>> Thanks, >>> Yann. >>> >> >> Hmm, does it make sense performance wise to create the mutex over and over >> again? >> Or is this planned to be improved once it is known to fix the crash issue? > > Yes, I'm thinking of it, but it's not easy because we need a pool to > create the mutex. > Using ptrans makes it cleared on recycle (hence re-created), and using > the parent pool (pconf) requires synchronization. > > Possibly we could recycle both the pool (or the allocator) and its > mutex, but ap_push/pop_pool() wouldn't be lockless anymore... Not sure it's really worth it either because apr_thread_mutex_create() should boil down to "*mutex = PTHREAD_MUTEXT_INIT" on *nix, and probably something equivalent for InitializeCriticalSection() on windows... We probably not spend many cycles here (compared to more synchronization).
Re: mod_http2 and Frequent wake-ups for mpm_event
On Mon, Feb 6, 2017 at 1:29 PM, Ruediger Pluem wrote: > > > On 02/06/2017 11:56 AM, Yann Ylavic wrote: >> Hi Stefan, >> >> On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG >> wrote: >>> >>> your last patch results in multiple crashes every second: >> >> Sorry about that, the changes in mpm_event were incorrect (the mutex >> was cleared with the pool when recycled, hence its pointer was >> dangling). >> >> New patch attached, this time tested with the httpd framework (where >> the previous patch segfaulted too). >> >> Thanks, >> Yann. >> > > Hmm, does it make sense performance wise to create the mutex over and over > again? > Or is this planned to be improved once it is known to fix the crash issue? Yes, I'm thinking of it, but it's not easy because we need a pool to create the mutex. Using ptrans makes it cleared on recycle (hence re-created), and using the parent pool (pconf) requires synchronization. Possibly we could recycle both the pool (or the allocator) and its mutex, but ap_push/pop_pool() wouldn't be lockless anymore...
Re: mod_http2 and Frequent wake-ups for mpm_event
On 02/06/2017 11:56 AM, Yann Ylavic wrote: > Hi Stefan, > > On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG > wrote: >> >> your last patch results in multiple crashes every second: > > Sorry about that, the changes in mpm_event were incorrect (the mutex > was cleared with the pool when recycled, hence its pointer was > dangling). > > New patch attached, this time tested with the httpd framework (where > the previous patch segfaulted too). > > Thanks, > Yann. > Hmm, does it make sense performance wise to create the mutex over and over again? Or is this planned to be improved once it is known to fix the crash issue? Regards Rüdiger
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 06.02.2017 um 11:56 schrieb Yann Ylavic: > Hi Stefan, > > On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG > wrote: >> >> your last patch results in multiple crashes every second: > > Sorry about that, the changes in mpm_event were incorrect (the mutex > was cleared with the pool when recycled, hence its pointer was > dangling). > > New patch attached, this time tested with the httpd framework (where > the previous patch segfaulted too). Thanks but that one had crashed already once again. This time: error.log: *** Error in `/usr/local/apache/bin/httpd': free(): invalid pointer: 0x7f4bdc023dc0 *** Core was generated by `/usr/local/apache/bin/httpd -DFOREGROUND'. Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x4480447f447e4466, allocator=0x7f4bc40008c0) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x4480447f447e4466, allocator=0x7f4bc40008c0) at memory/unix/apr_pools.c:381 #1 apr_pool_destroy (pool=0x7f4bc404a178) at memory/unix/apr_pools.c:856 #2 0x55dab5531bff in task_destroy (m=0x7f4c4c00e8f8, task=0x7f4bc404e210, called_from_master=0) at h2_mplx.c:396 #3 0x55dab5532e6b in task_done_iter (ctx=, val=) at h2_mplx.c:1060 #4 0x7f4c8b6965e6 in apr_hash_do ( comp=comp@entry=0x55dab5545140 , rec=rec@entry=0x7f4c6bfe6480, ht=) at tables/apr_hash.c:542 #5 0x55dab5545b1f in h2_ihash_iter (ih=, fn=fn@entry=0x55dab5532e60 , ctx=ctx@entry=0x7f4c4c00e8f8) at h2_util.c:315 #6 0x55dab5533433 in h2_mplx_release_and_join (m=0x7f4c4c00e8f8, wait=0x7f4c4c00e8a0) at h2_mplx.c:615 #7 0x55dab5538ab4 in session_pool_cleanup (data=0x7f4c04005c78) at h2_session.c:827 #8 0x7f4c8b69f48e in run_cleanups (cref=0x7f4c4c00e878) at memory/unix/apr_pools.c:2352 #9 apr_pool_destroy (pool=0x7f4c4c00e808) at memory/unix/apr_pools.c:804 #10 0x7f4c8b69f745 in apr_pool_clear (pool=0x7f4c78058198) at memory/unix/apr_pools.c:769 #11 0x55dab5570668 in ap_push_pool (queue_info=0x7f4bc40008c0, pool_to_recycle=0xc4041f91) at fdqueue.c:234 #12 0x55dab556b99a in process_lingering_close (cs=0x7f4c78058478, pfd=0x55dab6cf3fa8) at event.c:1513 #13 0x55dab556f4d0 in listener_thread (thd=0x7f4bc40008c0, dummy=0x547dbb6874f83) at event.c:1837 #14 0x7f4c8b46e0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #15 0x7f4c8b1a362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Greets, Stefan
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan,
On Mon, Feb 6, 2017 at 9:57 AM, Stefan Priebe - Profihost AG
wrote:
>
> your last patch results in multiple crashes every second:
Sorry about that, the changes in mpm_event were incorrect (the mutex
was cleared with the pool when recycled, hence its pointer was
dangling).
New patch attached, this time tested with the httpd framework (where
the previous patch segfaulted too).
Thanks,
Yann.
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c (revision 1781789)
+++ server/mpm/event/event.c (working copy)
@@ -1743,6 +1743,8 @@ static void * APR_THREAD_FUNC listener_thread(apr_
enable_listensocks(process_slot);
}
if (!listeners_disabled) {
+apr_thread_mutex_t *mutex;
+
lr = (ap_listen_rec *) pt->baton;
ap_pop_pool(&ptrans, worker_queue_info);
@@ -1762,8 +1764,10 @@ static void * APR_THREAD_FUNC listener_thread(apr_
signal_threads(ST_GRACEFUL);
return NULL;
}
+apr_pool_tag(ptrans, "transaction");
}
-apr_pool_tag(ptrans, "transaction");
+apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, ptrans);
+apr_allocator_mutex_set(apr_pool_allocator_get(ptrans), mutex);
get_worker(&have_idle_worker, 1, &workers_were_busy);
rc = lr->accept_func(&csd, lr, ptrans);
Index: modules/http2/h2_conn.c
===
--- modules/http2/h2_conn.c (revision 1781789)
+++ modules/http2/h2_conn.c (working copy)
@@ -247,9 +247,10 @@ apr_status_t h2_conn_pre_close(struct h2_ctx *ctx,
return status;
}
-conn_rec *h2_slave_create(conn_rec *master, int slave_id, apr_pool_t *parent)
+conn_rec *h2_slave_create(conn_rec *master, int slave_id)
{
apr_allocator_t *allocator;
+apr_thread_mutex_t *mutex;
apr_pool_t *pool;
conn_rec *c;
void *cfg;
@@ -264,9 +265,11 @@ apr_status_t h2_conn_pre_close(struct h2_ctx *ctx,
* another thread.
*/
apr_allocator_create(&allocator);
-apr_pool_create_ex(&pool, parent, NULL, allocator);
+apr_pool_create_ex(&pool, master->pool, NULL, allocator);
+apr_allocator_owner_set(allocator, pool);
apr_pool_tag(pool, "h2_slave_conn");
-apr_allocator_owner_set(allocator, pool);
+apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, pool);
+apr_allocator_mutex_set(allocator, mutex);
c = (conn_rec *) apr_palloc(pool, sizeof(conn_rec));
if (c == NULL) {
Index: modules/http2/h2_conn.h
===
--- modules/http2/h2_conn.h (revision 1781789)
+++ modules/http2/h2_conn.h (working copy)
@@ -66,7 +66,7 @@ typedef enum {
h2_mpm_type_t h2_conn_mpm_type(void);
-conn_rec *h2_slave_create(conn_rec *master, int slave_id, apr_pool_t *parent);
+conn_rec *h2_slave_create(conn_rec *master, int slave_id);
void h2_slave_destroy(conn_rec *slave);
apr_status_t h2_slave_run_pre_connection(conn_rec *slave, apr_socket_t *csd);
Index: modules/http2/h2_mplx.c
===
--- modules/http2/h2_mplx.c (revision 1781789)
+++ modules/http2/h2_mplx.c (working copy)
@@ -33,6 +33,7 @@
#include "h2_private.h"
#include "h2_bucket_beam.h"
#include "h2_config.h"
+#include "h2_session.h"
#include "h2_conn.h"
#include "h2_ctx.h"
#include "h2_h2.h"
@@ -245,7 +246,7 @@ static void h2_mplx_destroy(h2_mplx *m)
m->id, (int)h2_ihash_count(m->tasks));
check_tx_free(m);
/* pool will be destroyed as child of h2_session->pool,
- slave connection pools are children of m->pool */
+ slave connection pools are children of m->c->pool */
}
/**
@@ -259,32 +260,27 @@ static void h2_mplx_destroy(h2_mplx *m)
* their HTTP/1 cousins, the separate allocator seems to work better
* than protecting a shared h2_session one with an own lock.
*/
-h2_mplx *h2_mplx_create(conn_rec *c, apr_pool_t *parent,
-const h2_config *conf,
-apr_interval_time_t stream_timeout,
-h2_workers *workers)
+h2_mplx *h2_mplx_create(h2_session *session, h2_workers *workers)
{
apr_status_t status = APR_SUCCESS;
-apr_allocator_t *allocator = NULL;
+apr_pool_t *parent = session->pool;
+const h2_config *conf = session->config;
h2_mplx *m;
ap_assert(conf);
-status = apr_allocator_create(&allocator);
-if (status != APR_SUCCESS) {
-return NULL;
-}
-
m = apr_pcalloc(parent, sizeof(h2_mplx));
if (m) {
+conn_rec *c = session->c;
+
m->id = c->id;
APR_RING_ELEM_INIT(m, link);
m->c = c;
-apr
Re: mod_http2 and Frequent wake-ups for mpm_event
Thanks Yann (the one and only) for quickly adressing this. This explains maybe some of my frustrations in the past with making pools/allocators work in the h2 environment. -Stefan > Am 06.02.2017 um 01:19 schrieb Yann Ylavic : > > Hi Stefan, > > On Sun, Feb 5, 2017 at 7:51 PM, Stefan Priebe - Profihost AG > wrote: >> >> tested your patch against mod_ssl. I haven't seen any pool crashes again >> so it seems to fix this issue. >> >> But two new ones: > > Possibly a promising fix suggested (in another thread) by yet another > talented Stefan :) > He has spotted a possible issue in mpm_event which could affect mainly > mod_http2. > I applied his suggestion in the attached patch, and did some > extrapolation for similar code in mod_h2 (so all possible errors are > mine...). > Would you test this one please? > > @icing: I changed the parent pool of the slave connection from the > mplx pool to the master connection's (ptrans), just to simplify the > allocators in place for now. > I don't see it was needed from a concurrency POV, but if you wanted to > avoid locking when creating slaves' pool we can probably keep the > dedicated allocator finally (to reduce possible contention on > ptrans->mutex? No mutex needed I think since mplx doesn't seem to have > other subpools. Trade off with memory consumption, though). > > > Regards, > Yann. >
Re: mod_http2 and Frequent wake-ups for mpm_event
ok, I'll look at them. > Am 05.02.2017 um 19:51 schrieb Stefan Priebe - Profihost AG > : > > Hi Yann, > > tested your patch against mod_ssl. I haven't seen any pool crashes again > so it seems to fix this issue. > > But two new ones: > > Program terminated with signal SIGABRT, Aborted. > #0 0x7ff523558067 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x7ff523558067 in raise () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x7ff523559448 in abort () from /lib/x86_64-linux-gnu/libc.so.6 > #2 0x7ff5235961b4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #3 0x7ff52359b98e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #4 0x7ff52359c923 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #5 0x7ff523b06c83 in apr_allocator_destroy (allocator=0x7ff4de50) >at memory/unix/apr_pools.c:152 > #6 0x556e7986bbef in task_destroy (m=0x7ff46402a028, > task=0x7ff4d0029f00, >called_from_master=0) at h2_mplx.c:400 > #7 0x556e7986ce5b in task_done_iter (ctx=, >val=) at h2_mplx.c:1064 > #8 0x7ff523afe5e6 in apr_hash_do ( >comp=comp@entry=0x556e7987f180 , > rec=rec@entry=0x7ff5037e5480, >ht=) at tables/apr_hash.c:542 > #9 0x556e7987fb5f in h2_ihash_iter (ih=, >fn=fn@entry=0x556e7986ce50 , > ctx=ctx@entry=0x7ff46402a028) >at h2_util.c:315 > #10 0x556e7986d463 in h2_mplx_release_and_join (m=0x7ff46402a028, >wait=0x7ff464029fd0) at h2_mplx.c:619 > #11 0x556e79872ae4 in session_pool_cleanup (data=0x7ff464020318) >at h2_session.c:827 > #12 0x7ff523b0748e in run_cleanups (cref=0x7ff464029fa8) >at memory/unix/apr_pools.c:2352 > #13 apr_pool_destroy (pool=0x7ff464029f38) at memory/unix/apr_pools.c:804 > #14 0x7ff523b07745 in apr_pool_clear (pool=0x7ff4fc0150e8) >at memory/unix/apr_pools.c:769 > #15 0x556e798aa698 in ap_push_pool (queue_info=0x1e2e, >pool_to_recycle=0x1ebb) at fdqueue.c:234 > #16 0x556e798a59da in process_lingering_close (cs=0x7ff4fc015378, >pfd=0x556e7b8bd888) at event.c:1513 > #17 0x556e798a9510 in listener_thread (thd=0x1e2e, > dummy=0x547b44ea3e1b3) >at event.c:1837 > #18 0x7ff5238d60a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #19 0x7ff52360b62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > and > > Program terminated with signal SIGSEGV, Segmentation fault. > #0 apr_bucket_alloc (size=size@entry=64, > list=list@entry=0x3fb50d4a7aeb1d49) >at buckets/apr_buckets_alloc.c:128 > #0 apr_bucket_alloc (size=size@entry=64, > list=list@entry=0x3fb50d4a7aeb1d49) >at buckets/apr_buckets_alloc.c:128 > #1 0x7ff523d2b1d3 in apr_bucket_heap_create ( >buf=0x7ff51003b3a8 " > \311\021A\216y\034\347\034Wy\360\343R\275\226o\020iw\227r\337\377", > length=1300, free_func=0x7ff523d2ab50 , >list=0x3fb50d4a7aeb1d49) at buckets/apr_buckets_heap.c:81 > #2 0x556e79884f85 in append_scratch (io=0x7ff4440377c8) >at h2_conn_io.c:165 > #3 0x556e79884ffa in assure_scratch_space (io=0x7ff4440377c8) >at h2_conn_io.c:182 > #4 0x556e79885ce8 in h2_conn_io_pass (io=io@entry=0x7ff4440377c8, >bb=0x7ff444133698) at h2_conn_io.c:393 > #5 0x556e798732be in on_send_data_cb (ngh2=, >frame=, framehd=, length=1291, >source=, userp=0x7ff444037780) at h2_session.c:648 > #6 0x7ff5243dde95 in ?? () from > /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 > #7 0x7ff5243deea9 in nghttp2_session_send () > from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 > #8 0x556e79875857 in h2_session_send (session=0x7ff444037780) >at h2_session.c:1376 > #9 0x556e79878b6c in h2_session_process (session=0x7ff444037780, >async=2062228809) at h2_session.c:2208 > #10 0x556e79867788 in h2_conn_run (ctx=0x7ff4440376b0, c=0x7ff51003b6a8) >at h2_conn.c:214 > #11 0x556e7986a421 in h2_h2_process_conn (c=0x7ff51003b6a8) at > h2_h2.c:658 > #12 0x556e7980d050 in ap_run_process_connection (c=0x7ff51003b6a8) >at connection.c:42 > #13 0x556e798a7590 in process_socket (my_thread_num=, >my_child_num=, cs=0x7ff51003b618, sock=, >p=, thd=) at event.c:1134 > #14 worker_thread (thd=0x40, dummy=0x3fb50d4a7aeb1d49) at event.c:2137 > #15 0x7ff5238d60a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #16 0x7ff52360b62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Greets, > Stefan > >> Am 02.02.2017 um 11:09 schrieb Yann Ylavic: >> Hi Stefan, >> >> On Tue, Jan 31, 2017 at 4:01 PM, Stefan Priebe - Profihost AG >> wrote: >>> >>> any ideas? >> >> I wonder if the attached patch (related to mod_ssl and proposed for >> another segfault report) could help in your case. >> >> Would you mind give it a try? >> >> >> Thanks, >> Yann. >>
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Yann, your last patch results in multiple crashes every second: Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_pool_cleanup_kill (p=0x7f2b, data=data@entry=0x7f2bb40478c0, cleanup_fn=cleanup_fn@entry=0x7f2bc92285b0 ) at memory/unix/apr_pools.c:2264 2264memory/unix/apr_pools.c: No such file or directory. #0 apr_pool_cleanup_kill (p=0x7f2b, data=data@entry=0x7f2bb40478c0, cleanup_fn=cleanup_fn@entry=0x7f2bc92285b0 ) at memory/unix/apr_pools.c:2264 #1 0x7f2bc9224871 in apr_pool_cleanup_run (p=, data=0x7f2bb40478c0, cleanup_fn=0x7f2bc92285b0 ) at memory/unix/apr_pools.c:2342 #2 0x7f2bc9228892 in apr_socket_close (thesocket=) at network_io/unix/sockets.c:183 #3 0x555d4f07b99a in process_lingering_close (cs=0x7f2bb4047ac8, pfd=0x555d4f75dfa8) at event.c:1510 #4 0x555d4f07f4e0 in listener_thread (thd=0x7f2b, dummy=0x547d8caf3534b) at event.c:1837 #5 0x7f2bc8ff20a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f2bc8d2762d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 06.02.2017 um 09:33 schrieb Stefan Priebe - Profihost AG: > Hallo, > > up and running (apache 2.4.25 + mpm_event V7 + mpd_http2 v1.8.11 + > mod_ssl patch + your new allocator patch). Will report back. > > Greets, > Stefan > > Am 06.02.2017 um 01:19 schrieb Yann Ylavic: >> Hi Stefan, >> >> On Sun, Feb 5, 2017 at 7:51 PM, Stefan Priebe - Profihost AG >> wrote: >>> >>> tested your patch against mod_ssl. I haven't seen any pool crashes again >>> so it seems to fix this issue. >>> >>> But two new ones: >> >> Possibly a promising fix suggested (in another thread) by yet another >> talented Stefan :) >> He has spotted a possible issue in mpm_event which could affect mainly >> mod_http2. >> I applied his suggestion in the attached patch, and did some >> extrapolation for similar code in mod_h2 (so all possible errors are >> mine...). >> Would you test this one please? >> >> @icing: I changed the parent pool of the slave connection from the >> mplx pool to the master connection's (ptrans), just to simplify the >> allocators in place for now. >> I don't see it was needed from a concurrency POV, but if you wanted to >> avoid locking when creating slaves' pool we can probably keep the >> dedicated allocator finally (to reduce possible contention on >> ptrans->mutex? No mutex needed I think since mplx doesn't seem to have >> other subpools. Trade off with memory consumption, though). >> >> >> Regards, >> Yann. >>
Re: mod_http2 and Frequent wake-ups for mpm_event
Hallo, up and running (apache 2.4.25 + mpm_event V7 + mpd_http2 v1.8.11 + mod_ssl patch + your new allocator patch). Will report back. Greets, Stefan Am 06.02.2017 um 01:19 schrieb Yann Ylavic: > Hi Stefan, > > On Sun, Feb 5, 2017 at 7:51 PM, Stefan Priebe - Profihost AG > wrote: >> >> tested your patch against mod_ssl. I haven't seen any pool crashes again >> so it seems to fix this issue. >> >> But two new ones: > > Possibly a promising fix suggested (in another thread) by yet another > talented Stefan :) > He has spotted a possible issue in mpm_event which could affect mainly > mod_http2. > I applied his suggestion in the attached patch, and did some > extrapolation for similar code in mod_h2 (so all possible errors are > mine...). > Would you test this one please? > > @icing: I changed the parent pool of the slave connection from the > mplx pool to the master connection's (ptrans), just to simplify the > allocators in place for now. > I don't see it was needed from a concurrency POV, but if you wanted to > avoid locking when creating slaves' pool we can probably keep the > dedicated allocator finally (to reduce possible contention on > ptrans->mutex? No mutex needed I think since mplx doesn't seem to have > other subpools. Trade off with memory consumption, though). > > > Regards, > Yann. >
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan,
On Sun, Feb 5, 2017 at 7:51 PM, Stefan Priebe - Profihost AG
wrote:
>
> tested your patch against mod_ssl. I haven't seen any pool crashes again
> so it seems to fix this issue.
>
> But two new ones:
Possibly a promising fix suggested (in another thread) by yet another
talented Stefan :)
He has spotted a possible issue in mpm_event which could affect mainly
mod_http2.
I applied his suggestion in the attached patch, and did some
extrapolation for similar code in mod_h2 (so all possible errors are
mine...).
Would you test this one please?
@icing: I changed the parent pool of the slave connection from the
mplx pool to the master connection's (ptrans), just to simplify the
allocators in place for now.
I don't see it was needed from a concurrency POV, but if you wanted to
avoid locking when creating slaves' pool we can probably keep the
dedicated allocator finally (to reduce possible contention on
ptrans->mutex? No mutex needed I think since mplx doesn't seem to have
other subpools. Trade off with memory consumption, though).
Regards,
Yann.
Index: server/mpm/event/event.c
===
--- server/mpm/event/event.c (revision 1781789)
+++ server/mpm/event/event.c (working copy)
@@ -1749,6 +1749,7 @@ static void * APR_THREAD_FUNC listener_thread(apr_
if (ptrans == NULL) {
/* create a new transaction pool for each accepted socket */
apr_allocator_t *allocator;
+apr_thread_mutex_t *mutex = NULL;
apr_allocator_create(&allocator);
apr_allocator_max_free_set(allocator,
@@ -1762,6 +1763,8 @@ static void * APR_THREAD_FUNC listener_thread(apr_
signal_threads(ST_GRACEFUL);
return NULL;
}
+apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, ptrans);
+apr_allocator_mutex_set(allocator, mutex);
}
apr_pool_tag(ptrans, "transaction");
Index: modules/http2/h2_conn.c
===
--- modules/http2/h2_conn.c (revision 1781789)
+++ modules/http2/h2_conn.c (working copy)
@@ -247,9 +247,10 @@ apr_status_t h2_conn_pre_close(struct h2_ctx *ctx,
return status;
}
-conn_rec *h2_slave_create(conn_rec *master, int slave_id, apr_pool_t *parent)
+conn_rec *h2_slave_create(conn_rec *master, int slave_id)
{
-apr_allocator_t *allocator;
+apr_allocator_t *allocator = NULL;
+apr_thread_mutex_t *mutex = NULL;
apr_pool_t *pool;
conn_rec *c;
void *cfg;
@@ -264,9 +265,11 @@ apr_status_t h2_conn_pre_close(struct h2_ctx *ctx,
* another thread.
*/
apr_allocator_create(&allocator);
-apr_pool_create_ex(&pool, parent, NULL, allocator);
+apr_pool_create_ex(&pool, master->pool, NULL, allocator);
apr_pool_tag(pool, "h2_slave_conn");
apr_allocator_owner_set(allocator, pool);
+apr_thread_mutex_create(&mutex, APR_THREAD_MUTEX_DEFAULT, pool);
+apr_allocator_mutex_set(allocator, mutex);
c = (conn_rec *) apr_palloc(pool, sizeof(conn_rec));
if (c == NULL) {
Index: modules/http2/h2_conn.h
===
--- modules/http2/h2_conn.h (revision 1781789)
+++ modules/http2/h2_conn.h (working copy)
@@ -66,7 +66,7 @@ typedef enum {
h2_mpm_type_t h2_conn_mpm_type(void);
-conn_rec *h2_slave_create(conn_rec *master, int slave_id, apr_pool_t *parent);
+conn_rec *h2_slave_create(conn_rec *master, int slave_id);
void h2_slave_destroy(conn_rec *slave);
apr_status_t h2_slave_run_pre_connection(conn_rec *slave, apr_socket_t *csd);
Index: modules/http2/h2_mplx.c
===
--- modules/http2/h2_mplx.c (revision 1781789)
+++ modules/http2/h2_mplx.c (working copy)
@@ -33,6 +33,7 @@
#include "h2_private.h"
#include "h2_bucket_beam.h"
#include "h2_config.h"
+#include "h2_session.h"
#include "h2_conn.h"
#include "h2_ctx.h"
#include "h2_h2.h"
@@ -245,7 +246,7 @@ static void h2_mplx_destroy(h2_mplx *m)
m->id, (int)h2_ihash_count(m->tasks));
check_tx_free(m);
/* pool will be destroyed as child of h2_session->pool,
- slave connection pools are children of m->pool */
+ slave connection pools are children of m->c->pool */
}
/**
@@ -259,32 +260,27 @@ static void h2_mplx_destroy(h2_mplx *m)
* their HTTP/1 cousins, the separate allocator seems to work better
* than protecting a shared h2_session one with an own lock.
*/
-h2_mplx *h2_mplx_create(conn_rec *c, apr_pool_t *parent,
-const h2_config *conf,
-apr_interval_time_t stream_timeout,
-h2_workers *workers)
+h2_mplx *h2_mplx_create(h2_session *session, h2_workers *w
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Yann, tested your patch against mod_ssl. I haven't seen any pool crashes again so it seems to fix this issue. But two new ones: Program terminated with signal SIGABRT, Aborted. #0 0x7ff523558067 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x7ff523558067 in raise () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7ff523559448 in abort () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x7ff5235961b4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #3 0x7ff52359b98e in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #4 0x7ff52359c923 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #5 0x7ff523b06c83 in apr_allocator_destroy (allocator=0x7ff4de50) at memory/unix/apr_pools.c:152 #6 0x556e7986bbef in task_destroy (m=0x7ff46402a028, task=0x7ff4d0029f00, called_from_master=0) at h2_mplx.c:400 #7 0x556e7986ce5b in task_done_iter (ctx=, val=) at h2_mplx.c:1064 #8 0x7ff523afe5e6 in apr_hash_do ( comp=comp@entry=0x556e7987f180 , rec=rec@entry=0x7ff5037e5480, ht=) at tables/apr_hash.c:542 #9 0x556e7987fb5f in h2_ihash_iter (ih=, fn=fn@entry=0x556e7986ce50 , ctx=ctx@entry=0x7ff46402a028) at h2_util.c:315 #10 0x556e7986d463 in h2_mplx_release_and_join (m=0x7ff46402a028, wait=0x7ff464029fd0) at h2_mplx.c:619 #11 0x556e79872ae4 in session_pool_cleanup (data=0x7ff464020318) at h2_session.c:827 #12 0x7ff523b0748e in run_cleanups (cref=0x7ff464029fa8) at memory/unix/apr_pools.c:2352 #13 apr_pool_destroy (pool=0x7ff464029f38) at memory/unix/apr_pools.c:804 #14 0x7ff523b07745 in apr_pool_clear (pool=0x7ff4fc0150e8) at memory/unix/apr_pools.c:769 #15 0x556e798aa698 in ap_push_pool (queue_info=0x1e2e, pool_to_recycle=0x1ebb) at fdqueue.c:234 #16 0x556e798a59da in process_lingering_close (cs=0x7ff4fc015378, pfd=0x556e7b8bd888) at event.c:1513 #17 0x556e798a9510 in listener_thread (thd=0x1e2e, dummy=0x547b44ea3e1b3) at event.c:1837 #18 0x7ff5238d60a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #19 0x7ff52360b62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 and Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_bucket_alloc (size=size@entry=64, list=list@entry=0x3fb50d4a7aeb1d49) at buckets/apr_buckets_alloc.c:128 #0 apr_bucket_alloc (size=size@entry=64, list=list@entry=0x3fb50d4a7aeb1d49) at buckets/apr_buckets_alloc.c:128 #1 0x7ff523d2b1d3 in apr_bucket_heap_create ( buf=0x7ff51003b3a8 " \311\021A\216y\034\347\034Wy\360\343R\275\226o\020iw\227r\337\377", length=1300, free_func=0x7ff523d2ab50 , list=0x3fb50d4a7aeb1d49) at buckets/apr_buckets_heap.c:81 #2 0x556e79884f85 in append_scratch (io=0x7ff4440377c8) at h2_conn_io.c:165 #3 0x556e79884ffa in assure_scratch_space (io=0x7ff4440377c8) at h2_conn_io.c:182 #4 0x556e79885ce8 in h2_conn_io_pass (io=io@entry=0x7ff4440377c8, bb=0x7ff444133698) at h2_conn_io.c:393 #5 0x556e798732be in on_send_data_cb (ngh2=, frame=, framehd=, length=1291, source=, userp=0x7ff444037780) at h2_session.c:648 #6 0x7ff5243dde95 in ?? () from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 #7 0x7ff5243deea9 in nghttp2_session_send () from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 #8 0x556e79875857 in h2_session_send (session=0x7ff444037780) at h2_session.c:1376 #9 0x556e79878b6c in h2_session_process (session=0x7ff444037780, async=2062228809) at h2_session.c:2208 #10 0x556e79867788 in h2_conn_run (ctx=0x7ff4440376b0, c=0x7ff51003b6a8) at h2_conn.c:214 #11 0x556e7986a421 in h2_h2_process_conn (c=0x7ff51003b6a8) at h2_h2.c:658 #12 0x556e7980d050 in ap_run_process_connection (c=0x7ff51003b6a8) at connection.c:42 #13 0x556e798a7590 in process_socket (my_thread_num=, my_child_num=, cs=0x7ff51003b618, sock=, p=, thd=) at event.c:1134 #14 worker_thread (thd=0x40, dummy=0x3fb50d4a7aeb1d49) at event.c:2137 #15 0x7ff5238d60a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #16 0x7ff52360b62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Greets, Stefan Am 02.02.2017 um 11:09 schrieb Yann Ylavic: > Hi Stefan, > > On Tue, Jan 31, 2017 at 4:01 PM, Stefan Priebe - Profihost AG > wrote: >> >> any ideas? > > I wonder if the attached patch (related to mod_ssl and proposed for > another segfault report) could help in your case. > > Would you mind give it a try? > > > Thanks, > Yann. >
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 02.02.2017 um 11:09 schrieb Yann Ylavic: > Hi Stefan, > > On Tue, Jan 31, 2017 at 4:01 PM, Stefan Priebe - Profihost AG > wrote: >> >> any ideas? > > I wonder if the attached patch (related to mod_ssl and proposed for > another segfault report) could help in your case. > > Would you mind give it a try? Up and running. It will take some time until i can report back as those crashes are very rare. Stefan
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan,
On Tue, Jan 31, 2017 at 4:01 PM, Stefan Priebe - Profihost AG
wrote:
>
> any ideas?
I wonder if the attached patch (related to mod_ssl and proposed for
another segfault report) could help in your case.
Would you mind give it a try?
Thanks,
Yann.
Index: modules/ssl/ssl_engine_io.c
===
--- modules/ssl/ssl_engine_io.c (revision 1781324)
+++ modules/ssl/ssl_engine_io.c (working copy)
@@ -138,6 +138,7 @@ static int bio_filter_out_pass(bio_filter_out_ctx_
outctx->rc = ap_pass_brigade(outctx->filter_ctx->pOutputFilter->next,
outctx->bb);
+apr_brigade_cleanup(outctx->bb);
/* Fail if the connection was reset: */
if (outctx->rc == APR_SUCCESS && outctx->c->aborted) {
outctx->rc = APR_ECONNRESET;
@@ -1699,13 +1700,12 @@ static apr_status_t ssl_io_filter_output(ap_filter
while (!APR_BRIGADE_EMPTY(bb) && status == APR_SUCCESS) {
apr_bucket *bucket = APR_BRIGADE_FIRST(bb);
-if (APR_BUCKET_IS_METADATA(bucket)) {
+if (APR_BUCKET_IS_METADATA(bucket) || !filter_ctx->pssl) {
/* Pass through metadata buckets untouched. EOC is
* special; terminate the SSL layer first. */
if (AP_BUCKET_IS_EOC(bucket)) {
ssl_filter_io_shutdown(filter_ctx, f->c, 0);
}
-AP_DEBUG_ASSERT(APR_BRIGADE_EMPTY(outctx->bb));
/* Metadata buckets are passed one per brigade; it might
* be more efficient (but also more complex) to use
@@ -1712,11 +1712,10 @@ static apr_status_t ssl_io_filter_output(ap_filter
* outctx->bb as a true buffer and interleave these with
* data buckets. */
APR_BUCKET_REMOVE(bucket);
-APR_BRIGADE_INSERT_HEAD(outctx->bb, bucket);
-status = ap_pass_brigade(f->next, outctx->bb);
-if (status == APR_SUCCESS && f->c->aborted)
-status = APR_ECONNRESET;
-apr_brigade_cleanup(outctx->bb);
+APR_BRIGADE_INSERT_TAIL(outctx->bb, bucket);
+if (bio_filter_out_pass(outctx) < 0) {
+status = outctx->rc;
+}
}
else {
/* Filter a data bucket. */
Re: mod_http2 and Frequent wake-ups for mpm_event
As Rüdiger wrote, this stacktrace does not make any sense (to me), unless the pool cleanup messes up the stack. Hmmm. This points to a major misconception of mine how things are supposed to happen. Will continue to look at this, but no easy solution in sight. > Am 27.01.2017 um 20:30 schrieb Ruediger Pluem : > > > > On 01/27/2017 08:10 PM, Stefan Priebe - Profihost AG wrote: >> Hi Yann, >> >> this is now a segfault with mod_http2 + beam v4 patch + mpm v7 patch: >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 allocator_free (node=0x0, allocator=0x7f8974080c90) >>at memory/unix/apr_pools.c:381 >> #0 allocator_free (node=0x0, allocator=0x7f8974080c90) >>at memory/unix/apr_pools.c:381 >> #1 apr_pool_clear (pool=0x7f89740830d8) at memory/unix/apr_pools.c:793 > > pool from above != pool_to_recycle from below is strange. There is no modification to that parameter in this function. Which would mean the cleanup function/child pool destruction somehow garbles the stack??? Which would explain the strangeness below: >> #2 0x5611421447a8 in ap_push_pool (queue_info=0x0, > > queue_info NULL? That looks bad. And that should have crashed at fdqueue.c line 225 already if queue_info == NULLL. >>pool_to_recycle=0x7f8974080c98) at fdqueue.c:234 >> #3 0x56114213faea in process_lingering_close (cs=0x7f8974083368, >>pfd=0x561142cc97f8) at event.c:1513 >> #4 0x561142143620 in listener_thread (thd=0x0, dummy=0x54716c3684bbc) >>at event.c:1837 >> #5 0x7f8989bb20a4 in start_thread () >> from /lib/x86_64-linux-gnu/libpthread.so.0 >> #6 0x7f89898e762d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Regards > > Rüdiger > Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 01.02.2017 um 06:36 schrieb Stefan Eissing: Stefan, did not have the time to look at it really. Will do, but very busy right now. Thanks - no problem. Am 31.01.2017 um 16:01 schrieb Stefan Priebe - Profihost AG : Hi Yann, Hi Stefan, any ideas? Stefan Am 27.01.2017 um 20:30 schrieb Ruediger Pluem: On 01/27/2017 08:10 PM, Stefan Priebe - Profihost AG wrote: Hi Yann, this is now a segfault with mod_http2 + beam v4 patch + mpm v7 patch: Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x0, allocator=0x7f8974080c90) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x0, allocator=0x7f8974080c90) at memory/unix/apr_pools.c:381 #1 apr_pool_clear (pool=0x7f89740830d8) at memory/unix/apr_pools.c:793 pool from above != pool_to_recycle from below is strange. #2 0x5611421447a8 in ap_push_pool (queue_info=0x0, queue_info NULL? That looks bad. pool_to_recycle=0x7f8974080c98) at fdqueue.c:234 #3 0x56114213faea in process_lingering_close (cs=0x7f8974083368, pfd=0x561142cc97f8) at event.c:1513 #4 0x561142143620 in listener_thread (thd=0x0, dummy=0x54716c3684bbc) at event.c:1837 #5 0x7f8989bb20a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f89898e762d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Regards Rüdiger
Re: mod_http2 and Frequent wake-ups for mpm_event
Stefan, did not have the time to look at it really. Will do, but very busy right now. > Am 31.01.2017 um 16:01 schrieb Stefan Priebe - Profihost AG > : > > Hi Yann, > Hi Stefan, > > any ideas? > > Stefan > >> Am 27.01.2017 um 20:30 schrieb Ruediger Pluem: >> >> >>> On 01/27/2017 08:10 PM, Stefan Priebe - Profihost AG wrote: >>> Hi Yann, >>> >>> this is now a segfault with mod_http2 + beam v4 patch + mpm v7 patch: >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 allocator_free (node=0x0, allocator=0x7f8974080c90) >>>at memory/unix/apr_pools.c:381 >>> #0 allocator_free (node=0x0, allocator=0x7f8974080c90) >>>at memory/unix/apr_pools.c:381 >>> #1 apr_pool_clear (pool=0x7f89740830d8) at memory/unix/apr_pools.c:793 >> >> pool from above != pool_to_recycle from below is strange. >> >>> #2 0x5611421447a8 in ap_push_pool (queue_info=0x0, >> >> queue_info NULL? That looks bad. >> >>>pool_to_recycle=0x7f8974080c98) at fdqueue.c:234 >>> #3 0x56114213faea in process_lingering_close (cs=0x7f8974083368, >>>pfd=0x561142cc97f8) at event.c:1513 >>> #4 0x561142143620 in listener_thread (thd=0x0, dummy=0x54716c3684bbc) >>>at event.c:1837 >>> #5 0x7f8989bb20a4 in start_thread () >>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>> #6 0x7f89898e762d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> Regards >> >> Rüdiger >>
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Yann, Hi Stefan, any ideas? Stefan Am 27.01.2017 um 20:30 schrieb Ruediger Pluem: > > > On 01/27/2017 08:10 PM, Stefan Priebe - Profihost AG wrote: >> Hi Yann, >> >> this is now a segfault with mod_http2 + beam v4 patch + mpm v7 patch: >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 allocator_free (node=0x0, allocator=0x7f8974080c90) >> at memory/unix/apr_pools.c:381 >> #0 allocator_free (node=0x0, allocator=0x7f8974080c90) >> at memory/unix/apr_pools.c:381 >> #1 apr_pool_clear (pool=0x7f89740830d8) at memory/unix/apr_pools.c:793 > > pool from above != pool_to_recycle from below is strange. > >> #2 0x5611421447a8 in ap_push_pool (queue_info=0x0, > > queue_info NULL? That looks bad. > >> pool_to_recycle=0x7f8974080c98) at fdqueue.c:234 >> #3 0x56114213faea in process_lingering_close (cs=0x7f8974083368, >> pfd=0x561142cc97f8) at event.c:1513 >> #4 0x561142143620 in listener_thread (thd=0x0, dummy=0x54716c3684bbc) >> at event.c:1837 >> #5 0x7f8989bb20a4 in start_thread () >>from /lib/x86_64-linux-gnu/libpthread.so.0 >> #6 0x7f89898e762d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Regards > > Rüdiger >
Re: mod_http2 and Frequent wake-ups for mpm_event
On 01/27/2017 08:10 PM, Stefan Priebe - Profihost AG wrote: > Hi Yann, > > this is now a segfault with mod_http2 + beam v4 patch + mpm v7 patch: > Program terminated with signal SIGSEGV, Segmentation fault. > #0 allocator_free (node=0x0, allocator=0x7f8974080c90) > at memory/unix/apr_pools.c:381 > #0 allocator_free (node=0x0, allocator=0x7f8974080c90) > at memory/unix/apr_pools.c:381 > #1 apr_pool_clear (pool=0x7f89740830d8) at memory/unix/apr_pools.c:793 pool from above != pool_to_recycle from below is strange. > #2 0x5611421447a8 in ap_push_pool (queue_info=0x0, queue_info NULL? That looks bad. > pool_to_recycle=0x7f8974080c98) at fdqueue.c:234 > #3 0x56114213faea in process_lingering_close (cs=0x7f8974083368, > pfd=0x561142cc97f8) at event.c:1513 > #4 0x561142143620 in listener_thread (thd=0x0, dummy=0x54716c3684bbc) > at event.c:1837 > #5 0x7f8989bb20a4 in start_thread () >from /lib/x86_64-linux-gnu/libpthread.so.0 > #6 0x7f89898e762d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Regards Rüdiger
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Yann,
this is now a segfault with mod_http2 + beam v4 patch + mpm v7 patch:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f8974080c90)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f8974080c90)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f89740830d8) at memory/unix/apr_pools.c:793
#2 0x5611421447a8 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f8974080c98) at fdqueue.c:234
#3 0x56114213faea in process_lingering_close (cs=0x7f8974083368,
pfd=0x561142cc97f8) at event.c:1513
#4 0x561142143620 in listener_thread (thd=0x0, dummy=0x54716c3684bbc)
at event.c:1837
#5 0x7f8989bb20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f89898e762d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Stefan
Am 25.01.2017 um 12:10 schrieb Stefan Priebe - Profihost AG:
> Am 25.01.2017 um 10:46 schrieb Stefan Eissing:
>> Thanks Yann!
>>
>> Stefan: here is the patch as committed to trunk:
>
> Up and running. Will report back.
>
>>
>>
>> Cheers, Stefan
>>
>>> Am 25.01.2017 um 01:41 schrieb Yann Ylavic :
>>>
>>> Hi Stefan,
>>>
>>> On Tue, Jan 24, 2017 at 1:37 PM, Stefan Eissing
>>> wrote:
Yann, thanks for the patch. I agree that the cleanups need to be killed in
the right place. Not certain if it was wrong before, but that part is not
easy to see for every combination.
I did some rework and hope this makes it more readable. If you find the
time to look at it, feedback welcome.
>>>
>>> I still fear that if beam->pool gets destroyed while both
>>> beam_send_cleanup() and beam_cleanup() are registered, the former is
>>> called twice.
>>>
>>> I'd change:
>>>if (safe_send) {
>>>if (beam->send_pool && beam->send_pool != beam->pool) {
>>>apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
>>>}
>>>status = beam_send_cleanup(beam);
>>>}
>>>
>>> with:
>>>if (safe_send) {
>>>if (beam->send_pool) {
>>>if (beam->send_pool != beam->pool) {
>>>apr_pool_cleanup_kill(beam->send_pool, beam,
>>> beam_send_cleanup);
>>>}
>>>status = beam_send_cleanup(beam);
>>>}
>>>}
>>>
>>> since in the above case beam_send_cleanup is run first and sets
>>> send_pool=NULL.
>>>
>>> Attached v3 with this only change w.r.t. v2.
>>> Otherwise, looks good to me, thanks!
>>>
>>
>> Stefan Eissing
>>
>> bytes GmbH
>> Hafenstrasse 16
>> 48155 Münster
>> www.greenbytes.de
>>
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 25.01.2017 um 10:46 schrieb Stefan Eissing:
> Thanks Yann!
>
> Stefan: here is the patch as committed to trunk:
Up and running. Will report back.
>
>
> Cheers, Stefan
>
>> Am 25.01.2017 um 01:41 schrieb Yann Ylavic :
>>
>> Hi Stefan,
>>
>> On Tue, Jan 24, 2017 at 1:37 PM, Stefan Eissing
>> wrote:
>>> Yann, thanks for the patch. I agree that the cleanups need to be killed in
>>> the right place. Not certain if it was wrong before, but that part is not
>>> easy to see for every combination.
>>>
>>> I did some rework and hope this makes it more readable. If you find the
>>> time to look at it, feedback welcome.
>>
>> I still fear that if beam->pool gets destroyed while both
>> beam_send_cleanup() and beam_cleanup() are registered, the former is
>> called twice.
>>
>> I'd change:
>>if (safe_send) {
>>if (beam->send_pool && beam->send_pool != beam->pool) {
>>apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
>>}
>>status = beam_send_cleanup(beam);
>>}
>>
>> with:
>>if (safe_send) {
>>if (beam->send_pool) {
>>if (beam->send_pool != beam->pool) {
>>apr_pool_cleanup_kill(beam->send_pool, beam,
>> beam_send_cleanup);
>>}
>>status = beam_send_cleanup(beam);
>>}
>>}
>>
>> since in the above case beam_send_cleanup is run first and sets
>> send_pool=NULL.
>>
>> Attached v3 with this only change w.r.t. v2.
>> Otherwise, looks good to me, thanks!
>>
>
> Stefan Eissing
>
> bytes GmbH
> Hafenstrasse 16
> 48155 Münster
> www.greenbytes.de
>
Re: mod_http2 and Frequent wake-ups for mpm_event
Thanks Yann!
Stefan: here is the patch as committed to trunk:
h2_beams_cleanup_v4.diff
Description: Binary data
Cheers, Stefan
> Am 25.01.2017 um 01:41 schrieb Yann Ylavic :
>
> Hi Stefan,
>
> On Tue, Jan 24, 2017 at 1:37 PM, Stefan Eissing
> wrote:
>> Yann, thanks for the patch. I agree that the cleanups need to be killed in
>> the right place. Not certain if it was wrong before, but that part is not
>> easy to see for every combination.
>>
>> I did some rework and hope this makes it more readable. If you find the time
>> to look at it, feedback welcome.
>
> I still fear that if beam->pool gets destroyed while both
> beam_send_cleanup() and beam_cleanup() are registered, the former is
> called twice.
>
> I'd change:
>if (safe_send) {
>if (beam->send_pool && beam->send_pool != beam->pool) {
>apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
>}
>status = beam_send_cleanup(beam);
>}
>
> with:
>if (safe_send) {
>if (beam->send_pool) {
>if (beam->send_pool != beam->pool) {
>apr_pool_cleanup_kill(beam->send_pool, beam,
> beam_send_cleanup);
>}
>status = beam_send_cleanup(beam);
>}
>}
>
> since in the above case beam_send_cleanup is run first and sets
> send_pool=NULL.
>
> Attached v3 with this only change w.r.t. v2.
> Otherwise, looks good to me, thanks!
>
Stefan Eissing
bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
> Am 25.01.2017 um 08:04 schrieb Stefan Priebe - Profihost AG
> :
>
> Hi Yann,
>
>> Am 25.01.2017 um 01:41 schrieb Yann Ylavic:
>> Hi Stefan,
>>
>> On Tue, Jan 24, 2017 at 1:37 PM, Stefan Eissing
>> wrote:
>>> Yann, thanks for the patch. I agree that the cleanups need to be killed in
>>> the right place. Not certain if it was wrong before, but that part is not
>>> easy to see for every combination.
>>>
>>> I did some rework and hope this makes it more readable. If you find the
>>> time to look at it, feedback welcome.
>>
>> I still fear that if beam->pool gets destroyed while both
>> beam_send_cleanup() and beam_cleanup() are registered, the former is
>> called twice.
>>
>> I'd change:
>>if (safe_send) {
>>if (beam->send_pool && beam->send_pool != beam->pool) {
>>apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
>>}
>>status = beam_send_cleanup(beam);
>>}
>>
>> with:
>>if (safe_send) {
>>if (beam->send_pool) {
>>if (beam->send_pool != beam->pool) {
>>apr_pool_cleanup_kill(beam->send_pool, beam,
>> beam_send_cleanup);
>>}
>>status = beam_send_cleanup(beam);
>>}
>>}
>>
>> since in the above case beam_send_cleanup is run first and sets
>> send_pool=NULL.
>>
>> Attached v3 with this only change w.r.t. v2.
>> Otherwise, looks good to me, thanks!
>
> this one also contains changes to the even mpm, mod_proxy and
> mod_slotmem_shm. Is this expected?
Don't think so.
Yann is a busy man. :)
>
> Stefan
>>
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Yann,
Am 25.01.2017 um 01:41 schrieb Yann Ylavic:
> Hi Stefan,
>
> On Tue, Jan 24, 2017 at 1:37 PM, Stefan Eissing
> wrote:
>> Yann, thanks for the patch. I agree that the cleanups need to be killed in
>> the right place. Not certain if it was wrong before, but that part is not
>> easy to see for every combination.
>>
>> I did some rework and hope this makes it more readable. If you find the time
>> to look at it, feedback welcome.
>
> I still fear that if beam->pool gets destroyed while both
> beam_send_cleanup() and beam_cleanup() are registered, the former is
> called twice.
>
> I'd change:
> if (safe_send) {
> if (beam->send_pool && beam->send_pool != beam->pool) {
> apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
> }
> status = beam_send_cleanup(beam);
> }
>
> with:
> if (safe_send) {
> if (beam->send_pool) {
> if (beam->send_pool != beam->pool) {
> apr_pool_cleanup_kill(beam->send_pool, beam,
> beam_send_cleanup);
> }
> status = beam_send_cleanup(beam);
> }
> }
>
> since in the above case beam_send_cleanup is run first and sets
> send_pool=NULL.
>
> Attached v3 with this only change w.r.t. v2.
> Otherwise, looks good to me, thanks!
this one also contains changes to the even mpm, mod_proxy and
mod_slotmem_shm. Is this expected?
Stefan
>
Re: mod_http2 and Frequent wake-ups for mpm_event
On Wed, Jan 25, 2017 at 1:43 AM, Yann Ylavic wrote: > Hi, > > On Tue, Jan 24, 2017 at 4:09 PM, Stefan Priebe - Profihost AG > wrote: >> >> Thanks. Would currently wait if you had a look at stefans patch. Or >> isn't that the same but "better"? > > Could you please give "h2_beams_cleanup_v3.diff" (from previous message) a > try? (if that wasn't clear enough, I meant on top of 1.8.9 of course) > > Thanks Stefan.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, On Tue, Jan 24, 2017 at 4:09 PM, Stefan Priebe - Profihost AG wrote: > > Thanks. Would currently wait if you had a look at stefans patch. Or > isn't that the same but "better"? Could you please give "h2_beams_cleanup_v3.diff" (from previous message) a try? Thanks Stefan.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan,
On Tue, Jan 24, 2017 at 1:37 PM, Stefan Eissing
wrote:
> Yann, thanks for the patch. I agree that the cleanups need to be killed in
> the right place. Not certain if it was wrong before, but that part is not
> easy to see for every combination.
>
> I did some rework and hope this makes it more readable. If you find the time
> to look at it, feedback welcome.
I still fear that if beam->pool gets destroyed while both
beam_send_cleanup() and beam_cleanup() are registered, the former is
called twice.
I'd change:
if (safe_send) {
if (beam->send_pool && beam->send_pool != beam->pool) {
apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
}
status = beam_send_cleanup(beam);
}
with:
if (safe_send) {
if (beam->send_pool) {
if (beam->send_pool != beam->pool) {
apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
}
status = beam_send_cleanup(beam);
}
}
since in the above case beam_send_cleanup is run first and sets send_pool=NULL.
Attached v3 with this only change w.r.t. v2.
Otherwise, looks good to me, thanks!
Index: modules/http2/h2_bucket_beam.c
===
--- modules/http2/h2_bucket_beam.c (revision 1780129)
+++ modules/http2/h2_bucket_beam.c (working copy)
@@ -438,18 +438,37 @@ static apr_status_t beam_recv_cleanup(void *data)
return APR_SUCCESS;
}
+static int pool_register(h2_bucket_beam *beam, apr_pool_t *pool,
+ apr_status_t (*cleanup)(void *))
+{
+if (pool && pool != beam->pool) {
+apr_pool_pre_cleanup_register(pool, beam, cleanup);
+return 1;
+}
+return 0;
+}
+
+static int pool_kill(h2_bucket_beam *beam, apr_pool_t *pool,
+ apr_status_t (*cleanup)(void *)) {
+if (pool && pool != beam->pool) {
+apr_pool_cleanup_kill(pool, beam, cleanup);
+return 1;
+}
+return 0;
+}
+
static void beam_set_recv_pool(h2_bucket_beam *beam, apr_pool_t *pool)
{
-/* if the beam owner is the sender, monitor receiver pool lifetime */
-if (beam->owner == H2_BEAM_OWNER_SEND && beam->recv_pool != pool) {
-if (beam->recv_pool) {
-apr_pool_cleanup_kill(beam->recv_pool, beam, beam_recv_cleanup);
-}
-beam->recv_pool = pool;
-if (beam->recv_pool) {
-apr_pool_pre_cleanup_register(beam->recv_pool, beam,
beam_recv_cleanup);
-}
+if (beam->recv_pool == pool ||
+(beam->recv_pool && pool
+ && apr_pool_is_ancestor(beam->recv_pool, pool))) {
+/* when receiver same or sub-pool of existing, stick
+ * to the the pool we already have. */
+return;
}
+pool_kill(beam, beam->recv_pool, beam_recv_cleanup);
+beam->recv_pool = pool;
+pool_register(beam, beam->recv_pool, beam_recv_cleanup);
}
static apr_status_t beam_send_cleanup(void *data)
@@ -473,22 +492,16 @@ static apr_status_t beam_send_cleanup(void *data)
static void beam_set_send_pool(h2_bucket_beam *beam, apr_pool_t *pool)
{
-/* if the beam owner is the receiver, monitor sender pool lifetime */
-if (beam->owner == H2_BEAM_OWNER_RECV && beam->send_pool != pool) {
-if (beam->send_pool && pool
-&& apr_pool_is_ancestor(beam->send_pool, pool)) {
-/* when sender uses sub-pools to transmit data, stick
- * to the lifetime of the pool we already have. */
- return;
-}
-if (beam->send_pool) {
-apr_pool_cleanup_kill(beam->send_pool, beam, beam_send_cleanup);
-}
-beam->send_pool = pool;
-if (beam->send_pool) {
-apr_pool_pre_cleanup_register(beam->send_pool, beam,
beam_send_cleanup);
-}
+if (beam->send_pool == pool ||
+(beam->send_pool && pool
+ && apr_pool_is_ancestor(beam->send_pool, pool))) {
+/* when sender is same or sub-pool of existing, stick
+ * to the the pool we already have. */
+return;
}
+pool_kill(beam, beam->send_pool, beam_send_cleanup);
+beam->send_pool = pool;
+pool_register(beam, beam->send_pool, beam_send_cleanup);
}
static apr_status_t beam_cleanup(void *data)
@@ -495,44 +508,57 @@ static apr_status_t beam_cleanup(void *data)
{
h2_bucket_beam *beam = data;
apr_status_t status = APR_SUCCESS;
-/* owner of the beam is going away, depending on its role, cleanup
- * strategies differ. */
-beam_close(beam);
-switch (beam->owner) {
-case H2_BEAM_OWNER_SEND:
-status = beam_send_cleanup(beam);
-beam->recv_buffer = NULL;
+int safe_send = !beam->m_enter || (beam->owner == H2_BEAM_OWNER_SEND);
+int safe_recv = !beam->m_enter || (beam->owner == H2_BEAM_OWNER_RECV);
+
+/*
+ * Owner of the beam is going away, depending on which side it owns,
+ * cl
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 24.01.2017 um 14:22 schrieb Yann Ylavic:
> On Tue, Jan 24, 2017 at 10:02 AM, Stefan Priebe - Profihost AG
> wrote:
>>
>> Am 23.01.2017 um 23:42 schrieb Yann Ylavic:
>>> On Mon, Jan 23, 2017 at 11:37 PM, Yann Ylavic wrote:
Hi Stefan,
On Mon, Jan 23, 2017 at 9:54 PM, Stefan Eissing
wrote:
>
> I am not aware of any special expectations now. Almost all is triggered
> by (parent) pool cleanups and is therefore more deterministic than
> before. The only explicit destroy is done on finished streams and slave
> connections no longer used. When the master conn disappears, all is
> deallocated as the force wills it.
I wonder if the attached patch makes sense.
If beam_{recv,send}_cleanup() are to be executed on (parent) pool
destroy, there will be before beam_cleanup() itelf (which also calls
beam_send_cleanup() explicitly), so it should avoid potential a double
cleanup in this case.
WDYT?
>>>
>>> Please ignore the last (garbage) hunk.
>>
>> last garbage hunk?
>>
>> This one?
>
> Yes, this change is not necessary/suitable.
Thanks. Would currently wait if you had a look at stefans patch. Or
isn't that the same but "better"?
Stefan
Re: mod_http2 and Frequent wake-ups for mpm_event
On Tue, Jan 24, 2017 at 10:02 AM, Stefan Priebe - Profihost AG
wrote:
>
> Am 23.01.2017 um 23:42 schrieb Yann Ylavic:
>> On Mon, Jan 23, 2017 at 11:37 PM, Yann Ylavic wrote:
>>> Hi Stefan,
>>>
>>> On Mon, Jan 23, 2017 at 9:54 PM, Stefan Eissing
>>> wrote:
I am not aware of any special expectations now. Almost all is triggered by
(parent) pool cleanups and is therefore more deterministic than before.
The only explicit destroy is done on finished streams and slave
connections no longer used. When the master conn disappears, all is
deallocated as the force wills it.
>>>
>>> I wonder if the attached patch makes sense.
>>> If beam_{recv,send}_cleanup() are to be executed on (parent) pool
>>> destroy, there will be before beam_cleanup() itelf (which also calls
>>> beam_send_cleanup() explicitly), so it should avoid potential a double
>>> cleanup in this case.
>>>
>>> WDYT?
>>
>> Please ignore the last (garbage) hunk.
>
> last garbage hunk?
>
> This one?
Yes, this change is not necessary/suitable.
Re: mod_http2 and Frequent wake-ups for mpm_event
Yann, thanks for the patch. I agree that the cleanups need to be killed in the
right place. Not certain if it was wrong before, but that part is not easy to
see for every combination.
I did some rework and hope this makes it more readable. If you find the time to
look at it, feedback welcome.
Cheers, Stefan
h2_beams_cleanup_v2.diff
Description: Binary data
> Am 23.01.2017 um 23:37 schrieb Yann Ylavic :
>
> Hi Stefan,
>
> On Mon, Jan 23, 2017 at 9:54 PM, Stefan Eissing
> wrote:
>>
>> I am not aware of any special expectations now. Almost all is triggered by
>> (parent) pool cleanups and is therefore more deterministic than before. The
>> only explicit destroy is done on finished streams and slave connections no
>> longer used. When the master conn disappears, all is deallocated as the
>> force wills it.
>
> I wonder if the attached patch makes sense.
> If beam_{recv,send}_cleanup() are to be executed on (parent) pool
> destroy, there will be before beam_cleanup() itelf (which also calls
> beam_send_cleanup() explicitly), so it should avoid potential a double
> cleanup in this case.
>
> WDYT?
>
Stefan Eissing
bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 23.01.2017 um 23:42 schrieb Yann Ylavic:
> On Mon, Jan 23, 2017 at 11:37 PM, Yann Ylavic wrote:
>> Hi Stefan,
>>
>> On Mon, Jan 23, 2017 at 9:54 PM, Stefan Eissing
>> wrote:
>>>
>>> I am not aware of any special expectations now. Almost all is triggered by
>>> (parent) pool cleanups and is therefore more deterministic than before. The
>>> only explicit destroy is done on finished streams and slave connections no
>>> longer used. When the master conn disappears, all is deallocated as the
>>> force wills it.
>>
>> I wonder if the attached patch makes sense.
>> If beam_{recv,send}_cleanup() are to be executed on (parent) pool
>> destroy, there will be before beam_cleanup() itelf (which also calls
>> beam_send_cleanup() explicitly), so it should avoid potential a double
>> cleanup in this case.
>>
>> WDYT?
>
> Please ignore the last (garbage) hunk.
last garbage hunk?
This one?
@@ -520,8 +529,8 @@ static apr_status_t beam_cleanup(void *data)
* beam should have lost its mutex protection, meaning
* it is no longer used multi-threaded and we can safely
* purge all remaining sender buckets. */
-apr_pool_cleanup_kill(beam->send_pool, beam,
beam_send_cleanup);
ap_assert(!beam->m_enter);
+beam_set_send_pool(beam, NULL);
beam_send_cleanup(beam);
}
ap_assert(H2_BPROXY_LIST_EMPTY(&beam->proxies));
Stefan
Re: mod_http2 and Frequent wake-ups for mpm_event
On Mon, Jan 23, 2017 at 11:37 PM, Yann Ylavic wrote:
> Hi Stefan,
>
> On Mon, Jan 23, 2017 at 9:54 PM, Stefan Eissing
> wrote:
>>
>> I am not aware of any special expectations now. Almost all is triggered by
>> (parent) pool cleanups and is therefore more deterministic than before. The
>> only explicit destroy is done on finished streams and slave connections no
>> longer used. When the master conn disappears, all is deallocated as the
>> force wills it.
>
> I wonder if the attached patch makes sense.
> If beam_{recv,send}_cleanup() are to be executed on (parent) pool
> destroy, there will be before beam_cleanup() itelf (which also calls
> beam_send_cleanup() explicitly), so it should avoid potential a double
> cleanup in this case.
>
> WDYT?
Please ignore the last (garbage) hunk.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan,
On Mon, Jan 23, 2017 at 9:54 PM, Stefan Eissing
wrote:
>
> I am not aware of any special expectations now. Almost all is triggered by
> (parent) pool cleanups and is therefore more deterministic than before. The
> only explicit destroy is done on finished streams and slave connections no
> longer used. When the master conn disappears, all is deallocated as the force
> wills it.
I wonder if the attached patch makes sense.
If beam_{recv,send}_cleanup() are to be executed on (parent) pool
destroy, there will be before beam_cleanup() itelf (which also calls
beam_send_cleanup() explicitly), so it should avoid potential a double
cleanup in this case.
WDYT?
Index: modules/http2/h2_bucket_beam.c
===
--- modules/http2/h2_bucket_beam.c (revision 1779961)
+++ modules/http2/h2_bucket_beam.c (working copy)
@@ -500,16 +500,25 @@ static apr_status_t beam_cleanup(void *data)
beam_close(beam);
switch (beam->owner) {
case H2_BEAM_OWNER_SEND:
-status = beam_send_cleanup(beam);
-beam->recv_buffer = NULL;
+if (beam->send_pool) {
+apr_pool_cleanup_kill(beam->send_pool, beam,
beam_send_cleanup);
+status = beam_send_cleanup(beam);
+}
+if (beam->recv_buffer) {
+apr_brigade_destroy(beam->recv_buffer);
+beam->recv_buffer = NULL;
+}
beam->recv_pool = NULL;
break;
case H2_BEAM_OWNER_RECV:
if (beam->recv_buffer) {
apr_brigade_destroy(beam->recv_buffer);
+beam->recv_buffer = NULL;
}
-beam->recv_buffer = NULL;
-beam->recv_pool = NULL;
+if (beam->recv_pool) {
+apr_pool_cleanup_kill(beam->recv_pool, beam,
beam_recv_cleanup);
+status = beam_recv_cleanup(beam);
+}
if (!H2_BLIST_EMPTY(&beam->send_list)) {
ap_assert(beam->send_pool);
}
@@ -520,8 +529,8 @@ static apr_status_t beam_cleanup(void *data)
* beam should have lost its mutex protection, meaning
* it is no longer used multi-threaded and we can safely
* purge all remaining sender buckets. */
-apr_pool_cleanup_kill(beam->send_pool, beam,
beam_send_cleanup);
ap_assert(!beam->m_enter);
+beam_set_send_pool(beam, NULL);
beam_send_cleanup(beam);
}
ap_assert(H2_BPROXY_LIST_EMPTY(&beam->proxies));
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 23.01.2017 um 21:54 schrieb Stefan Eissing: > >> Am 22.01.2017 um 22:22 schrieb Yann Ylavic : >> >> @icing: Any special expectation in mod_h2 with regard to mpm workers >> threads' lifetime (or keepalive connections that should stay alive for >> the configured limit)? >> I see that beam buckets make use of thread local storage/keys for >> locking, and that they also handle the double cleanup like eoc buckets >> did before 1.8.9, but can't follow all the paths yet. >> Maybe something to look at there? > > - With your help, mod_http2 1.8.9 simplified cleanup by triggering it by the > connection pool clean up only. That helped. The bucket beam still register > for pool cleanup which could be done without, but I think it's a good > failsafe. > - The thread local is used for recursive locking and, once the outermost lock > is released, are supposed to be NULL again. This I would like to eliminate > one day. > - the special handling for apr_files is gone as well. That was a work-around > when shared file buckets were transported through a bucket beam. No more. > Shared file buckets are copied now. Only file buckets with refcount == 1 are > beamed now. That makes the beam the sole controller of the lifetime and makes > setasides on the master connection work without special handling. > > I am not aware of any special expectations now. Almost all is triggered by > (parent) pool cleanups and is therefore more deterministic than before. The > only explicit destroy is done on finished streams and slave connections no > longer used. When the master conn disappears, all is deallocated as the force > wills it. Thanks and + 1 - only two crashes in 48h. ,-) before i had around 2 per hour. Need some time to debug them. Don't have dumps yet. Is there an easier way to get core dumps than manually starting httpd while using systemd? ulimit -c unlimited httpd -k start Greets, Stefan > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
> Am 22.01.2017 um 22:22 schrieb Yann Ylavic : > > @icing: Any special expectation in mod_h2 with regard to mpm workers > threads' lifetime (or keepalive connections that should stay alive for > the configured limit)? > I see that beam buckets make use of thread local storage/keys for > locking, and that they also handle the double cleanup like eoc buckets > did before 1.8.9, but can't follow all the paths yet. > Maybe something to look at there? - With your help, mod_http2 1.8.9 simplified cleanup by triggering it by the connection pool clean up only. That helped. The bucket beam still register for pool cleanup which could be done without, but I think it's a good failsafe. - The thread local is used for recursive locking and, once the outermost lock is released, are supposed to be NULL again. This I would like to eliminate one day. - the special handling for apr_files is gone as well. That was a work-around when shared file buckets were transported through a bucket beam. No more. Shared file buckets are copied now. Only file buckets with refcount == 1 are beamed now. That makes the beam the sole controller of the lifetime and makes setasides on the master connection work without special handling. I am not aware of any special expectations now. Almost all is triggered by (parent) pool cleanups and is therefore more deterministic than before. The only explicit destroy is done on finished streams and slave connections no longer used. When the master conn disappears, all is deallocated as the force wills it. Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 22.01.2017 um 22:22 schrieb Yann Ylavic: > Hi Stefan, > > On Sun, Jan 22, 2017 at 8:00 PM, Stefan Priebe - Profihost AG > wrote: >> Am 22.01.2017 um 18:02 schrieb Eric Covener: >>> On Sun, Jan 22, 2017 at 11:21 AM, Stefan Priebe - Profihost AG >>> wrote: Hi Stefan, no i was mistaken customer isn't using mod_proxy - but I think this is the patch causing me problems: https://github.com/apache/httpd/commit/a61a4bd02e483fb45d433343740d0130ee3d8d5d What do you think? >>> >>> If that is the culprit, you could likely minimize it & help confirm with >>> >>> * Increase MaxSpareTheads to the current value of MaxClients (aka >>> MaxRequestWorkers) >>> * Make sure MaxRequestsPerChild is 0. >> >> Why not just revert that one? > > This commit is likely not the "culprit", but probably the one that > brings up the issue. > > It makes so that mpm event's threads/keepalive connections get > terminated more agressively/quickly on graceful restart, for the new > generation of children processes to be able to handle new connections > also more quickly. > > I agree with Eric that the next test would be to avoid "maintenance" > graceful restarts by tunning MaxSpareTheads and MaxRequestsPerChild as > suggested, mod_http2 should then expect the same behaviour from the > mpm as with 2.4.23. > > You may still reproduce the crash with "explicit" graceful restarts > (e.g. "apache[2]ctl -k graceful" or "/etc/init.d/apache2 reload"), but > if it proves to be stable otherwise the issue is still about double > cleanup/close when http2 pools/buckets are in place. Thanks for this great explanation this makes sense. Currently i'm waiting for a next crash. But didn't get one... no idea which kind of clients are able todo them - but they were very rare. 99% of the crashes are gone with mod_http2 v1.8.9. May be i should retest your V7 patch? Greets, Stefan > @icing: Any special expectation in mod_h2 with regard to mpm workers > threads' lifetime (or keepalive connections that should stay alive for > the configured limit)? > I see that beam buckets make use of thread local storage/keys for > locking, and that they also handle the double cleanup like eoc buckets > did before 1.8.9, but can't follow all the paths yet. > Maybe something to look at there? >
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan, On Sun, Jan 22, 2017 at 8:00 PM, Stefan Priebe - Profihost AG wrote: > Am 22.01.2017 um 18:02 schrieb Eric Covener: >> On Sun, Jan 22, 2017 at 11:21 AM, Stefan Priebe - Profihost AG >> wrote: >>> Hi Stefan, >>> >>> no i was mistaken customer isn't using mod_proxy - but I think this is >>> the patch causing me problems: >>> https://github.com/apache/httpd/commit/a61a4bd02e483fb45d433343740d0130ee3d8d5d >>> >>> What do you think? >>> >> >> If that is the culprit, you could likely minimize it & help confirm with >> >> * Increase MaxSpareTheads to the current value of MaxClients (aka >> MaxRequestWorkers) >> * Make sure MaxRequestsPerChild is 0. > > Why not just revert that one? This commit is likely not the "culprit", but probably the one that brings up the issue. It makes so that mpm event's threads/keepalive connections get terminated more agressively/quickly on graceful restart, for the new generation of children processes to be able to handle new connections also more quickly. I agree with Eric that the next test would be to avoid "maintenance" graceful restarts by tunning MaxSpareTheads and MaxRequestsPerChild as suggested, mod_http2 should then expect the same behaviour from the mpm as with 2.4.23. You may still reproduce the crash with "explicit" graceful restarts (e.g. "apache[2]ctl -k graceful" or "/etc/init.d/apache2 reload"), but if it proves to be stable otherwise the issue is still about double cleanup/close when http2 pools/buckets are in place. @icing: Any special expectation in mod_h2 with regard to mpm workers threads' lifetime (or keepalive connections that should stay alive for the configured limit)? I see that beam buckets make use of thread local storage/keys for locking, and that they also handle the double cleanup like eoc buckets did before 1.8.9, but can't follow all the paths yet. Maybe something to look at there?
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 22.01.2017 um 18:02 schrieb Eric Covener: > On Sun, Jan 22, 2017 at 11:21 AM, Stefan Priebe - Profihost AG > wrote: >> Hi Stefan, >> >> no i was mistaken customer isn't using mod_proxy - but I think this is >> the patch causing me problems: >> https://github.com/apache/httpd/commit/a61a4bd02e483fb45d433343740d0130ee3d8d5d >> >> What do you think? >> > > If that is the culprit, you could likely minimize it & help confirm with > > * Increase MaxSpareTheads to the current value of MaxClients (aka > MaxRequestWorkers) > * Make sure MaxRequestsPerChild is 0. Why not just revert that one? Greets, Stefan
Re: mod_http2 and Frequent wake-ups for mpm_event
On Sun, Jan 22, 2017 at 11:21 AM, Stefan Priebe - Profihost AG wrote: > Hi Stefan, > > no i was mistaken customer isn't using mod_proxy - but I think this is > the patch causing me problems: > https://github.com/apache/httpd/commit/a61a4bd02e483fb45d433343740d0130ee3d8d5d > > What do you think? > If that is the culprit, you could likely minimize it & help confirm with * Increase MaxSpareTheads to the current value of MaxClients (aka MaxRequestWorkers) * Make sure MaxRequestsPerChild is 0.
Re: mod_http2 and Frequent wake-ups for mpm_event
Not my forté. Yann probably has a better idea here. Let's see if he has time to look at it tomorrow. > Am 22.01.2017 um 17:21 schrieb Stefan Priebe - Profihost AG > : > > Hi Stefan, > > no i was mistaken customer isn't using mod_proxy - but I think this is > the patch causing me problems: > https://github.com/apache/httpd/commit/a61a4bd02e483fb45d433343740d0130ee3d8d5d > > What do you think? > > Greets, > Stefan > > Am 22.01.2017 um 17:17 schrieb Stefan Eissing: >> >>> Am 22.01.2017 um 17:14 schrieb Stefan Priebe - Profihost AG >>> : >>> >>> *arg* it's just mod_proxy - just saw thread safety and apr bucket aloc. >> >> ??? Can you elaborate? Is your finding the known hcheck bug or something >> else? >> >>> Stefan >>> >>> Am 22.01.2017 um 17:06 schrieb Stefan Priebe - Profihost AG: Looks like others have the same crashes too: https://bz.apache.org/bugzilla/show_bug.cgi?id=60071 and https://github.com/apache/httpd/commit/8e63c3c9372cd398f57357099aa941cbba695758 So it looks like mod_http2 is running fine now. Thanks a lot Stefan. Yann i think i can start testing your mpm patch again after the segfaults in 2.4 branch are fixed. Greets, Stefan Am 22.01.2017 um 13:16 schrieb Stefan Priebe: > Hi, > > and a new one but also in ap_start_lingering_close: > > Program terminated with signal SIGSEGV, Segmentation fault. > #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) > at memory/unix/apr_pools.c:684 > #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) > at memory/unix/apr_pools.c:684 > #1 0x7f456bc5d8b4 in apr_brigade_create (p=0x7f455805e138, > list=0x7f45040034e8) at buckets/apr_brigade.c:61 > #2 0x55e165efa319 in ap_shutdown_conn (c=c@entry=0x7f455805e458, > flush=flush@entry=1) at connection.c:76 > #3 0x55e165efa40d in ap_flush_conn (c=0x7f455805e458) at > connection.c:95 > #4 ap_start_lingering_close (c=0x7f455805e458) at connection.c:145 > #5 0x55e165f942dd in start_lingering_close_blocking (cs= out>) > at event.c:876 > #6 process_socket (my_thread_num=, > my_child_num=, cs=0x7f455805e3c8, sock=, > p=, thd=) at event.c:1153 > #7 worker_thread (thd=0x7f455805e138, dummy=0x20) at event.c:2001 > #8 0x7f456b80a0a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #9 0x7f456b53f62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Stefan > > Am 21.01.2017 um 19:31 schrieb Stefan Priebe: >> All last traces come from event, proces_longering_close ap_push_pool but >> end in different functions. It looks like a race somewhere and it just >> races at different function in the event of close and pool clear. >> >> Might there be two places where the same pool gets cleared? >> >> Stefan >> >> Am 21.01.2017 um 19:07 schrieb Stefan Priebe: >>> Hi Stefan, >>> >>> thanks. No crashes where h2 comes up. But i still have these and no idea >>> how to find out who and why they're crashing. >>> >>> Using host libthread_db library >>> "/lib/x86_64-linux-gnu/libthread_db.so.1". >>> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 allocator_free (node=0x0, allocator=0x7f6e08066540) >>> at memory/unix/apr_pools.c:381 >>> #0 allocator_free (node=0x0, allocator=0x7f6e08066540) >>> at memory/unix/apr_pools.c:381 >>> #1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793 >>> #2 0x004fe528 in ap_push_pool (queue_info=0x0, >>> pool_to_recycle=0x7f6e08066548) at fdqueue.c:234 >>> #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8, >>> pfd=0x1d3bf98) at event.c:1439 >>> #4 0x004fd410 in listener_thread (thd=0x1d3cb70, >>> dummy=0x7f6e0808d4c8) >>> at event.c:1704 >>> #5 0x7f6e1aed20a4 in start_thread () >>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>> #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >>> (gdb) (gdb) quit >>> >>> Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from >>> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >>> done. >>> [Thread debugging using libthread_db enabled] >>> Using host libthread_db library >>> "/lib/x86_64-linux-gnu/libthread_db.so.1". >>> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) >>> at memory/unix/apr_pools.c:381 >>> #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) >>> at memory/unix/apr_pools.c:381 >>> #1 apr_pool_clear (pool=0x7f6e08076
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan, no i was mistaken customer isn't using mod_proxy - but I think this is the patch causing me problems: https://github.com/apache/httpd/commit/a61a4bd02e483fb45d433343740d0130ee3d8d5d What do you think? Greets, Stefan Am 22.01.2017 um 17:17 schrieb Stefan Eissing: > >> Am 22.01.2017 um 17:14 schrieb Stefan Priebe - Profihost AG >> : >> >> *arg* it's just mod_proxy - just saw thread safety and apr bucket aloc. > > ??? Can you elaborate? Is your finding the known hcheck bug or something else? > >> Stefan >> >> Am 22.01.2017 um 17:06 schrieb Stefan Priebe - Profihost AG: >>> Looks like others have the same crashes too: >>> https://bz.apache.org/bugzilla/show_bug.cgi?id=60071 >>> and >>> https://github.com/apache/httpd/commit/8e63c3c9372cd398f57357099aa941cbba695758 >>> >>> So it looks like mod_http2 is running fine now. Thanks a lot Stefan. >>> >>> Yann i think i can start testing your mpm patch again after the >>> segfaults in 2.4 branch are fixed. >>> >>> Greets, >>> Stefan >>> >>> Am 22.01.2017 um 13:16 schrieb Stefan Priebe: Hi, and a new one but also in ap_start_lingering_close: Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) at memory/unix/apr_pools.c:684 #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) at memory/unix/apr_pools.c:684 #1 0x7f456bc5d8b4 in apr_brigade_create (p=0x7f455805e138, list=0x7f45040034e8) at buckets/apr_brigade.c:61 #2 0x55e165efa319 in ap_shutdown_conn (c=c@entry=0x7f455805e458, flush=flush@entry=1) at connection.c:76 #3 0x55e165efa40d in ap_flush_conn (c=0x7f455805e458) at connection.c:95 #4 ap_start_lingering_close (c=0x7f455805e458) at connection.c:145 #5 0x55e165f942dd in start_lingering_close_blocking (cs=>>> out>) at event.c:876 #6 process_socket (my_thread_num=, my_child_num=, cs=0x7f455805e3c8, sock=, p=, thd=) at event.c:1153 #7 worker_thread (thd=0x7f455805e138, dummy=0x20) at event.c:2001 #8 0x7f456b80a0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #9 0x7f456b53f62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 21.01.2017 um 19:31 schrieb Stefan Priebe: > All last traces come from event, proces_longering_close ap_push_pool but > end in different functions. It looks like a race somewhere and it just > races at different function in the event of close and pool clear. > > Might there be two places where the same pool gets cleared? > > Stefan > > Am 21.01.2017 um 19:07 schrieb Stefan Priebe: >> Hi Stefan, >> >> thanks. No crashes where h2 comes up. But i still have these and no idea >> how to find out who and why they're crashing. >> >> Using host libthread_db library >> "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 allocator_free (node=0x0, allocator=0x7f6e08066540) >>at memory/unix/apr_pools.c:381 >> #0 allocator_free (node=0x0, allocator=0x7f6e08066540) >>at memory/unix/apr_pools.c:381 >> #1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793 >> #2 0x004fe528 in ap_push_pool (queue_info=0x0, >>pool_to_recycle=0x7f6e08066548) at fdqueue.c:234 >> #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8, >>pfd=0x1d3bf98) at event.c:1439 >> #4 0x004fd410 in listener_thread (thd=0x1d3cb70, >> dummy=0x7f6e0808d4c8) >>at event.c:1704 >> #5 0x7f6e1aed20a4 in start_thread () >> from /lib/x86_64-linux-gnu/libpthread.so.0 >> #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> (gdb) (gdb) quit >> >> Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from >> /usr/lib/debug//usr/local/apache2/bin/httpd...done. >> done. >> [Thread debugging using libthread_db enabled] >> Using host libthread_db library >> "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) >>at memory/unix/apr_pools.c:381 >> #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) >>at memory/unix/apr_pools.c:381 >> #1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793 >> #2 0x004fe528 in ap_push_pool (queue_info=0x0, >>pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234 >> #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48, >>pfd=0x1d3bf98) at event.c:1439 >> #4 0x004fd41
Re: mod_http2 and Frequent wake-ups for mpm_event
> Am 22.01.2017 um 17:14 schrieb Stefan Priebe - Profihost AG > : > > *arg* it's just mod_proxy - just saw thread safety and apr bucket aloc. ??? Can you elaborate? Is your finding the known hcheck bug or something else? > Stefan > > Am 22.01.2017 um 17:06 schrieb Stefan Priebe - Profihost AG: >> Looks like others have the same crashes too: >> https://bz.apache.org/bugzilla/show_bug.cgi?id=60071 >> and >> https://github.com/apache/httpd/commit/8e63c3c9372cd398f57357099aa941cbba695758 >> >> So it looks like mod_http2 is running fine now. Thanks a lot Stefan. >> >> Yann i think i can start testing your mpm patch again after the >> segfaults in 2.4 branch are fixed. >> >> Greets, >> Stefan >> >> Am 22.01.2017 um 13:16 schrieb Stefan Priebe: >>> Hi, >>> >>> and a new one but also in ap_start_lingering_close: >>> >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) >>>at memory/unix/apr_pools.c:684 >>> #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) >>>at memory/unix/apr_pools.c:684 >>> #1 0x7f456bc5d8b4 in apr_brigade_create (p=0x7f455805e138, >>>list=0x7f45040034e8) at buckets/apr_brigade.c:61 >>> #2 0x55e165efa319 in ap_shutdown_conn (c=c@entry=0x7f455805e458, >>>flush=flush@entry=1) at connection.c:76 >>> #3 0x55e165efa40d in ap_flush_conn (c=0x7f455805e458) at >>> connection.c:95 >>> #4 ap_start_lingering_close (c=0x7f455805e458) at connection.c:145 >>> #5 0x55e165f942dd in start_lingering_close_blocking (cs=>> out>) >>>at event.c:876 >>> #6 process_socket (my_thread_num=, >>>my_child_num=, cs=0x7f455805e3c8, sock=, >>>p=, thd=) at event.c:1153 >>> #7 worker_thread (thd=0x7f455805e138, dummy=0x20) at event.c:2001 >>> #8 0x7f456b80a0a4 in start_thread () >>> from /lib/x86_64-linux-gnu/libpthread.so.0 >>> #9 0x7f456b53f62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >>> >>> Stefan >>> >>> Am 21.01.2017 um 19:31 schrieb Stefan Priebe: All last traces come from event, proces_longering_close ap_push_pool but end in different functions. It looks like a race somewhere and it just races at different function in the event of close and pool clear. Might there be two places where the same pool gets cleared? Stefan Am 21.01.2017 um 19:07 schrieb Stefan Priebe: > Hi Stefan, > > thanks. No crashes where h2 comes up. But i still have these and no idea > how to find out who and why they're crashing. > > Using host libthread_db library > "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 allocator_free (node=0x0, allocator=0x7f6e08066540) >at memory/unix/apr_pools.c:381 > #0 allocator_free (node=0x0, allocator=0x7f6e08066540) >at memory/unix/apr_pools.c:381 > #1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793 > #2 0x004fe528 in ap_push_pool (queue_info=0x0, >pool_to_recycle=0x7f6e08066548) at fdqueue.c:234 > #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8, >pfd=0x1d3bf98) at event.c:1439 > #4 0x004fd410 in listener_thread (thd=0x1d3cb70, > dummy=0x7f6e0808d4c8) >at event.c:1704 > #5 0x7f6e1aed20a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > (gdb) (gdb) quit > > Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from > /usr/lib/debug//usr/local/apache2/bin/httpd...done. > done. > [Thread debugging using libthread_db enabled] > Using host libthread_db library > "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) >at memory/unix/apr_pools.c:381 > #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) >at memory/unix/apr_pools.c:381 > #1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793 > #2 0x004fe528 in ap_push_pool (queue_info=0x0, >pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234 > #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48, >pfd=0x1d3bf98) at event.c:1439 > #4 0x004fd410 in listener_thread (thd=0x1d3cb70, > dummy=0x7f6e08076e48) >at event.c:1704 > #5 0x7f6e1aed20a4 in start_thread () > from /lib/x86_64-linux-gnu/libpthread.so.0 > #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > (gdb) (gdb) quit > > Stefan > > Am 21.01.2017 um 17:03 schrieb Stefan Eissing:
Re: mod_http2 and Frequent wake-ups for mpm_event
*arg* it's just mod_proxy - just saw thread safety and apr bucket aloc. Stefan Am 22.01.2017 um 17:06 schrieb Stefan Priebe - Profihost AG: > Looks like others have the same crashes too: > https://bz.apache.org/bugzilla/show_bug.cgi?id=60071 > and > https://github.com/apache/httpd/commit/8e63c3c9372cd398f57357099aa941cbba695758 > > So it looks like mod_http2 is running fine now. Thanks a lot Stefan. > > Yann i think i can start testing your mpm patch again after the > segfaults in 2.4 branch are fixed. > > Greets, > Stefan > > Am 22.01.2017 um 13:16 schrieb Stefan Priebe: >> Hi, >> >> and a new one but also in ap_start_lingering_close: >> >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) >> at memory/unix/apr_pools.c:684 >> #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32) >> at memory/unix/apr_pools.c:684 >> #1 0x7f456bc5d8b4 in apr_brigade_create (p=0x7f455805e138, >> list=0x7f45040034e8) at buckets/apr_brigade.c:61 >> #2 0x55e165efa319 in ap_shutdown_conn (c=c@entry=0x7f455805e458, >> flush=flush@entry=1) at connection.c:76 >> #3 0x55e165efa40d in ap_flush_conn (c=0x7f455805e458) at >> connection.c:95 >> #4 ap_start_lingering_close (c=0x7f455805e458) at connection.c:145 >> #5 0x55e165f942dd in start_lingering_close_blocking (cs=> out>) >> at event.c:876 >> #6 process_socket (my_thread_num=, >> my_child_num=, cs=0x7f455805e3c8, sock=, >> p=, thd=) at event.c:1153 >> #7 worker_thread (thd=0x7f455805e138, dummy=0x20) at event.c:2001 >> #8 0x7f456b80a0a4 in start_thread () >>from /lib/x86_64-linux-gnu/libpthread.so.0 >> #9 0x7f456b53f62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> Stefan >> >> Am 21.01.2017 um 19:31 schrieb Stefan Priebe: >>> All last traces come from event, proces_longering_close ap_push_pool but >>> end in different functions. It looks like a race somewhere and it just >>> races at different function in the event of close and pool clear. >>> >>> Might there be two places where the same pool gets cleared? >>> >>> Stefan >>> >>> Am 21.01.2017 um 19:07 schrieb Stefan Priebe: Hi Stefan, thanks. No crashes where h2 comes up. But i still have these and no idea how to find out who and why they're crashing. Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x0, allocator=0x7f6e08066540) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x0, allocator=0x7f6e08066540) at memory/unix/apr_pools.c:381 #1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793 #2 0x004fe528 in ap_push_pool (queue_info=0x0, pool_to_recycle=0x7f6e08066548) at fdqueue.c:234 #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8, pfd=0x1d3bf98) at event.c:1439 #4 0x004fd410 in listener_thread (thd=0x1d3cb70, dummy=0x7f6e0808d4c8) at event.c:1704 #5 0x7f6e1aed20a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) (gdb) quit Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from /usr/lib/debug//usr/local/apache2/bin/httpd...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) at memory/unix/apr_pools.c:381 #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0) at memory/unix/apr_pools.c:381 #1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793 #2 0x004fe528 in ap_push_pool (queue_info=0x0, pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234 #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48, pfd=0x1d3bf98) at event.c:1439 #4 0x004fd410 in listener_thread (thd=0x1d3cb70, dummy=0x7f6e08076e48) at event.c:1704 #5 0x7f6e1aed20a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) (gdb) quit Stefan Am 21.01.2017 um 17:03 schrieb Stefan Eissing: > Stefan, > > made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9 > with all patches and improved (hopefully) on them a bit. If you dare > to drop that into your installation, that'd be great. > > Cheers, > >
Re: mod_http2 and Frequent wake-ups for mpm_event
Looks like others have the same crashes too:
https://bz.apache.org/bugzilla/show_bug.cgi?id=60071
and
https://github.com/apache/httpd/commit/8e63c3c9372cd398f57357099aa941cbba695758
So it looks like mod_http2 is running fine now. Thanks a lot Stefan.
Yann i think i can start testing your mpm patch again after the
segfaults in 2.4 branch are fixed.
Greets,
Stefan
Am 22.01.2017 um 13:16 schrieb Stefan Priebe:
> Hi,
>
> and a new one but also in ap_start_lingering_close:
>
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32)
> at memory/unix/apr_pools.c:684
> #0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32)
> at memory/unix/apr_pools.c:684
> #1 0x7f456bc5d8b4 in apr_brigade_create (p=0x7f455805e138,
> list=0x7f45040034e8) at buckets/apr_brigade.c:61
> #2 0x55e165efa319 in ap_shutdown_conn (c=c@entry=0x7f455805e458,
> flush=flush@entry=1) at connection.c:76
> #3 0x55e165efa40d in ap_flush_conn (c=0x7f455805e458) at
> connection.c:95
> #4 ap_start_lingering_close (c=0x7f455805e458) at connection.c:145
> #5 0x55e165f942dd in start_lingering_close_blocking (cs= out>)
> at event.c:876
> #6 process_socket (my_thread_num=,
> my_child_num=, cs=0x7f455805e3c8, sock=,
> p=, thd=) at event.c:1153
> #7 worker_thread (thd=0x7f455805e138, dummy=0x20) at event.c:2001
> #8 0x7f456b80a0a4 in start_thread ()
>from /lib/x86_64-linux-gnu/libpthread.so.0
> #9 0x7f456b53f62d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>
> Stefan
>
> Am 21.01.2017 um 19:31 schrieb Stefan Priebe:
>> All last traces come from event, proces_longering_close ap_push_pool but
>> end in different functions. It looks like a race somewhere and it just
>> races at different function in the event of close and pool clear.
>>
>> Might there be two places where the same pool gets cleared?
>>
>> Stefan
>>
>> Am 21.01.2017 um 19:07 schrieb Stefan Priebe:
>>> Hi Stefan,
>>>
>>> thanks. No crashes where h2 comes up. But i still have these and no idea
>>> how to find out who and why they're crashing.
>>>
>>> Using host libthread_db library
>>> "/lib/x86_64-linux-gnu/libthread_db.so.1".
>>> Core was generated by `/usr/local/apache2/bin/httpd -k start'.
>>> Program terminated with signal SIGSEGV, Segmentation fault.
>>> #0 allocator_free (node=0x0, allocator=0x7f6e08066540)
>>> at memory/unix/apr_pools.c:381
>>> #0 allocator_free (node=0x0, allocator=0x7f6e08066540)
>>> at memory/unix/apr_pools.c:381
>>> #1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793
>>> #2 0x004fe528 in ap_push_pool (queue_info=0x0,
>>> pool_to_recycle=0x7f6e08066548) at fdqueue.c:234
>>> #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8,
>>> pfd=0x1d3bf98) at event.c:1439
>>> #4 0x004fd410 in listener_thread (thd=0x1d3cb70,
>>> dummy=0x7f6e0808d4c8)
>>> at event.c:1704
>>> #5 0x7f6e1aed20a4 in start_thread ()
>>>from /lib/x86_64-linux-gnu/libpthread.so.0
>>> #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>>> (gdb) (gdb) quit
>>>
>>> Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from
>>> /usr/lib/debug//usr/local/apache2/bin/httpd...done.
>>> done.
>>> [Thread debugging using libthread_db enabled]
>>> Using host libthread_db library
>>> "/lib/x86_64-linux-gnu/libthread_db.so.1".
>>> Core was generated by `/usr/local/apache2/bin/httpd -k start'.
>>> Program terminated with signal SIGSEGV, Segmentation fault.
>>> #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
>>> at memory/unix/apr_pools.c:381
>>> #0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
>>> at memory/unix/apr_pools.c:381
>>> #1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793
>>> #2 0x004fe528 in ap_push_pool (queue_info=0x0,
>>> pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234
>>> #3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48,
>>> pfd=0x1d3bf98) at event.c:1439
>>> #4 0x004fd410 in listener_thread (thd=0x1d3cb70,
>>> dummy=0x7f6e08076e48)
>>> at event.c:1704
>>> #5 0x7f6e1aed20a4 in start_thread ()
>>>from /lib/x86_64-linux-gnu/libpthread.so.0
>>> #6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
>>> (gdb) (gdb) quit
>>>
>>> Stefan
>>>
>>> Am 21.01.2017 um 17:03 schrieb Stefan Eissing:
Stefan,
made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9
with all patches and improved (hopefully) on them a bit. If you dare
to drop that into your installation, that'd be great.
Cheers,
Stefan
> Am 21.01.2017 um 15:25 schrieb Stefan Priebe :
>
> and i got another crash here:
>
> 2346 static void run_cleanups(cleanup_t **cref)
> 2347 {
> 2348 cleanup_t *c = *cref;
> 2349
> 2350 while (c) {
> 2351
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi,
and a new one but also in ap_start_lingering_close:
Program terminated with signal SIGSEGV, Segmentation fault.
#0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32)
at memory/unix/apr_pools.c:684
#0 apr_palloc (pool=pool@entry=0x7f455805e138, in_size=in_size@entry=32)
at memory/unix/apr_pools.c:684
#1 0x7f456bc5d8b4 in apr_brigade_create (p=0x7f455805e138,
list=0x7f45040034e8) at buckets/apr_brigade.c:61
#2 0x55e165efa319 in ap_shutdown_conn (c=c@entry=0x7f455805e458,
flush=flush@entry=1) at connection.c:76
#3 0x55e165efa40d in ap_flush_conn (c=0x7f455805e458) at
connection.c:95
#4 ap_start_lingering_close (c=0x7f455805e458) at connection.c:145
#5 0x55e165f942dd in start_lingering_close_blocking (cs=out>)
at event.c:876
#6 process_socket (my_thread_num=,
my_child_num=, cs=0x7f455805e3c8, sock=,
p=, thd=) at event.c:1153
#7 worker_thread (thd=0x7f455805e138, dummy=0x20) at event.c:2001
#8 0x7f456b80a0a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#9 0x7f456b53f62d in clone () from /lib/x86_64-linux-gnu/libc.so.6
Stefan
Am 21.01.2017 um 19:31 schrieb Stefan Priebe:
All last traces come from event, proces_longering_close ap_push_pool but
end in different functions. It looks like a race somewhere and it just
races at different function in the event of close and pool clear.
Might there be two places where the same pool gets cleared?
Stefan
Am 21.01.2017 um 19:07 schrieb Stefan Priebe:
Hi Stefan,
thanks. No crashes where h2 comes up. But i still have these and no idea
how to find out who and why they're crashing.
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f6e08066540)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f6e08066540)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793
#2 0x004fe528 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f6e08066548) at fdqueue.c:234
#3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8,
pfd=0x1d3bf98) at event.c:1439
#4 0x004fd410 in listener_thread (thd=0x1d3cb70,
dummy=0x7f6e0808d4c8)
at event.c:1704
#5 0x7f6e1aed20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) (gdb) quit
Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from
/usr/lib/debug//usr/local/apache2/bin/httpd...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793
#2 0x004fe528 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234
#3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48,
pfd=0x1d3bf98) at event.c:1439
#4 0x004fd410 in listener_thread (thd=0x1d3cb70,
dummy=0x7f6e08076e48)
at event.c:1704
#5 0x7f6e1aed20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) (gdb) quit
Stefan
Am 21.01.2017 um 17:03 schrieb Stefan Eissing:
Stefan,
made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9
with all patches and improved (hopefully) on them a bit. If you dare
to drop that into your installation, that'd be great.
Cheers,
Stefan
Am 21.01.2017 um 15:25 schrieb Stefan Priebe :
and i got another crash here:
2346 static void run_cleanups(cleanup_t **cref)
2347 {
2348 cleanup_t *c = *cref;
2349
2350 while (c) {
2351 *cref = c->next;
2352 (*c->plain_cleanup_fn)((void *)c->data); <== here
2353 c = *cref;
2354
which looks similar to the other crash.
#0 0x7fe4bbd33e1b in run_cleanups (cref=) at
memory/unix/apr_pools.c:2352
#1 apr_pool_clear (pool=0x7fe4a804dac8) at memory/unix/apr_pools.c:772
#2 0x004feb38 in ap_push_pool
(queue_info=0x6d616e79642d3733, pool_to_recycle=0x2) at fdqueue.c:234
#3 0x004fa8d8 in process_lingering_close (cs=0x7fe4a804dd58,
pfd=0x25d3f98) at event.c:1439
Details:
(gdb) print c
$1 = (cleanup_t *) 0x7fe4a804e9f0
(gdb) print *c
$2 = {next = 0x7fe4a804e870, data = 0x6d616e79642d3733,
plain_cleanup_fn = 0x392d3734322e6369,
child_cleanup_fn = 0x617465722e722d35}
(gdb) print *c->data
Attempt to dereference a generic pointer.
(gdb) print *
Re: mod_http2 and Frequent wake-ups for mpm_event
All last traces come from event, proces_longering_close ap_push_pool but
end in different functions. It looks like a race somewhere and it just
races at different function in the event of close and pool clear.
Might there be two places where the same pool gets cleared?
Stefan
Am 21.01.2017 um 19:07 schrieb Stefan Priebe:
Hi Stefan,
thanks. No crashes where h2 comes up. But i still have these and no idea
how to find out who and why they're crashing.
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f6e08066540)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f6e08066540)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793
#2 0x004fe528 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f6e08066548) at fdqueue.c:234
#3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8,
pfd=0x1d3bf98) at event.c:1439
#4 0x004fd410 in listener_thread (thd=0x1d3cb70,
dummy=0x7f6e0808d4c8)
at event.c:1704
#5 0x7f6e1aed20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) (gdb) quit
Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from
/usr/lib/debug//usr/local/apache2/bin/httpd...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793
#2 0x004fe528 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234
#3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48,
pfd=0x1d3bf98) at event.c:1439
#4 0x004fd410 in listener_thread (thd=0x1d3cb70,
dummy=0x7f6e08076e48)
at event.c:1704
#5 0x7f6e1aed20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) (gdb) quit
Stefan
Am 21.01.2017 um 17:03 schrieb Stefan Eissing:
Stefan,
made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9
with all patches and improved (hopefully) on them a bit. If you dare
to drop that into your installation, that'd be great.
Cheers,
Stefan
Am 21.01.2017 um 15:25 schrieb Stefan Priebe :
and i got another crash here:
2346 static void run_cleanups(cleanup_t **cref)
2347 {
2348 cleanup_t *c = *cref;
2349
2350 while (c) {
2351 *cref = c->next;
2352 (*c->plain_cleanup_fn)((void *)c->data); <== here
2353 c = *cref;
2354
which looks similar to the other crash.
#0 0x7fe4bbd33e1b in run_cleanups (cref=) at
memory/unix/apr_pools.c:2352
#1 apr_pool_clear (pool=0x7fe4a804dac8) at memory/unix/apr_pools.c:772
#2 0x004feb38 in ap_push_pool
(queue_info=0x6d616e79642d3733, pool_to_recycle=0x2) at fdqueue.c:234
#3 0x004fa8d8 in process_lingering_close (cs=0x7fe4a804dd58,
pfd=0x25d3f98) at event.c:1439
Details:
(gdb) print c
$1 = (cleanup_t *) 0x7fe4a804e9f0
(gdb) print *c
$2 = {next = 0x7fe4a804e870, data = 0x6d616e79642d3733,
plain_cleanup_fn = 0x392d3734322e6369,
child_cleanup_fn = 0x617465722e722d35}
(gdb) print *c->data
Attempt to dereference a generic pointer.
(gdb) print *c->plain_cleanup_fn
Cannot access memory at address 0x392d3734322e6369
(gdb)
Stefan
Am 21.01.2017 um 15:18 schrieb Stefan Priebe:
Hi,
#0 apr_pool_cleanup_kill (p=0x7fe4a8072358,
data=data@entry=0x7fe4a80723e0,
cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at
memory/unix/apr_pools.c:2276
it crashes here in apr:
2276 if (c->data == data && c->plain_cleanup_fn ==
cleanup_fn) {
some lines before c becomes this
2264 c = p->cleanups;
p is:
(gdb) print *p
$1 = {parent = 0x256f138, child = 0x7fe46c0751c8, sibling =
0x7fe4a8096888, ref = 0x7fe4a8069fe8, cleanups = 0x7fe478159748,
free_cleanups = 0x7fe478159788, allocator = 0x7fe4a803b490,
subprocesses = 0x0, abort_fn = 0x43da00 ,
user_data = 0x0, tag = 0x502285 "transaction", active =
0x7fe478158d70, self = 0x7fe4a8072330,
self_first_avail = 0x7fe4a80723d0 "X#\a\250\344\177", pre_cleanups =
0x7fe4a8072ab8}
wouldn't the error mean that p->cleanups is NULL?
(gdb) print *p->cleanups
$2 = {next = 0x7fe478159628, data = 0x7fe478159648, plain_cleanup_fn =
0x7fe4bbd2ffd0 ,
child_cleanup_fn = 0x7fe4bbd2ff70 }
So p->cleanups->data is 0x7fe478159648 and data is 0x7fe4a80723e0?
I don't get why it's segfaulting
Stef
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan,
thanks. No crashes where h2 comes up. But i still have these and no idea
how to find out who and why they're crashing.
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f6e08066540)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f6e08066540)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f6e0808d238) at memory/unix/apr_pools.c:793
#2 0x004fe528 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f6e08066548) at fdqueue.c:234
#3 0x004fa2c8 in process_lingering_close (cs=0x7f6e0808d4c8,
pfd=0x1d3bf98) at event.c:1439
#4 0x004fd410 in listener_thread (thd=0x1d3cb70,
dummy=0x7f6e0808d4c8)
at event.c:1704
#5 0x7f6e1aed20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) (gdb) quit
Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from
/usr/lib/debug//usr/local/apache2/bin/httpd...done.
done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/local/apache2/bin/httpd -k start'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
at memory/unix/apr_pools.c:381
#0 allocator_free (node=0x0, allocator=0x7f6e08053ae0)
at memory/unix/apr_pools.c:381
#1 apr_pool_clear (pool=0x7f6e08076bb8) at memory/unix/apr_pools.c:793
#2 0x004fe528 in ap_push_pool (queue_info=0x0,
pool_to_recycle=0x7f6e08053ae8) at fdqueue.c:234
#3 0x004fa2c8 in process_lingering_close (cs=0x7f6e08076e48,
pfd=0x1d3bf98) at event.c:1439
#4 0x004fd410 in listener_thread (thd=0x1d3cb70,
dummy=0x7f6e08076e48)
at event.c:1704
#5 0x7f6e1aed20a4 in start_thread ()
from /lib/x86_64-linux-gnu/libpthread.so.0
#6 0x7f6e1aa0362d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) (gdb) quit
Stefan
Am 21.01.2017 um 17:03 schrieb Stefan Eissing:
Stefan,
made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9 with all
patches and improved (hopefully) on them a bit. If you dare to drop that into
your installation, that'd be great.
Cheers,
Stefan
Am 21.01.2017 um 15:25 schrieb Stefan Priebe :
and i got another crash here:
2346 static void run_cleanups(cleanup_t **cref)
2347 {
2348 cleanup_t *c = *cref;
2349
2350 while (c) {
2351 *cref = c->next;
2352 (*c->plain_cleanup_fn)((void *)c->data); <== here
2353 c = *cref;
2354
which looks similar to the other crash.
#0 0x7fe4bbd33e1b in run_cleanups (cref=) at
memory/unix/apr_pools.c:2352
#1 apr_pool_clear (pool=0x7fe4a804dac8) at memory/unix/apr_pools.c:772
#2 0x004feb38 in ap_push_pool (queue_info=0x6d616e79642d3733,
pool_to_recycle=0x2) at fdqueue.c:234
#3 0x004fa8d8 in process_lingering_close (cs=0x7fe4a804dd58,
pfd=0x25d3f98) at event.c:1439
Details:
(gdb) print c
$1 = (cleanup_t *) 0x7fe4a804e9f0
(gdb) print *c
$2 = {next = 0x7fe4a804e870, data = 0x6d616e79642d3733, plain_cleanup_fn =
0x392d3734322e6369,
child_cleanup_fn = 0x617465722e722d35}
(gdb) print *c->data
Attempt to dereference a generic pointer.
(gdb) print *c->plain_cleanup_fn
Cannot access memory at address 0x392d3734322e6369
(gdb)
Stefan
Am 21.01.2017 um 15:18 schrieb Stefan Priebe:
Hi,
#0 apr_pool_cleanup_kill (p=0x7fe4a8072358,
data=data@entry=0x7fe4a80723e0,
cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at
memory/unix/apr_pools.c:2276
it crashes here in apr:
2276 if (c->data == data && c->plain_cleanup_fn == cleanup_fn) {
some lines before c becomes this
2264 c = p->cleanups;
p is:
(gdb) print *p
$1 = {parent = 0x256f138, child = 0x7fe46c0751c8, sibling =
0x7fe4a8096888, ref = 0x7fe4a8069fe8, cleanups = 0x7fe478159748,
free_cleanups = 0x7fe478159788, allocator = 0x7fe4a803b490,
subprocesses = 0x0, abort_fn = 0x43da00 ,
user_data = 0x0, tag = 0x502285 "transaction", active =
0x7fe478158d70, self = 0x7fe4a8072330,
self_first_avail = 0x7fe4a80723d0 "X#\a\250\344\177", pre_cleanups =
0x7fe4a8072ab8}
wouldn't the error mean that p->cleanups is NULL?
(gdb) print *p->cleanups
$2 = {next = 0x7fe478159628, data = 0x7fe478159648, plain_cleanup_fn =
0x7fe4bbd2ffd0 ,
child_cleanup_fn = 0x7fe4bbd2ff70 }
So p->cleanups->data is 0x7fe478159648 and data is 0x7fe4a80723e0?
I don't get why it's segfaulting
Stefan
Am 21.01.2017 um 09:50 schrieb Yann Ylavic:
Hi Stefan,
On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe
wrote:
after running the whole night. These are the only ones still happening.
Should i revert the mpm patch to check whether it's the source?
Yes please, we need to determine...
Thanks,
Yann.
Stefan
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 21.01.2017 um 17:03 schrieb Stefan Eissing:
Stefan,
made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9 with all
patches and improved (hopefully) on them a bit. If you dare to drop that into
your installation, that'd be great.
thx it's running - will report back shortly.
Stefan
Cheers,
Stefan
Am 21.01.2017 um 15:25 schrieb Stefan Priebe :
and i got another crash here:
2346 static void run_cleanups(cleanup_t **cref)
2347 {
2348 cleanup_t *c = *cref;
2349
2350 while (c) {
2351 *cref = c->next;
2352 (*c->plain_cleanup_fn)((void *)c->data); <== here
2353 c = *cref;
2354
which looks similar to the other crash.
#0 0x7fe4bbd33e1b in run_cleanups (cref=) at
memory/unix/apr_pools.c:2352
#1 apr_pool_clear (pool=0x7fe4a804dac8) at memory/unix/apr_pools.c:772
#2 0x004feb38 in ap_push_pool (queue_info=0x6d616e79642d3733,
pool_to_recycle=0x2) at fdqueue.c:234
#3 0x004fa8d8 in process_lingering_close (cs=0x7fe4a804dd58,
pfd=0x25d3f98) at event.c:1439
Details:
(gdb) print c
$1 = (cleanup_t *) 0x7fe4a804e9f0
(gdb) print *c
$2 = {next = 0x7fe4a804e870, data = 0x6d616e79642d3733, plain_cleanup_fn =
0x392d3734322e6369,
child_cleanup_fn = 0x617465722e722d35}
(gdb) print *c->data
Attempt to dereference a generic pointer.
(gdb) print *c->plain_cleanup_fn
Cannot access memory at address 0x392d3734322e6369
(gdb)
Stefan
Am 21.01.2017 um 15:18 schrieb Stefan Priebe:
Hi,
#0 apr_pool_cleanup_kill (p=0x7fe4a8072358,
data=data@entry=0x7fe4a80723e0,
cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at
memory/unix/apr_pools.c:2276
it crashes here in apr:
2276 if (c->data == data && c->plain_cleanup_fn == cleanup_fn) {
some lines before c becomes this
2264 c = p->cleanups;
p is:
(gdb) print *p
$1 = {parent = 0x256f138, child = 0x7fe46c0751c8, sibling =
0x7fe4a8096888, ref = 0x7fe4a8069fe8, cleanups = 0x7fe478159748,
free_cleanups = 0x7fe478159788, allocator = 0x7fe4a803b490,
subprocesses = 0x0, abort_fn = 0x43da00 ,
user_data = 0x0, tag = 0x502285 "transaction", active =
0x7fe478158d70, self = 0x7fe4a8072330,
self_first_avail = 0x7fe4a80723d0 "X#\a\250\344\177", pre_cleanups =
0x7fe4a8072ab8}
wouldn't the error mean that p->cleanups is NULL?
(gdb) print *p->cleanups
$2 = {next = 0x7fe478159628, data = 0x7fe478159648, plain_cleanup_fn =
0x7fe4bbd2ffd0 ,
child_cleanup_fn = 0x7fe4bbd2ff70 }
So p->cleanups->data is 0x7fe478159648 and data is 0x7fe4a80723e0?
I don't get why it's segfaulting
Stefan
Am 21.01.2017 um 09:50 schrieb Yann Ylavic:
Hi Stefan,
On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe
wrote:
after running the whole night. These are the only ones still happening.
Should i revert the mpm patch to check whether it's the source?
Yes please, we need to determine...
Thanks,
Yann.
Stefan Eissing
bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Stefan,
made a release at https://github.com/icing/mod_h2/releases/tag/v1.8.9 with all
patches and improved (hopefully) on them a bit. If you dare to drop that into
your installation, that'd be great.
Cheers,
Stefan
> Am 21.01.2017 um 15:25 schrieb Stefan Priebe :
>
> and i got another crash here:
>
> 2346 static void run_cleanups(cleanup_t **cref)
> 2347 {
> 2348 cleanup_t *c = *cref;
> 2349
> 2350 while (c) {
> 2351 *cref = c->next;
> 2352 (*c->plain_cleanup_fn)((void *)c->data); <== here
> 2353 c = *cref;
> 2354
>
> which looks similar to the other crash.
>
> #0 0x7fe4bbd33e1b in run_cleanups (cref=) at
> memory/unix/apr_pools.c:2352
> #1 apr_pool_clear (pool=0x7fe4a804dac8) at memory/unix/apr_pools.c:772
> #2 0x004feb38 in ap_push_pool (queue_info=0x6d616e79642d3733,
> pool_to_recycle=0x2) at fdqueue.c:234
> #3 0x004fa8d8 in process_lingering_close (cs=0x7fe4a804dd58,
> pfd=0x25d3f98) at event.c:1439
>
> Details:
> (gdb) print c
> $1 = (cleanup_t *) 0x7fe4a804e9f0
> (gdb) print *c
> $2 = {next = 0x7fe4a804e870, data = 0x6d616e79642d3733, plain_cleanup_fn =
> 0x392d3734322e6369,
> child_cleanup_fn = 0x617465722e722d35}
> (gdb) print *c->data
> Attempt to dereference a generic pointer.
> (gdb) print *c->plain_cleanup_fn
> Cannot access memory at address 0x392d3734322e6369
> (gdb)
>
> Stefan
>
> Am 21.01.2017 um 15:18 schrieb Stefan Priebe:
>> Hi,
>>
>> #0 apr_pool_cleanup_kill (p=0x7fe4a8072358,
>> data=data@entry=0x7fe4a80723e0,
>>cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at
>> memory/unix/apr_pools.c:2276
>>
>> it crashes here in apr:
>> 2276 if (c->data == data && c->plain_cleanup_fn == cleanup_fn) {
>>
>> some lines before c becomes this
>> 2264 c = p->cleanups;
>>
>> p is:
>> (gdb) print *p
>> $1 = {parent = 0x256f138, child = 0x7fe46c0751c8, sibling =
>> 0x7fe4a8096888, ref = 0x7fe4a8069fe8, cleanups = 0x7fe478159748,
>> free_cleanups = 0x7fe478159788, allocator = 0x7fe4a803b490,
>> subprocesses = 0x0, abort_fn = 0x43da00 ,
>> user_data = 0x0, tag = 0x502285 "transaction", active =
>> 0x7fe478158d70, self = 0x7fe4a8072330,
>> self_first_avail = 0x7fe4a80723d0 "X#\a\250\344\177", pre_cleanups =
>> 0x7fe4a8072ab8}
>>
>> wouldn't the error mean that p->cleanups is NULL?
>>
>> (gdb) print *p->cleanups
>> $2 = {next = 0x7fe478159628, data = 0x7fe478159648, plain_cleanup_fn =
>> 0x7fe4bbd2ffd0 ,
>> child_cleanup_fn = 0x7fe4bbd2ff70 }
>>
>> So p->cleanups->data is 0x7fe478159648 and data is 0x7fe4a80723e0?
>>
>> I don't get why it's segfaulting
>>
>> Stefan
>> Am 21.01.2017 um 09:50 schrieb Yann Ylavic:
>>> Hi Stefan,
>>>
>>> On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe
>>> wrote:
after running the whole night. These are the only ones still happening.
Should i revert the mpm patch to check whether it's the source?
>>>
>>> Yes please, we need to determine...
>>>
>>> Thanks,
>>> Yann.
>>>
Stefan Eissing
bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
and i got another crash here:
2346 static void run_cleanups(cleanup_t **cref)
2347 {
2348 cleanup_t *c = *cref;
2349
2350 while (c) {
2351 *cref = c->next;
2352 (*c->plain_cleanup_fn)((void *)c->data); <== here
2353 c = *cref;
2354
which looks similar to the other crash.
#0 0x7fe4bbd33e1b in run_cleanups (cref=) at
memory/unix/apr_pools.c:2352
#1 apr_pool_clear (pool=0x7fe4a804dac8) at memory/unix/apr_pools.c:772
#2 0x004feb38 in ap_push_pool (queue_info=0x6d616e79642d3733,
pool_to_recycle=0x2) at fdqueue.c:234
#3 0x004fa8d8 in process_lingering_close (cs=0x7fe4a804dd58,
pfd=0x25d3f98) at event.c:1439
Details:
(gdb) print c
$1 = (cleanup_t *) 0x7fe4a804e9f0
(gdb) print *c
$2 = {next = 0x7fe4a804e870, data = 0x6d616e79642d3733, plain_cleanup_fn
= 0x392d3734322e6369,
child_cleanup_fn = 0x617465722e722d35}
(gdb) print *c->data
Attempt to dereference a generic pointer.
(gdb) print *c->plain_cleanup_fn
Cannot access memory at address 0x392d3734322e6369
(gdb)
Stefan
Am 21.01.2017 um 15:18 schrieb Stefan Priebe:
Hi,
#0 apr_pool_cleanup_kill (p=0x7fe4a8072358,
data=data@entry=0x7fe4a80723e0,
cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at
memory/unix/apr_pools.c:2276
it crashes here in apr:
2276 if (c->data == data && c->plain_cleanup_fn == cleanup_fn) {
some lines before c becomes this
2264 c = p->cleanups;
p is:
(gdb) print *p
$1 = {parent = 0x256f138, child = 0x7fe46c0751c8, sibling =
0x7fe4a8096888, ref = 0x7fe4a8069fe8, cleanups = 0x7fe478159748,
free_cleanups = 0x7fe478159788, allocator = 0x7fe4a803b490,
subprocesses = 0x0, abort_fn = 0x43da00 ,
user_data = 0x0, tag = 0x502285 "transaction", active =
0x7fe478158d70, self = 0x7fe4a8072330,
self_first_avail = 0x7fe4a80723d0 "X#\a\250\344\177", pre_cleanups =
0x7fe4a8072ab8}
wouldn't the error mean that p->cleanups is NULL?
(gdb) print *p->cleanups
$2 = {next = 0x7fe478159628, data = 0x7fe478159648, plain_cleanup_fn =
0x7fe4bbd2ffd0 ,
child_cleanup_fn = 0x7fe4bbd2ff70 }
So p->cleanups->data is 0x7fe478159648 and data is 0x7fe4a80723e0?
I don't get why it's segfaulting
Stefan
Am 21.01.2017 um 09:50 schrieb Yann Ylavic:
Hi Stefan,
On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe
wrote:
after running the whole night. These are the only ones still happening.
Should i revert the mpm patch to check whether it's the source?
Yes please, we need to determine...
Thanks,
Yann.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi,
#0 apr_pool_cleanup_kill (p=0x7fe4a8072358,
data=data@entry=0x7fe4a80723e0,
cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at
memory/unix/apr_pools.c:2276
it crashes here in apr:
2276 if (c->data == data && c->plain_cleanup_fn == cleanup_fn) {
some lines before c becomes this
2264 c = p->cleanups;
p is:
(gdb) print *p
$1 = {parent = 0x256f138, child = 0x7fe46c0751c8, sibling =
0x7fe4a8096888, ref = 0x7fe4a8069fe8, cleanups = 0x7fe478159748,
free_cleanups = 0x7fe478159788, allocator = 0x7fe4a803b490,
subprocesses = 0x0, abort_fn = 0x43da00 ,
user_data = 0x0, tag = 0x502285 "transaction", active =
0x7fe478158d70, self = 0x7fe4a8072330,
self_first_avail = 0x7fe4a80723d0 "X#\a\250\344\177", pre_cleanups =
0x7fe4a8072ab8}
wouldn't the error mean that p->cleanups is NULL?
(gdb) print *p->cleanups
$2 = {next = 0x7fe478159628, data = 0x7fe478159648, plain_cleanup_fn =
0x7fe4bbd2ffd0 ,
child_cleanup_fn = 0x7fe4bbd2ff70 }
So p->cleanups->data is 0x7fe478159648 and data is 0x7fe4a80723e0?
I don't get why it's segfaulting
Stefan
Am 21.01.2017 um 09:50 schrieb Yann Ylavic:
Hi Stefan,
On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe wrote:
after running the whole night. These are the only ones still happening.
Should i revert the mpm patch to check whether it's the source?
Yes please, we need to determine...
Thanks,
Yann.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Yann, looks better so far. This is the only one i got without mpm patch: Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_pool_cleanup_kill (p=0x7fe4a808b538, data=data@entry=0x7fe4a808b5c0, cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at memory/unix/apr_pools.c:2276 #0 apr_pool_cleanup_kill (p=0x7fe4a808b538, data=data@entry=0x7fe4a808b5c0, cleanup_fn=cleanup_fn@entry=0x7fe4bbd38a40 ) at memory/unix/apr_pools.c:2276 #1 0x7fe4bbd34e91 in apr_pool_cleanup_run (p=, data=0x7fe4a808b5c0, cleanup_fn=0x7fe4bbd38a40 ) at memory/unix/apr_pools.c:2342 #2 0x7fe4bbd38d22 in apr_socket_close (thesocket=) at network_io/unix/sockets.c:183 #3 0x004fa88f in process_lingering_close (cs=0x7fe4a808b7c8, pfd=0x25d3f98) at event.c:1432 #4 0x004fda20 in listener_thread (thd=0x25d4b70, dummy=0x7fe4a808b7c8) at event.c:1704 #5 0x7fe4bb4bf0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7fe4baff062d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 21.01.2017 um 09:50 schrieb Yann Ylavic: Hi Stefan, On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe wrote: after running the whole night. These are the only ones still happening. Should i revert the mpm patch to check whether it's the source? Yes please, we need to determine... Thanks, Yann.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan, On Sat, Jan 21, 2017 at 9:45 AM, Stefan Priebe wrote: > > after running the whole night. These are the only ones still happening. > Should i revert the mpm patch to check whether it's the source? Yes please, we need to determine... Thanks, Yann.
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, after running the whole night. These are the only ones still happening. Should i revert the mpm patch to check whether it's the source? I'm only seeing one related to mod_http2. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_pool_cleanup_kill (p=0x7f128c039378, data=data@entry=0x7f128c039400, cleanup_fn=cleanup_fn@entry=0x7f129fe89a40 ) at memory/unix/apr_pools.c:2276 #0 apr_pool_cleanup_kill (p=0x7f128c039378, data=data@entry=0x7f128c039400, cleanup_fn=cleanup_fn@entry=0x7f129fe89a40 ) at memory/unix/apr_pools.c:2276 #1 0x7f129fe85e91 in apr_pool_cleanup_run (p=, data=0x7f128c039400, cleanup_fn=0x7f129fe89a40 ) at memory/unix/apr_pools.c:2342 #2 0x7f129fe89d22 in apr_socket_close (thesocket=) at network_io/unix/sockets.c:183 #3 0x004fa54e in process_lingering_close (cs=0x7f128c039608, pfd=0x1a4afa8) at event.c:1510 #4 0x004fdc30 in listener_thread (thd=0x7f128c039378, dummy=0x7f128c039608) at event.c:1837 #5 0x7f129f6100a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f129f14162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from /usr/lib/debug//usr/local/apache2/bin/httpd...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_pool_cleanup_kill (p=0x7f128c075f88, data=data@entry=0x7f128c076010, cleanup_fn=cleanup_fn@entry=0x7f129fe89a40 ) at memory/unix/apr_pools.c:2276 #0 apr_pool_cleanup_kill (p=0x7f128c075f88, data=data@entry=0x7f128c076010, cleanup_fn=cleanup_fn@entry=0x7f129fe89a40 ) at memory/unix/apr_pools.c:2276 #1 0x7f129fe85e91 in apr_pool_cleanup_run (p=, data=0x7f128c076010, cleanup_fn=0x7f129fe89a40 ) at memory/unix/apr_pools.c:2342 #2 0x7f129fe89d22 in apr_socket_close (thesocket=) at network_io/unix/sockets.c:183 #3 0x004fa54e in process_lingering_close (cs=0x7f128c076218, pfd=0x1a4afa8) at event.c:1510 #4 0x004fdc30 in listener_thread (thd=0x7f128c075f88, dummy=0x7f128c076218) at event.c:1837 #5 0x7f129f6100a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #6 0x7f129f14162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Reading symbols from /usr/local/apache/bin/httpd...Reading symbols from /usr/lib/debug//usr/local/apache2/bin/httpd...done. done. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f12a02ce832 in apr_bucket_free (mem=0x7f11d802d838) at buckets/apr_buckets_alloc.c:200 #0 0x7f12a02ce832 in apr_bucket_free (mem=0x7f11d802d838) at buckets/apr_buckets_alloc.c:200 #1 0x7f12a02ced97 in heap_bucket_destroy (data=0x7f11d00769c8) at buckets/apr_buckets_heap.c:36 #2 0x0045b9d3 in writev_nonblocking (s=0x7f128c086b20, vec=0x7f12837ed890, nvec=4, bb=0x7f128c087378, cumulative_bytes_written=0x7f11d007f7e8, c=0x7f128c086db8) at core_filters.c:796 #3 0x0045bba1 in send_brigade_nonblocking (s=0x7f128c086b20, bb=0x7f128c087378, bytes_written=0x76ce00 , c=0x7f128c087380) at core_filters.c:704 #4 0x0045c996 in ap_core_output_filter (f=0x7f11d802d838, new_bb=0x7f128c087378) at core_filters.c:556 #5 0x004aac18 in bio_filter_out_pass ( outctx=outctx@entry=0x7f128c087358) at ssl_engine_io.c:139 #6 0x004aacbe in bio_filter_out_write (bio=, in=0x7f11d803b8a3 "\027\003\003\005,\f\037h<5-O\005\272", inl=1329) at ssl_engine_io.c:234 #7 0x7f12a0cc124c in BIO_write () from /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 #8 0x7f12a1024fe2 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 #9 0x7f12a10256c5 in ?? () from /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 #10 0x004ae04a in ssl_filter_write (f=, f=, len=, data=) at ssl_engine_io.c:793 #11 ssl_io_filter_output (f=0x7f128c087330, bb=0x7f11d007f898) at ssl_engine_io.c:1746 #12 0x004ab30a in ssl_io_filter_coalesce (f=0x7f11d802d838, bb=0x7f11d007f898) at ssl_engine_io.c:1663 #13 0x004db9ed in pass_output (io=0x7f11d0026148, session_eoc=0x0, flush=) at h2_conn_io.c:296 #14 0x004dbf20 in h2_conn_io_flush (io=0x7f11d0026148) at h2_conn_io.c:346 #15 0x004d012d in h2_session_process (session=0x7f11d0026100, async=1) at h2_session.c:2248 #16 0x004bf641 in h2_conn_run
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, this patch solved the file_close segfault. Never saw that again. Right now i got this one: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f129f612274 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0 (gdb) bt #0 0x7f129f612274 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0 #1 0x7f129fe83bc9 in apr_thread_mutex_lock (mutex=) at locks/unix/thread_mutex.c:92 #2 0x7f129fe82712 in apr_file_seek (thefile=thefile@entry=0x7f120c0670c0, where=where@entry=0, offset=offset@entry=0x7f12897f9b60) at file_io/unix/seek.c:62 #3 0x004dc413 in read_to_scratch (b=0x7f11c807ea08, io=0x7f11c80806d8) at h2_conn_io.c:220 #4 h2_conn_io_pass (io=io@entry=0x7f11c80806d8, bb=0x7f11c8080a08) at h2_conn_io.c:434 #5 0x004ca3ae in on_send_data_cb (ngh2=, frame=, framehd=, length=1291, source=, userp=0x7f11c8080690) at h2_session.c:649 #6 0x7f12a0980e95 in ?? () from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 #7 0x7f12a0981ea9 in nghttp2_session_send () from /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 #8 0x004cca89 in h2_session_send (session=0x7f11c8080690) at h2_session.c:1414 #9 0x004cfc8c in h2_session_process (session=0x7f11c8080690, async=1) at h2_session.c:2246 #10 0x004bf641 in h2_conn_run (ctx=0x7f120c066730, c=0x7f128c06cd98) at h2_conn.c:214 #11 0x004c202a in h2_h2_process_conn (c=0x3732323535395f39) at h2_h2.c:658 #12 0x0046a450 in ap_run_process_connection (c=0x7f128c06cd98) at connection.c:42 #13 0x004fbf10 in process_socket (my_thread_num=, my_child_num=, cs=0x7f128c06cd08, sock=, p=, thd=) at event.c:1134 #14 worker_thread (thd=0x3732323535395f39, dummy=0x0) at event.c:2137 #15 0x7f129f6100a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #16 0x7f129f14162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 20.01.2017 um 18:17 schrieb Yann Ylavic: > On Fri, Jan 20, 2017 at 5:51 PM, Stefan Priebe - Profihost AG > wrote: >> Yes. Until now I got 4 traces. But all are the same pointing to apr kill >> pool. Not like before where i got many different ones. > > Could you try this new patch on mod_http2 please? > > Thanks, > Yann. >
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 20.01.2017 um 18:17 schrieb Yann Ylavic: > On Fri, Jan 20, 2017 at 5:51 PM, Stefan Priebe - Profihost AG > wrote: >> Yes. Until now I got 4 traces. But all are the same pointing to apr kill >> pool. Not like before where i got many different ones. > > Could you try this new patch on mod_http2 please? status ist still used in the log_cerror function. compilation fails. For now i just commented the ap_log_cerror call. Stefan > > Thanks, > Yann. >
Re: mod_http2 and Frequent wake-ups for mpm_event
Good catch, Yann. I have local modifications that throw most of the manual cleanup out and rely on auto pool cleanup as much as possible. Curious if that will solve Stefan's crashes. > Am 20.01.2017 um 18:17 schrieb Yann Ylavic : > > On Fri, Jan 20, 2017 at 5:51 PM, Stefan Priebe - Profihost AG > wrote: >> Yes. Until now I got 4 traces. But all are the same pointing to apr kill >> pool. Not like before where i got many different ones. > > Could you try this new patch on mod_http2 please? > > Thanks, > Yann. > Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
On Fri, Jan 20, 2017 at 5:51 PM, Stefan Priebe - Profihost AG
wrote:
> Yes. Until now I got 4 traces. But all are the same pointing to apr kill
> pool. Not like before where i got many different ones.
Could you try this new patch on mod_http2 please?
Thanks,
Yann.
Index: modules/http2/h2_stream.c
===
--- modules/http2/h2_stream.c (revision 1778313)
+++ modules/http2/h2_stream.c (working copy)
@@ -171,21 +171,20 @@ static void prepend_response(h2_stream *stream, h2
static apr_status_t stream_pool_cleanup(void *ctx)
{
h2_stream *stream = ctx;
-apr_status_t status;
ap_assert(stream->can_be_cleaned);
-if (stream->files) {
+/* Files are cleaned-up/closed already, just log */
+if (APLOGctrace3(stream->session->c) && stream->files) {
apr_file_t *file;
int i;
for (i = 0; i < stream->files->nelts; ++i) {
file = APR_ARRAY_IDX(stream->files, i, apr_file_t*);
-status = apr_file_close(file);
ap_log_cerror(APLOG_MARK, APLOG_TRACE3, status, stream->session->c,
"h2_stream(%ld-%d): destroy, closed file %d",
stream->session->id, stream->id, i);
}
-stream->files = NULL;
}
+stream->files = NULL;
return APR_SUCCESS;
}
Re: mod_http2 and Frequent wake-ups for mpm_event
Yes. Until now I got 4 traces. But all are the same pointing to apr kill pool. Not like before where i got many different ones. Greets, Stefan Excuse my typo sent from my mobile phone. > Am 20.01.2017 um 17:44 schrieb Yann Ylavic : > > On Fri, Jan 20, 2017 at 5:41 PM, Stefan Priebe - Profihost AG > wrote: >> Sorry misstyped it's V7 with mod_http 1.8.8 unpatched. > > Unpatched but still compiled with -fno-strict-aliasing (along with APR)? > > Thanks, > Yann.
Re: mod_http2 and Frequent wake-ups for mpm_event
On Fri, Jan 20, 2017 at 5:41 PM, Stefan Priebe - Profihost AG wrote: > Sorry misstyped it's V7 with mod_http 1.8.8 unpatched. Unpatched but still compiled with -fno-strict-aliasing (along with APR)? Thanks, Yann.
Re: mod_http2 and Frequent wake-ups for mpm_event
Sorry misstyped it's V7 with mod_http 1.8.8 unpatched. Stefan Excuse my typo sent from my mobile phone. > Am 20.01.2017 um 16:23 schrieb Stefan Priebe - Profihost AG > : > > Hi, > > it crashed again with V6 and plain mod_http2 v1.8.8. > > This is the crash incl. line numbers: > Program terminated with signal SIGSEGV, Segmentation fault. > #0 apr_pool_cleanup_kill (p=0x2d392d322d32, > data=data@entry=0x7f330006bc70, >cleanup_fn=cleanup_fn@entry=0x7f33155b0f90 ) > at memory/unix/apr_pools.c:2264 > 2264memory/unix/apr_pools.c: No such file or directory. > (gdb) bt > #0 apr_pool_cleanup_kill (p=0x2d392d322d32, > data=data@entry=0x7f330006bc70, >cleanup_fn=cleanup_fn@entry=0x7f33155b0f90 ) > at memory/unix/apr_pools.c:2264 > #1 0x7f33155b5e51 in apr_pool_cleanup_run (p=, > data=0x7f330006bc70, >cleanup_fn=0x7f33155b0f90 ) at > memory/unix/apr_pools.c:2342 > #2 0x7f33155b1322 in apr_file_close (file=) at > file_io/unix/open.c:255 > #3 0x004d0012 in stream_pool_cleanup (ctx=0x7f3294022480) at > h2_stream.c:182 > #4 0x7f33155b4b1e in run_cleanups (cref=) at > memory/unix/apr_pools.c:2352 > #5 apr_pool_destroy (pool=0x7f3294022408) at memory/unix/apr_pools.c:814 > #6 0x004d0786 in h2_stream_destroy (stream=) at > h2_stream.c:249 > #7 0x004c334c in stream_done (m=, > stream=, rst_error=) at h2_mplx.c:470 > #8 0x004c335b in stream_done_iter (ctx=, > val=) at h2_mplx.c:475 > #9 0x7f33155ac156 in apr_hash_do (comp=comp@entry=0x4d5880 > , rec=rec@entry=0x7f32f7feea50, ht=) >at tables/apr_hash.c:542 > #10 0x004d623d in h2_ihash_iter (ih=, > fn=fn@entry=0x4c3350 , ctx=ctx@entry=0x7f3294034c88) >at h2_util.c:315 > #11 0x004c4649 in h2_mplx_release_and_join (m=0x7f3294034c88, > wait=0x7f3294034c30) at h2_mplx.c:579 > #12 0x004ca7cf in h2_session_destroy (session=0x7f3294034a50) at > h2_session.c:739 > #13 0x0045b726 in remove_empty_buckets (bb=0x7f330006af18) at > core_filters.c:720 > #14 0x0045be28 in setaside_remaining_output (f=0x7f32ec0c5b88, > ctx=0x7f32ec0c5e48, bb=0x7f330006af18, c=, >c=) at core_filters.c:584 > #15 0x0045c896 in ap_core_output_filter (f=0x2d392d322d32, > new_bb=0x7f330006af18) at core_filters.c:568 > #16 0x004ad932 in ssl_io_filter_output (f=0x7f330006aed0, > bb=0x7f32ec0c5f10) at ssl_engine_io.c:1716 > #17 0x004aad0a in ssl_io_filter_coalesce (f=0x2d392d322d32, > bb=0x7f32ec0c5f10) at ssl_engine_io.c:1663 > #18 0x004db543 in pass_output (io=0x7f3294034a98, > session_eoc=0x7f3294034a50, flush=) at h2_conn_io.c:311 > #19 0x004cf50a in h2_session_process (session=0x7f3294034a50, > async=1) at h2_session.c:2347 > #20 0x004befb2 in h2_conn_run (ctx=0x7f32ec0c5e18, > c=0x7f330006a958) at h2_conn.c:214 > #21 0x004c198a in h2_h2_process_conn (c=0x2d392d322d32) at > h2_h2.c:658 > #22 0x0046a2b0 in ap_run_process_connection (c=0x7f330006a958) > at connection.c:42 > #23 0x004fb890 in process_socket (my_thread_num=, > my_child_num=, cs=0x7f330006a8c8, >sock=, p=, thd=) at > event.c:1134 > #24 worker_thread (thd=0x2d392d322d32, dummy=0x7f330006bc70) at > event.c:2137 > #25 0x7f3314d400a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #26 0x7f331487162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Stefan >> Am 20.01.2017 um 13:20 schrieb Stefan Eissing: >> Please without. Then I least know if that version behaves. Planning on more >> extensive changes for a 1.9.0 now. Thanks! >> >> -Stefan >> >>> Am 20.01.2017 um 13:18 schrieb Stefan Priebe - Profihost AG >>> : >>> >>> Will start retesting V6 patch. Should I use mod_http2 1.8.8 with or without >>> patches? >>> >>> Greets, >>> Stefan >>> >>> Excuse my typo sent from my mobile phone. >>> Am 20.01.2017 um 13:04 schrieb Stefan Eissing : Different apr versions? Might there have been a bugfix affecting us? > Am 20.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG > : > > might this be a debian bug? i can't reproduce with apr-included. > >> Am 20.01.2017 um 08:20 schrieb Yann Ylavic: >> Hi, >> >> On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG >> wrote: >>> Hi Stefan, >>> Am 19.01.2017 um 22:44 schrieb Stefan Eissing: this seems to be a tough bone to chew. Therefore we need to go deeper: - can you compile the module so that we see line numbers in the trace? >>> >>> Do you have any idea how to arrange this? I've no idea how to pass the >>> -ggdb option through Apache. >> >> DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... >> >>> - which apr version are you using? >>> this one: >>> https://packages.debian.org/jessie/libapr1 >> >> Could you also build libapr1 with this same flags? >> >>>
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, it crashed again with V6 and plain mod_http2 v1.8.8. This is the crash incl. line numbers: Program terminated with signal SIGSEGV, Segmentation fault. #0 apr_pool_cleanup_kill (p=0x2d392d322d32, data=data@entry=0x7f330006bc70, cleanup_fn=cleanup_fn@entry=0x7f33155b0f90 ) at memory/unix/apr_pools.c:2264 2264memory/unix/apr_pools.c: No such file or directory. (gdb) bt #0 apr_pool_cleanup_kill (p=0x2d392d322d32, data=data@entry=0x7f330006bc70, cleanup_fn=cleanup_fn@entry=0x7f33155b0f90 ) at memory/unix/apr_pools.c:2264 #1 0x7f33155b5e51 in apr_pool_cleanup_run (p=, data=0x7f330006bc70, cleanup_fn=0x7f33155b0f90 ) at memory/unix/apr_pools.c:2342 #2 0x7f33155b1322 in apr_file_close (file=) at file_io/unix/open.c:255 #3 0x004d0012 in stream_pool_cleanup (ctx=0x7f3294022480) at h2_stream.c:182 #4 0x7f33155b4b1e in run_cleanups (cref=) at memory/unix/apr_pools.c:2352 #5 apr_pool_destroy (pool=0x7f3294022408) at memory/unix/apr_pools.c:814 #6 0x004d0786 in h2_stream_destroy (stream=) at h2_stream.c:249 #7 0x004c334c in stream_done (m=, stream=, rst_error=) at h2_mplx.c:470 #8 0x004c335b in stream_done_iter (ctx=, val=) at h2_mplx.c:475 #9 0x7f33155ac156 in apr_hash_do (comp=comp@entry=0x4d5880 , rec=rec@entry=0x7f32f7feea50, ht=) at tables/apr_hash.c:542 #10 0x004d623d in h2_ihash_iter (ih=, fn=fn@entry=0x4c3350 , ctx=ctx@entry=0x7f3294034c88) at h2_util.c:315 #11 0x004c4649 in h2_mplx_release_and_join (m=0x7f3294034c88, wait=0x7f3294034c30) at h2_mplx.c:579 #12 0x004ca7cf in h2_session_destroy (session=0x7f3294034a50) at h2_session.c:739 #13 0x0045b726 in remove_empty_buckets (bb=0x7f330006af18) at core_filters.c:720 #14 0x0045be28 in setaside_remaining_output (f=0x7f32ec0c5b88, ctx=0x7f32ec0c5e48, bb=0x7f330006af18, c=, c=) at core_filters.c:584 #15 0x0045c896 in ap_core_output_filter (f=0x2d392d322d32, new_bb=0x7f330006af18) at core_filters.c:568 #16 0x004ad932 in ssl_io_filter_output (f=0x7f330006aed0, bb=0x7f32ec0c5f10) at ssl_engine_io.c:1716 #17 0x004aad0a in ssl_io_filter_coalesce (f=0x2d392d322d32, bb=0x7f32ec0c5f10) at ssl_engine_io.c:1663 #18 0x004db543 in pass_output (io=0x7f3294034a98, session_eoc=0x7f3294034a50, flush=) at h2_conn_io.c:311 #19 0x004cf50a in h2_session_process (session=0x7f3294034a50, async=1) at h2_session.c:2347 #20 0x004befb2 in h2_conn_run (ctx=0x7f32ec0c5e18, c=0x7f330006a958) at h2_conn.c:214 #21 0x004c198a in h2_h2_process_conn (c=0x2d392d322d32) at h2_h2.c:658 #22 0x0046a2b0 in ap_run_process_connection (c=0x7f330006a958) at connection.c:42 #23 0x004fb890 in process_socket (my_thread_num=, my_child_num=, cs=0x7f330006a8c8, sock=, p=, thd=) at event.c:1134 #24 worker_thread (thd=0x2d392d322d32, dummy=0x7f330006bc70) at event.c:2137 #25 0x7f3314d400a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #26 0x7f331487162d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 20.01.2017 um 13:20 schrieb Stefan Eissing: > Please without. Then I least know if that version behaves. Planning on more > extensive changes for a 1.9.0 now. Thanks! > > -Stefan > >> Am 20.01.2017 um 13:18 schrieb Stefan Priebe - Profihost AG >> : >> >> Will start retesting V6 patch. Should I use mod_http2 1.8.8 with or without >> patches? >> >> Greets, >> Stefan >> >> Excuse my typo sent from my mobile phone. >> >> Am 20.01.2017 um 13:04 schrieb Stefan Eissing : >> >>> Different apr versions? Might there have been a bugfix affecting us? >>> Am 20.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG : might this be a debian bug? i can't reproduce with apr-included. Am 20.01.2017 um 08:20 schrieb Yann Ylavic: > Hi, > > On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG > wrote: >> Hi Stefan, >> >> Am 19.01.2017 um 22:44 schrieb Stefan Eissing: >>> this seems to be a tough bone to chew. Therefore we need to go deeper: >>> - can you compile the module so that we see line numbers in the trace? >> >> Do you have any idea how to arrange this? I've no idea how to pass the >> -ggdb option through Apache. > > DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... > >> >>> - which apr version are you using? >> this one: >> https://packages.debian.org/jessie/libapr1 > > Could you also build libapr1 with this same flags? > >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Please use V7 too, I don't think I'll revert it in trunk (it makes sense). On Fri, Jan 20, 2017 at 1:20 PM, Stefan Eissing wrote: > Please without. Then I least know if that version behaves. Planning on more > extensive changes for a 1.9.0 now. Thanks! > > -Stefan > >> Am 20.01.2017 um 13:18 schrieb Stefan Priebe - Profihost AG >> : >> >> Will start retesting V6 patch. Should I use mod_http2 1.8.8 with or without >> patches? >> >> Greets, >> Stefan >> >> Excuse my typo sent from my mobile phone. >> >> Am 20.01.2017 um 13:04 schrieb Stefan Eissing : >> >>> Different apr versions? Might there have been a bugfix affecting us? >>> Am 20.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG : might this be a debian bug? i can't reproduce with apr-included. Am 20.01.2017 um 08:20 schrieb Yann Ylavic: > Hi, > > On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG > wrote: >> Hi Stefan, >> >> Am 19.01.2017 um 22:44 schrieb Stefan Eissing: >>> this seems to be a tough bone to chew. Therefore we need to go deeper: >>> - can you compile the module so that we see line numbers in the trace? >> >> Do you have any idea how to arrange this? I've no idea how to pass the >> -ggdb option through Apache. > > DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... > >> >>> - which apr version are you using? >> this one: >> https://packages.debian.org/jessie/libapr1 > > Could you also build libapr1 with this same flags? > >>> >>> Stefan Eissing >>> >>> bytes GmbH >>> Hafenstrasse 16 >>> 48155 Münster >>> www.greenbytes.de >>> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Please without. Then I least know if that version behaves. Planning on more extensive changes for a 1.9.0 now. Thanks! -Stefan > Am 20.01.2017 um 13:18 schrieb Stefan Priebe - Profihost AG > : > > Will start retesting V6 patch. Should I use mod_http2 1.8.8 with or without > patches? > > Greets, > Stefan > > Excuse my typo sent from my mobile phone. > > Am 20.01.2017 um 13:04 schrieb Stefan Eissing : > >> Different apr versions? Might there have been a bugfix affecting us? >> >>> Am 20.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG >>> : >>> >>> might this be a debian bug? i can't reproduce with apr-included. >>> >>> Am 20.01.2017 um 08:20 schrieb Yann Ylavic: Hi, On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG wrote: > Hi Stefan, > > Am 19.01.2017 um 22:44 schrieb Stefan Eissing: >> this seems to be a tough bone to chew. Therefore we need to go deeper: >> - can you compile the module so that we see line numbers in the trace? > > Do you have any idea how to arrange this? I've no idea how to pass the > -ggdb option through Apache. DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... > >> - which apr version are you using? > this one: > https://packages.debian.org/jessie/libapr1 Could you also build libapr1 with this same flags? >> >> Stefan Eissing >> >> bytes GmbH >> Hafenstrasse 16 >> 48155 Münster >> www.greenbytes.de >> Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
Will start retesting V6 patch. Should I use mod_http2 1.8.8 with or without patches? Greets, Stefan Excuse my typo sent from my mobile phone. > Am 20.01.2017 um 13:04 schrieb Stefan Eissing : > > Different apr versions? Might there have been a bugfix affecting us? > >> Am 20.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG >> : >> >> might this be a debian bug? i can't reproduce with apr-included. >> >>> Am 20.01.2017 um 08:20 schrieb Yann Ylavic: >>> Hi, >>> >>> On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG >>> wrote: Hi Stefan, > Am 19.01.2017 um 22:44 schrieb Stefan Eissing: > this seems to be a tough bone to chew. Therefore we need to go deeper: > - can you compile the module so that we see line numbers in the trace? Do you have any idea how to arrange this? I've no idea how to pass the -ggdb option through Apache. >>> >>> DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... >>> > - which apr version are you using? this one: https://packages.debian.org/jessie/libapr1 >>> >>> Could you also build libapr1 with this same flags? >>> > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Different apr versions? Might there have been a bugfix affecting us? > Am 20.01.2017 um 12:49 schrieb Stefan Priebe - Profihost AG > : > > might this be a debian bug? i can't reproduce with apr-included. > > Am 20.01.2017 um 08:20 schrieb Yann Ylavic: >> Hi, >> >> On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG >> wrote: >>> Hi Stefan, >>> >>> Am 19.01.2017 um 22:44 schrieb Stefan Eissing: this seems to be a tough bone to chew. Therefore we need to go deeper: - can you compile the module so that we see line numbers in the trace? >>> >>> Do you have any idea how to arrange this? I've no idea how to pass the >>> -ggdb option through Apache. >> >> DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... >> >>> - which apr version are you using? >>> this one: >>> https://packages.debian.org/jessie/libapr1 >> >> Could you also build libapr1 with this same flags? >> Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
On Fri, Jan 20, 2017 at 12:49 PM, Stefan Priebe - Profihost AG wrote: > might this be a debian bug? i can't reproduce with apr-included. Did you build the included-apr (and httpd) with -fno-strict-aliasing ? Not sure Debian sets it for its builds either, just a speculation (I used to have issues in APR_RING without it).
Re: mod_http2 and Frequent wake-ups for mpm_event
might this be a debian bug? i can't reproduce with apr-included. Am 20.01.2017 um 08:20 schrieb Yann Ylavic: > Hi, > > On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG > wrote: >> Hi Stefan, >> >> Am 19.01.2017 um 22:44 schrieb Stefan Eissing: >>> this seems to be a tough bone to chew. Therefore we need to go deeper: >>> - can you compile the module so that we see line numbers in the trace? >> >> Do you have any idea how to arrange this? I've no idea how to pass the >> -ggdb option through Apache. > > DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... > >> >>> - which apr version are you using? >> this one: >> https://packages.debian.org/jessie/libapr1 > > Could you also build libapr1 with this same flags? >
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 20.01.2017 um 08:20 schrieb Yann Ylavic: > Hi, > > On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG > wrote: >> Hi Stefan, >> >> Am 19.01.2017 um 22:44 schrieb Stefan Eissing: >>> this seems to be a tough bone to chew. Therefore we need to go deeper: >>> - can you compile the module so that we see line numbers in the trace? >> >> Do you have any idea how to arrange this? I've no idea how to pass the >> -ggdb option through Apache. > > DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... OK i switched now to included apr and included apr-util. And i added those CFLAGS to my configure line. Have to wait for a new crash. Stefan > >> >>> - which apr version are you using? >> this one: >> https://packages.debian.org/jessie/libapr1 > > Could you also build libapr1 with this same flags? >
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi, On Fri, Jan 20, 2017 at 8:03 AM, Stefan Priebe - Profihost AG wrote: > Hi Stefan, > > Am 19.01.2017 um 22:44 schrieb Stefan Eissing: >> this seems to be a tough bone to chew. Therefore we need to go deeper: >> - can you compile the module so that we see line numbers in the trace? > > Do you have any idea how to arrange this? I've no idea how to pass the > -ggdb option through Apache. DEB_CFLAGS_SET="-O2 -ggdb -fno-strict-aliasing ..." dpkg-buildpackage ... > >> - which apr version are you using? > this one: > https://packages.debian.org/jessie/libapr1 Could you also build libapr1 with this same flags?
Re: mod_http2 and Frequent wake-ups for mpm_event
Hi Stefan, Am 19.01.2017 um 22:44 schrieb Stefan Eissing: > this seems to be a tough bone to chew. Therefore we need to go deeper: > - can you compile the module so that we see line numbers in the trace? Do you have any idea how to arrange this? I've no idea how to pass the -ggdb option through Apache. I'm compiling mod_http2 as a static module included in httpd. > - which apr version are you using? this one: https://packages.debian.org/jessie/libapr1 > - can you reproduce this at will? How? Which client? Just a GET or something > more sophisticated? No i can't ;-) i've just a bunch of live webshops producing this. So just real users - most probably GET + POST. Stefan > > Thanks for the help! > > Cheers, > > Stefan > >> Am 19.01.2017 um 22:01 schrieb Stefan Priebe - Profihost AG >> : >> >> new segfault with both patches on top of v1.8.8: >> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #0 0x7fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x7fbefd2a0036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #2 0x7fbefd2a046f in apr_hash_set () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #3 0x0052a26b in h2_ihash_remove () >> #4 0x00506b24 in purge_stream () >> #5 0x0052a1c2 in ihash_iter () >> #6 0x7fbefd2a08a6 in apr_hash_do () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #7 0x0052a200 in h2_ihash_iter () >> #8 0x00506b65 in purge_streams () >> #9 0x005082dd in h2_mplx_release_and_join () >> #10 0x005158f9 in h2_session_cleanup () >> #11 0x005164a4 in session_pool_cleanup () >> #12 0x7fbefd2a9976 in apr_pool_destroy () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #13 0x7fbefd2a9c55 in apr_pool_clear () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #14 0x00566acd in ap_push_pool () >> #15 0x0056008a in process_lingering_close () >> #16 0x00560ced in listener_thread () >> #17 0x7fbefd0780a4 in start_thread () from >> /lib/x86_64-linux-gnu/libpthread.so.0 >> #18 0x7fbefcdad62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> Stefan >> >> Am 19.01.2017 um 21:48 schrieb Stefan Eissing: >>> On top please. There is only one way: forward! >>> Am 19.01.2017 um 21:47 schrieb Stefan Priebe - Profihost AG : Am 19.01.2017 um 21:39 schrieb Stefan Eissing: > Thanks, Stefan. Can you given the attached Patch a try? sure. On top of the last one? Or should i drop it? Stefan >> Am 19.01.2017 um 19:33 schrieb Stefan Priebe : >> >> Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: >> >> # >> >> Using host libthread_db library >> "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x7f61f6bce036 in ?? () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #2 0x7f61f6bce46f in apr_hash_set () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #3 0x0052a26d in h2_ihash_remove () >> #4 0x00506b24 in purge_stream () >> #5 0x0052a1c4 in ihash_iter () >> #6 0x7f61f6bce8a6 in apr_hash_do () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #7 0x0052a202 in h2_ihash_iter () >> #8 0x00506b65 in purge_streams () >> #9 0x005082df in h2_mplx_release_and_join () >> #10 0x005158fb in h2_session_cleanup () >> #11 0x005164a6 in session_pool_cleanup () >> #12 0x7f61f6bd7976 in apr_pool_destroy () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #13 0x7f61f6bd7c55 in apr_pool_clear () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #14 0x00566acf in ap_push_pool () >> #15 0x0056008c in process_lingering_close () >> #16 0x00560cef in listener_thread () >> #17 0x7f61f69a60a4 in start_thread () from >> /lib/x86_64-linux-gnu/libpthread.so.0 >> #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> # >> >> Using host libthread_db library >> "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7f61f673b014 in ??
Re: mod_http2 and Frequent wake-ups for mpm_event
Stefan, this seems to be a tough bone to chew. Therefore we need to go deeper: - can you compile the module so that we see line numbers in the trace? - which apr version are you using? - can you reproduce this at will? How? Which client? Just a GET or something more sophisticated? Thanks for the help! Cheers, Stefan > Am 19.01.2017 um 22:01 schrieb Stefan Priebe - Profihost AG > : > > new segfault with both patches on top of v1.8.8: > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x7fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x7fbefd2a0036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x7fbefd2a046f in apr_hash_set () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #3 0x0052a26b in h2_ihash_remove () > #4 0x00506b24 in purge_stream () > #5 0x0052a1c2 in ihash_iter () > #6 0x7fbefd2a08a6 in apr_hash_do () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #7 0x0052a200 in h2_ihash_iter () > #8 0x00506b65 in purge_streams () > #9 0x005082dd in h2_mplx_release_and_join () > #10 0x005158f9 in h2_session_cleanup () > #11 0x005164a4 in session_pool_cleanup () > #12 0x7fbefd2a9976 in apr_pool_destroy () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #13 0x7fbefd2a9c55 in apr_pool_clear () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #14 0x00566acd in ap_push_pool () > #15 0x0056008a in process_lingering_close () > #16 0x00560ced in listener_thread () > #17 0x7fbefd0780a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #18 0x7fbefcdad62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > Stefan > > Am 19.01.2017 um 21:48 schrieb Stefan Eissing: >> On top please. There is only one way: forward! >> >>> Am 19.01.2017 um 21:47 schrieb Stefan Priebe - Profihost AG >>> : >>> >>> >>> Am 19.01.2017 um 21:39 schrieb Stefan Eissing: Thanks, Stefan. Can you given the attached Patch a try? >>> >>> sure. On top of the last one? Or should i drop it? >>> >>> >>> Stefan >>> > Am 19.01.2017 um 19:33 schrieb Stefan Priebe : > > Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: > > # > > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x7f61f6bce036 in ?? () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x7f61f6bce46f in apr_hash_set () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #3 0x0052a26d in h2_ihash_remove () > #4 0x00506b24 in purge_stream () > #5 0x0052a1c4 in ihash_iter () > #6 0x7f61f6bce8a6 in apr_hash_do () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #7 0x0052a202 in h2_ihash_iter () > #8 0x00506b65 in purge_streams () > #9 0x005082df in h2_mplx_release_and_join () > #10 0x005158fb in h2_session_cleanup () > #11 0x005164a6 in session_pool_cleanup () > #12 0x7f61f6bd7976 in apr_pool_destroy () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #13 0x7f61f6bd7c55 in apr_pool_clear () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #14 0x00566acf in ap_push_pool () > #15 0x0056008c in process_lingering_close () > #16 0x00560cef in listener_thread () > #17 0x7f61f69a60a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > # > > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x7f61f6bce036 in ?? () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x7f61f6bce46f in apr_hash_set () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #3 0x0052a26d in h2_ihash_remove () > #4 0x00506b24 in purge_stream () > #5 0x0052a1c4 in ihash_iter () > #6 0x7f61f6bce8a6 in apr_hash_do () from
Re: mod_http2 and Frequent wake-ups for mpm_event
new segfault with both patches on top of v1.8.8: Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x7fbefce0d014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7fbefd2a0036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x7fbefd2a046f in apr_hash_set () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #3 0x0052a26b in h2_ihash_remove () #4 0x00506b24 in purge_stream () #5 0x0052a1c2 in ihash_iter () #6 0x7fbefd2a08a6 in apr_hash_do () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #7 0x0052a200 in h2_ihash_iter () #8 0x00506b65 in purge_streams () #9 0x005082dd in h2_mplx_release_and_join () #10 0x005158f9 in h2_session_cleanup () #11 0x005164a4 in session_pool_cleanup () #12 0x7fbefd2a9976 in apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #13 0x7fbefd2a9c55 in apr_pool_clear () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #14 0x00566acd in ap_push_pool () #15 0x0056008a in process_lingering_close () #16 0x00560ced in listener_thread () #17 0x7fbefd0780a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #18 0x7fbefcdad62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 19.01.2017 um 21:48 schrieb Stefan Eissing: > On top please. There is only one way: forward! > >> Am 19.01.2017 um 21:47 schrieb Stefan Priebe - Profihost AG >> : >> >> >> Am 19.01.2017 um 21:39 schrieb Stefan Eissing: >>> Thanks, Stefan. Can you given the attached Patch a try? >> >> sure. On top of the last one? Or should i drop it? >> >> >> Stefan >> Am 19.01.2017 um 19:33 schrieb Stefan Priebe : Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: # Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x7f61f6bce46f in apr_hash_set () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #3 0x0052a26d in h2_ihash_remove () #4 0x00506b24 in purge_stream () #5 0x0052a1c4 in ihash_iter () #6 0x7f61f6bce8a6 in apr_hash_do () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #7 0x0052a202 in h2_ihash_iter () #8 0x00506b65 in purge_streams () #9 0x005082df in h2_mplx_release_and_join () #10 0x005158fb in h2_session_cleanup () #11 0x005164a6 in session_pool_cleanup () #12 0x7f61f6bd7976 in apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #13 0x7f61f6bd7c55 in apr_pool_clear () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #14 0x00566acf in ap_push_pool () #15 0x0056008c in process_lingering_close () #16 0x00560cef in listener_thread () #17 0x7f61f69a60a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 # Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x7f61f6bce46f in apr_hash_set () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #3 0x0052a26d in h2_ihash_remove () #4 0x00506b24 in purge_stream () #5 0x0052a1c4 in ihash_iter () #6 0x7f61f6bce8a6 in apr_hash_do () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #7 0x0052a202 in h2_ihash_iter () #8 0x00506b65 in purge_streams () #9 0x005082df in h2_mplx_release_and_join () #10 0x005158fb in h2_session_cleanup () #11 0x005164a6 in session_pool_cleanup () #12 0x7f61f6bd7976 in apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #13 0x7f61f6bd7c55 in apr_pool_clear () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #14 0x00566acf in
Re: mod_http2 and Frequent wake-ups for mpm_event
On top please. There is only one way: forward! > Am 19.01.2017 um 21:47 schrieb Stefan Priebe - Profihost AG > : > > > Am 19.01.2017 um 21:39 schrieb Stefan Eissing: >> Thanks, Stefan. Can you given the attached Patch a try? > > sure. On top of the last one? Or should i drop it? > > > Stefan > >>> Am 19.01.2017 um 19:33 schrieb Stefan Priebe : >>> >>> Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: >>> >>> # >>> >>> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >>> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >>> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >>> #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #2 0x7f61f6bce46f in apr_hash_set () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #3 0x0052a26d in h2_ihash_remove () >>> #4 0x00506b24 in purge_stream () >>> #5 0x0052a1c4 in ihash_iter () >>> #6 0x7f61f6bce8a6 in apr_hash_do () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #7 0x0052a202 in h2_ihash_iter () >>> #8 0x00506b65 in purge_streams () >>> #9 0x005082df in h2_mplx_release_and_join () >>> #10 0x005158fb in h2_session_cleanup () >>> #11 0x005164a6 in session_pool_cleanup () >>> #12 0x7f61f6bd7976 in apr_pool_destroy () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #13 0x7f61f6bd7c55 in apr_pool_clear () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #14 0x00566acf in ap_push_pool () >>> #15 0x0056008c in process_lingering_close () >>> #16 0x00560cef in listener_thread () >>> #17 0x7f61f69a60a4 in start_thread () from >>> /lib/x86_64-linux-gnu/libpthread.so.0 >>> #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >>> >>> # >>> >>> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >>> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >>> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >>> #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #2 0x7f61f6bce46f in apr_hash_set () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #3 0x0052a26d in h2_ihash_remove () >>> #4 0x00506b24 in purge_stream () >>> #5 0x0052a1c4 in ihash_iter () >>> #6 0x7f61f6bce8a6 in apr_hash_do () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #7 0x0052a202 in h2_ihash_iter () >>> #8 0x00506b65 in purge_streams () >>> #9 0x005082df in h2_mplx_release_and_join () >>> #10 0x005158fb in h2_session_cleanup () >>> #11 0x005164a6 in session_pool_cleanup () >>> #12 0x7f61f6bd7976 in apr_pool_destroy () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #13 0x7f61f6bd7c55 in apr_pool_clear () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #14 0x00566acf in ap_push_pool () >>> #15 0x0056008c in process_lingering_close () >>> #16 0x00560cef in listener_thread () >>> #17 0x7f61f69a60a4 in start_thread () from >>> /lib/x86_64-linux-gnu/libpthread.so.0 >>> #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >>> >>> # >>> >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 0x7f204f922d63 in apr_pool_cleanup_kill () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> (gdb) bt >>> #0 0x7f204f922d63 in apr_pool_cleanup_kill () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #1 0x7f204f922e21 in apr_pool_cleanup_run () from >>> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >>> #2 0x0055ffe9 in process_lingering_close () >>> #3 0x00560cef in listener_thread () >>> #4 0x7f204f6f00a4 in start_thread () from >>> /lib/x86_64-linux-gnu/libpthread.so.0 >>> #5 0x7f204f42562d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >>> >>> full bt: >>> http://pastebin.com/raw/bP1vaYjw >>> >>> # >>> >>> Greets, >>> Stefan >>> >>> Am 19.01.2017 um 16:47 schrieb Stefan Priebe - Profihost AG: arg sorry my fault. Here is a complete trace: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7fc1c23e0f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 (gdb) bt #0 0x7fc1c23e0f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libap
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 19.01.2017 um 21:39 schrieb Stefan Eissing: > Thanks, Stefan. Can you given the attached Patch a try? sure. On top of the last one? Or should i drop it? Stefan >> Am 19.01.2017 um 19:33 schrieb Stefan Priebe : >> >> Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: >> >> # >> >> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #2 0x7f61f6bce46f in apr_hash_set () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #3 0x0052a26d in h2_ihash_remove () >> #4 0x00506b24 in purge_stream () >> #5 0x0052a1c4 in ihash_iter () >> #6 0x7f61f6bce8a6 in apr_hash_do () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #7 0x0052a202 in h2_ihash_iter () >> #8 0x00506b65 in purge_streams () >> #9 0x005082df in h2_mplx_release_and_join () >> #10 0x005158fb in h2_session_cleanup () >> #11 0x005164a6 in session_pool_cleanup () >> #12 0x7f61f6bd7976 in apr_pool_destroy () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #13 0x7f61f6bd7c55 in apr_pool_clear () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #14 0x00566acf in ap_push_pool () >> #15 0x0056008c in process_lingering_close () >> #16 0x00560cef in listener_thread () >> #17 0x7f61f69a60a4 in start_thread () from >> /lib/x86_64-linux-gnu/libpthread.so.0 >> #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> # >> >> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". >> Core was generated by `/usr/local/apache2/bin/httpd -k start'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 >> #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #2 0x7f61f6bce46f in apr_hash_set () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #3 0x0052a26d in h2_ihash_remove () >> #4 0x00506b24 in purge_stream () >> #5 0x0052a1c4 in ihash_iter () >> #6 0x7f61f6bce8a6 in apr_hash_do () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #7 0x0052a202 in h2_ihash_iter () >> #8 0x00506b65 in purge_streams () >> #9 0x005082df in h2_mplx_release_and_join () >> #10 0x005158fb in h2_session_cleanup () >> #11 0x005164a6 in session_pool_cleanup () >> #12 0x7f61f6bd7976 in apr_pool_destroy () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #13 0x7f61f6bd7c55 in apr_pool_clear () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #14 0x00566acf in ap_push_pool () >> #15 0x0056008c in process_lingering_close () >> #16 0x00560cef in listener_thread () >> #17 0x7f61f69a60a4 in start_thread () from >> /lib/x86_64-linux-gnu/libpthread.so.0 >> #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> # >> >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7f204f922d63 in apr_pool_cleanup_kill () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> (gdb) bt >> #0 0x7f204f922d63 in apr_pool_cleanup_kill () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #1 0x7f204f922e21 in apr_pool_cleanup_run () from >> /usr/lib/x86_64-linux-gnu/libapr-1.so.0 >> #2 0x0055ffe9 in process_lingering_close () >> #3 0x00560cef in listener_thread () >> #4 0x7f204f6f00a4 in start_thread () from >> /lib/x86_64-linux-gnu/libpthread.so.0 >> #5 0x7f204f42562d in clone () from /lib/x86_64-linux-gnu/libc.so.6 >> >> full bt: >> http://pastebin.com/raw/bP1vaYjw >> >> # >> >> Greets, >> Stefan >> >> Am 19.01.2017 um 16:47 schrieb Stefan Priebe - Profihost AG: >>> arg sorry my fault. >>> >>> Here is a complete trace: >>> Program terminated with signal SIGSEGV, Segmentation fault. >>> #0 0x7fc1c23e0f23 in apr_brigade_length () from >>> /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 >>> (gdb) bt >>> #0 0x7fc1c23e0f23 in apr_brigade_length () from >>> /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 >>> #1 0x0052a8f1 in h2_util_bb_avail () >>> #2 0x00521eaa in h2_stream_out_prepare () >>> #3 0x0051a55a in on_stream_resume () >>> #4 0x0050bdff in h2_mplx_dispatch_master_events () >>> #5 0x0
Re: mod_http2 and Frequent wake-ups for mpm_event
Thanks, Stefan. Can you given the attached Patch a try? h2_mplx_pool_life.diff Description: Binary data > Am 19.01.2017 um 19:33 schrieb Stefan Priebe : > > Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: > > # > > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x7f61f6bce46f in apr_hash_set () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #3 0x0052a26d in h2_ihash_remove () > #4 0x00506b24 in purge_stream () > #5 0x0052a1c4 in ihash_iter () > #6 0x7f61f6bce8a6 in apr_hash_do () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #7 0x0052a202 in h2_ihash_iter () > #8 0x00506b65 in purge_streams () > #9 0x005082df in h2_mplx_release_and_join () > #10 0x005158fb in h2_session_cleanup () > #11 0x005164a6 in session_pool_cleanup () > #12 0x7f61f6bd7976 in apr_pool_destroy () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #13 0x7f61f6bd7c55 in apr_pool_clear () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #14 0x00566acf in ap_push_pool () > #15 0x0056008c in process_lingering_close () > #16 0x00560cef in listener_thread () > #17 0x7f61f69a60a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > # > > Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". > Core was generated by `/usr/local/apache2/bin/httpd -k start'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 > #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x7f61f6bce46f in apr_hash_set () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #3 0x0052a26d in h2_ihash_remove () > #4 0x00506b24 in purge_stream () > #5 0x0052a1c4 in ihash_iter () > #6 0x7f61f6bce8a6 in apr_hash_do () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #7 0x0052a202 in h2_ihash_iter () > #8 0x00506b65 in purge_streams () > #9 0x005082df in h2_mplx_release_and_join () > #10 0x005158fb in h2_session_cleanup () > #11 0x005164a6 in session_pool_cleanup () > #12 0x7f61f6bd7976 in apr_pool_destroy () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #13 0x7f61f6bd7c55 in apr_pool_clear () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #14 0x00566acf in ap_push_pool () > #15 0x0056008c in process_lingering_close () > #16 0x00560cef in listener_thread () > #17 0x7f61f69a60a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > # > > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x7f204f922d63 in apr_pool_cleanup_kill () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > (gdb) bt > #0 0x7f204f922d63 in apr_pool_cleanup_kill () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #1 0x7f204f922e21 in apr_pool_cleanup_run () from > /usr/lib/x86_64-linux-gnu/libapr-1.so.0 > #2 0x0055ffe9 in process_lingering_close () > #3 0x00560cef in listener_thread () > #4 0x7f204f6f00a4 in start_thread () from > /lib/x86_64-linux-gnu/libpthread.so.0 > #5 0x7f204f42562d in clone () from /lib/x86_64-linux-gnu/libc.so.6 > > full bt: > http://pastebin.com/raw/bP1vaYjw > > # > > Greets, > Stefan > > Am 19.01.2017 um 16:47 schrieb Stefan Priebe - Profihost AG: >> arg sorry my fault. >> >> Here is a complete trace: >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x7fc1c23e0f23 in apr_brigade_length () from >> /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 >> (gdb) bt >> #0 0x7fc1c23e0f23 in apr_brigade_length () from >> /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 >> #1 0x0052a8f1 in h2_util_bb_avail () >> #2 0x00521eaa in h2_stream_out_prepare () >> #3 0x0051a55a in on_stream_resume () >> #4 0x0050bdff in h2_mplx_dispatch_master_events () >> #5 0x0051dc32 in h2_session_process () >> #6 0x00500115 in h2_conn_run () >> #7 0x00504b51 in h2_h2_process_conn () >> #8 0x0047
Re: mod_http2 and Frequent wake-ups for mpm_event
Here some more segfaults from 2.4.25 no mpm patch but http2 v1.8.8: # Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x7f61f6bce46f in apr_hash_set () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #3 0x0052a26d in h2_ihash_remove () #4 0x00506b24 in purge_stream () #5 0x0052a1c4 in ihash_iter () #6 0x7f61f6bce8a6 in apr_hash_do () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #7 0x0052a202 in h2_ihash_iter () #8 0x00506b65 in purge_streams () #9 0x005082df in h2_mplx_release_and_join () #10 0x005158fb in h2_session_cleanup () #11 0x005164a6 in session_pool_cleanup () #12 0x7f61f6bd7976 in apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #13 0x7f61f6bd7c55 in apr_pool_clear () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #14 0x00566acf in ap_push_pool () #15 0x0056008c in process_lingering_close () #16 0x00560cef in listener_thread () #17 0x7f61f69a60a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 # Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/local/apache2/bin/httpd -k start'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #0 0x7f61f673b014 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x7f61f6bce036 in ?? () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x7f61f6bce46f in apr_hash_set () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #3 0x0052a26d in h2_ihash_remove () #4 0x00506b24 in purge_stream () #5 0x0052a1c4 in ihash_iter () #6 0x7f61f6bce8a6 in apr_hash_do () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #7 0x0052a202 in h2_ihash_iter () #8 0x00506b65 in purge_streams () #9 0x005082df in h2_mplx_release_and_join () #10 0x005158fb in h2_session_cleanup () #11 0x005164a6 in session_pool_cleanup () #12 0x7f61f6bd7976 in apr_pool_destroy () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #13 0x7f61f6bd7c55 in apr_pool_clear () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #14 0x00566acf in ap_push_pool () #15 0x0056008c in process_lingering_close () #16 0x00560cef in listener_thread () #17 0x7f61f69a60a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #18 0x7f61f66db62d in clone () from /lib/x86_64-linux-gnu/libc.so.6 # Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7f204f922d63 in apr_pool_cleanup_kill () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 (gdb) bt #0 0x7f204f922d63 in apr_pool_cleanup_kill () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #1 0x7f204f922e21 in apr_pool_cleanup_run () from /usr/lib/x86_64-linux-gnu/libapr-1.so.0 #2 0x0055ffe9 in process_lingering_close () #3 0x00560cef in listener_thread () #4 0x7f204f6f00a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #5 0x7f204f42562d in clone () from /lib/x86_64-linux-gnu/libc.so.6 full bt: http://pastebin.com/raw/bP1vaYjw # Greets, Stefan Am 19.01.2017 um 16:47 schrieb Stefan Priebe - Profihost AG: arg sorry my fault. Here is a complete trace: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7fc1c23e0f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 (gdb) bt #0 0x7fc1c23e0f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 #1 0x0052a8f1 in h2_util_bb_avail () #2 0x00521eaa in h2_stream_out_prepare () #3 0x0051a55a in on_stream_resume () #4 0x0050bdff in h2_mplx_dispatch_master_events () #5 0x0051dc32 in h2_session_process () #6 0x00500115 in h2_conn_run () #7 0x00504b51 in h2_h2_process_conn () #8 0x0047cb19 in ap_run_process_connection () #9 0x0055e755 in process_socket () #10 0x00560f5c in worker_thread () #11 0x7fc1c1f8d0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #12 0x7fc1c1cc262d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 19.01.2017 um 16:45 schrieb Stefan Priebe - Profihost AG: Am 19.01.2017 um 16:34 s
Re: mod_http2 and Frequent wake-ups for mpm_event
On Thu, Jan 19, 2017 at 4:45 PM, Stefan Priebe - Profihost AG wrote: > > Am 19.01.2017 um 16:34 schrieb Stefan Eissing: >> Yann might already have asked this: any chance to compile with symbols and >> get a more readable stacktrace? > > Yes just tell me how ;-) i'm using dpkg-buildpackage and dh_strip. I've > no idea why not all symbols are available. dh_strip will strip the symbols, so you probably should not use it.
Re: mod_http2 and Frequent wake-ups for mpm_event
arg sorry my fault. Here is a complete trace: Program terminated with signal SIGSEGV, Segmentation fault. #0 0x7fc1c23e0f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 (gdb) bt #0 0x7fc1c23e0f23 in apr_brigade_length () from /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0 #1 0x0052a8f1 in h2_util_bb_avail () #2 0x00521eaa in h2_stream_out_prepare () #3 0x0051a55a in on_stream_resume () #4 0x0050bdff in h2_mplx_dispatch_master_events () #5 0x0051dc32 in h2_session_process () #6 0x00500115 in h2_conn_run () #7 0x00504b51 in h2_h2_process_conn () #8 0x0047cb19 in ap_run_process_connection () #9 0x0055e755 in process_socket () #10 0x00560f5c in worker_thread () #11 0x7fc1c1f8d0a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0 #12 0x7fc1c1cc262d in clone () from /lib/x86_64-linux-gnu/libc.so.6 Stefan Am 19.01.2017 um 16:45 schrieb Stefan Priebe - Profihost AG: > > Am 19.01.2017 um 16:34 schrieb Stefan Eissing: >> Yann might already have asked this: any chance to compile with symbols and >> get a more readable stacktrace? > > Yes just tell me how ;-) i'm using dpkg-buildpackage and dh_strip. I've > no idea why not all symbols are available. > > Do i need to pass a specific option to configure > > Stefan > >>> Am 19.01.2017 um 16:30 schrieb Stefan Priebe - Profihost AG >>> : >>> >>> With stock 2.4.25 + patch i'm getting this one again: >>> (gdb) bt >>> #0 0x00521dcd in h2_stream_out_prepare () >>> #1 0x7fc1a2feca80 in ?? () >>> #2 0x7fc1a2feca8c in ?? () >>> #3 0x7fc1a2feca90 in ?? () >>> #4 0x7fc1a057c0a0 in ?? () >>> #5 0x7fc1a057cdd8 in ?? () >>> #6 0x7fc1a2fecac0 in ?? () >>> #7 0x in ?? () >>> >>> Stefan >>> >>> Am 19.01.2017 um 16:28 schrieb Stefan Priebe - Profihost AG: I'm now testing stock 2.4.25 + patch. May this configure option have an influence? --enable-nonportable-atomics=yes Greets, Stefan Am 19.01.2017 um 15:35 schrieb Yann Ylavic: > Hi, > > On Thu, Jan 19, 2017 at 3:00 PM, Stefan Priebe - Profihost AG > wrote: >> >> @Yann: >> should i use V7 or V6? > > I'd prefer you'd use none (such that we can verify the patch with > stock 2.4.25, modulo mod_http2), but if it's easier for you to > reproduce with an event's patch, please use the v6 (and if it fails > then v7, and if it fails then no patch, really). > >> >> Stefan Eissing >> >> bytes GmbH >> Hafenstrasse 16 >> 48155 Münster >> www.greenbytes.de >>
Re: mod_http2 and Frequent wake-ups for mpm_event
Am 19.01.2017 um 16:34 schrieb Stefan Eissing: > Yann might already have asked this: any chance to compile with symbols and > get a more readable stacktrace? Yes just tell me how ;-) i'm using dpkg-buildpackage and dh_strip. I've no idea why not all symbols are available. Do i need to pass a specific option to configure Stefan >> Am 19.01.2017 um 16:30 schrieb Stefan Priebe - Profihost AG >> : >> >> With stock 2.4.25 + patch i'm getting this one again: >> (gdb) bt >> #0 0x00521dcd in h2_stream_out_prepare () >> #1 0x7fc1a2feca80 in ?? () >> #2 0x7fc1a2feca8c in ?? () >> #3 0x7fc1a2feca90 in ?? () >> #4 0x7fc1a057c0a0 in ?? () >> #5 0x7fc1a057cdd8 in ?? () >> #6 0x7fc1a2fecac0 in ?? () >> #7 0x in ?? () >> >> Stefan >> >> Am 19.01.2017 um 16:28 schrieb Stefan Priebe - Profihost AG: >>> I'm now testing stock 2.4.25 + patch. >>> >>> May this configure option have an influence? >>> --enable-nonportable-atomics=yes >>> >>> Greets, >>> Stefan >>> >>> Am 19.01.2017 um 15:35 schrieb Yann Ylavic: Hi, On Thu, Jan 19, 2017 at 3:00 PM, Stefan Priebe - Profihost AG wrote: > > @Yann: > should i use V7 or V6? I'd prefer you'd use none (such that we can verify the patch with stock 2.4.25, modulo mod_http2), but if it's easier for you to reproduce with an event's patch, please use the v6 (and if it fails then v7, and if it fails then no patch, really). > > Stefan Eissing > > bytes GmbH > Hafenstrasse 16 > 48155 Münster > www.greenbytes.de >
Re: mod_http2 and Frequent wake-ups for mpm_event
Yann might already have asked this: any chance to compile with symbols and get a more readable stacktrace? > Am 19.01.2017 um 16:30 schrieb Stefan Priebe - Profihost AG > : > > With stock 2.4.25 + patch i'm getting this one again: > (gdb) bt > #0 0x00521dcd in h2_stream_out_prepare () > #1 0x7fc1a2feca80 in ?? () > #2 0x7fc1a2feca8c in ?? () > #3 0x7fc1a2feca90 in ?? () > #4 0x7fc1a057c0a0 in ?? () > #5 0x7fc1a057cdd8 in ?? () > #6 0x7fc1a2fecac0 in ?? () > #7 0x in ?? () > > Stefan > > Am 19.01.2017 um 16:28 schrieb Stefan Priebe - Profihost AG: >> I'm now testing stock 2.4.25 + patch. >> >> May this configure option have an influence? >> --enable-nonportable-atomics=yes >> >> Greets, >> Stefan >> >> Am 19.01.2017 um 15:35 schrieb Yann Ylavic: >>> Hi, >>> >>> On Thu, Jan 19, 2017 at 3:00 PM, Stefan Priebe - Profihost AG >>> wrote: @Yann: should i use V7 or V6? >>> >>> I'd prefer you'd use none (such that we can verify the patch with >>> stock 2.4.25, modulo mod_http2), but if it's easier for you to >>> reproduce with an event's patch, please use the v6 (and if it fails >>> then v7, and if it fails then no patch, really). >>> Stefan Eissing bytes GmbH Hafenstrasse 16 48155 Münster www.greenbytes.de
Re: mod_http2 and Frequent wake-ups for mpm_event
On Thu, Jan 19, 2017 at 4:28 PM, Stefan Priebe - Profihost AG
wrote:
>
> May this configure option have an influence?
> --enable-nonportable-atomics=yes
Sure, for speed, but not correctness ("yes" never caused an issue for
me/linux, AFAICT).
