Lately, I've been seeing httpd/mod_proxy seg faulting in reverse proxy
setups, frequency increasing.
#0 apr_palloc (pool=0x8b52518, in_size=16) at memory/unix/apr_pools.c:684
#1 0xf756fc10 in apr_pool_cleanup_register (p=0x8b52518, data=0x8b52528,
plain_cleanup_fn=0xf756edb0 <thread_cond_cleanup>,
child_cleanup_fn=0x8069e20 <apr_pool_cleanup_null@plt>) at
memory/unix/apr_pools.c:2218
#2 0xf756ef9c in apr_thread_cond_create (cond=0x8b524e0, pool=0x8b52390)
at locks/unix/thread_cond.c:55
#3 0xf76fac64 in apr_reslist_create (reslist=0x8a6c488, min=0, smax=50,
hmax=50, ttl=0, con=0xf72cb8d0 <connection_constructor>, de=0xf72cb890
<connection_destructor>,
params=0x89a8268, pool=0x8b52390) at misc/apr_reslist.c:299
#4 0xf72cc0d8 in ap_proxy_initialize_worker (worker=0x89a8268,
s=0x89b3a50, p=0x890b0a8) at proxy_util.c:1751
#5 0xf72c7d8f in proxy_handler (r=0x8b3e1f0) at mod_proxy.c:1011
#6 0x0808ba41 in ap_run_handler (r=0x8b3e1f0) at config.c:168
#7 0x0808f8e6 in ap_invoke_handler (r=0x8b3e1f0) at config.c:432
#8 0x080a200a in ap_process_async_request (r=0x8b3e1f0) at
http_request.c:317
#9 0x080a213d in ap_process_request (r=0x8b3e1f0) at http_request.c:363
#10 0x0809ea60 in ap_process_http_sync_connection (c=<optimized out>) at
http_core.c:190
#11 ap_process_http_connection (c=0x8b3a390) at http_core.c:231
#12 0x08095e51 in ap_run_process_connection (c=0x8b3a390) at connection.c:41
#13 0x080a9470 in process_socket (bucket_alloc=<optimized out>,
my_thread_num=<optimized out>, my_child_num=<optimized out>,
sock=<optimized out>, p=<optimized out>,
thd=<optimized out>) at worker.c:620
#14 worker_thread (thd=0x8a6ddf8, dummy=0x8ac73f8) at worker.c:979
#15 0xf757cf96 in dummy_worker (opaque=0x8a6ddf8) at
threadproc/unix/thread.c:142
#16 0xf7503809 in start_thread () from /lib/libpthread.so.0
#17 0xf746430e in clone () from /lib/libc.so.6
Backtrace stopped: Not enough registers or memory available to unwind
further
(gdb) frame 0
#0 apr_palloc (pool=0x8b52518, in_size=16) at memory/unix/apr_pools.c:684
684 in memory/unix/apr_pools.c
(gdb) info locals
active = 0x0
node = <optimized out>
mem = <optimized out>
size = 16
free_index = <optimized out>
This is awkward. Shall I assume httpd ran out of memory ? That's about the
only plausible reason I could see why the apr would run into active=0x0.
#0 0xf74aa2ed in pthread_mutex_lock () from /lib/libpthread.so.0
#1 0xf75140c0 in apr_thread_mutex_lock (mutex=0x0) at
locks/unix/thread_mutex.c:92
#2 0xf769f702 in apr_reslist_maintain (reslist=0x9dc2da8) at
misc/apr_reslist.c:184
#3 0xf769fc79 in apr_reslist_create (reslist=0x9dbe448, min=0, smax=50,
hmax=50, ttl=0, con=0xf72708d0 <connection_constructor>, de=0xf7270890
<connection_destructor>,
params=0x9cfa268, pool=0x9dc2cb0) at misc/apr_reslist.c:305
#4 0xf72710d8 in ap_proxy_initialize_worker (worker=0x9cfa268,
s=0x9d05a50, p=0x9c5d0a8) at proxy_util.c:1751
#5 0xf726cd8f in proxy_handler (r=0xdc9004c0) at mod_proxy.c:1011
#6 0x0808ba41 in ap_run_handler (r=0xdc9004c0) at config.c:168
#7 0x0808f8e6 in ap_invoke_handler (r=0xdc9004c0) at config.c:432
#8 0x080a200a in ap_process_async_request (r=0xdc9004c0) at
http_request.c:317
#9 0x080a213d in ap_process_request (r=0xdc9004c0) at http_request.c:363
#10 0x0809ea60 in ap_process_http_sync_connection (c=<optimized out>) at
http_core.c:190
#11 ap_process_http_connection (c=0x9e5f940) at http_core.c:231
#12 0x08095e51 in ap_run_process_connection (c=0x9e5f940) at connection.c:41
#13 0x080a9470 in process_socket (bucket_alloc=<optimized out>,
my_thread_num=<optimized out>, my_child_num=<optimized out>,
sock=<optimized out>, p=<optimized out>,
thd=<optimized out>) at worker.c:620
#14 worker_thread (thd=0x9dbfd98, dummy=0x9e19318) at worker.c:979
#15 0xf7521f96 in dummy_worker (opaque=0x9dbfd98) at
threadproc/unix/thread.c:142
#16 0xf74a8809 in start_thread () from /lib/libpthread.so.0
#17 0xf740930e in clone () from /lib/libc.so.6
This is even stranger: it's trying to lock on a null pointer.
I'm just starting to dig into this myself. Has anyone seen stuff like this
(recently) ? I checked the bug reports but didn't see anything obvious.
