On Mon, Nov 23, 2020 at 11:12 AM Yann Ylavic <[email protected]> wrote:
>
> I think that apr-1.7.0 and apr-util-1.6.1 are missing r1481186 and
> r1822315 respectively. Both fixed quite some APR_POOL_DEBUG
> correctness according to my late ASAN testing.
> They were backported already, but there was no apr[-util] releases since then.
Nope, didn't help.
>
> I'll try running ab with and without them, and see how it goes..
> Also, Rüdiger (in some thread) proposed to lock pool_destroy_debug()
> like it's done for pool_clear_debug() already, that looks interesting
> too.
That helped! Stressing with ab and using this patch:
Index: srclib/apr-trunk/memory/unix/apr_pools.c
===================================================================
--- srclib/apr-trunk/memory/unix/apr_pools.c (revision 1883742)
+++ srclib/apr-trunk/memory/unix/apr_pools.c (working copy)
@@ -1951,10 +1951,8 @@ APR_DECLARE(void) apr_pool_clear_debug(apr_pool_t
static void pool_destroy_debug(apr_pool_t *pool, const char *file_line)
{
- apr_pool_check_lifetime(pool);
+ apr_pool_clear_debug(pool, file_line);
- pool_clear_debug(pool, file_line);
-
#if (APR_POOL_DEBUG & APR_POOL_DEBUG_VERBOSE)
apr_pool_log_event(pool, "DESTROY", file_line, 1);
#endif /* (APR_POOL_DEBUG & APR_POOL_DEBUG_VERBOSE) */):
--
No more use-after-free, while without this patch there are many ones
(like in the attached ASAN report).
Note: pool_clear_debug is not mutex protected, while apr_pool_clear_debug is.
Could we run APR_POOL_DEBUG test --with-inclued-apr (only) on travis,
or fetching the sources causes issues still?
Not sure distros pick up APR_POOL_DEBUG changes in their libapr..
Or we might simply accept ci sporadic failures with APR_POOL_DEBUG until then.
Regards;
Yann.
=================================================================
==2026856==ERROR: AddressSanitizer: heap-use-after-free on address
0x60600025acf0 at pc 0x7fe738f4c5be bp 0x7fe718598110 sp 0x7fe718598108
READ of size 8 at 0x60600025acf0 thread T51
#0 0x7fe738f4c5bd in apr_thread_mutex_lock locks/unix/thread_mutex.c:124
#1 0x7fe738f4e01c in apr_pool_walk_tree memory/unix/apr_pools.c:1505
#2 0x7fe738f4e066 in apr_pool_walk_tree memory/unix/apr_pools.c:1511
#3 0x7fe738f4e066 in apr_pool_walk_tree memory/unix/apr_pools.c:1511
#4 0x7fe738f4e066 in apr_pool_walk_tree memory/unix/apr_pools.c:1511
#5 0x7fe738f5027c in apr_pool_find memory/unix/apr_pools.c:2291
#6 0x7fe738f14aba in apr_table_mergen tables/apr_tables.c:746
#7 0x5578ad926a25 in ap_set_keepalive
/home/yle/src/apache/httpd/trunk.ro/modules/http/http_protocol.c:309
#8 0x5578ad93933f in ap_http_header_filter
/home/yle/src/apache/httpd/trunk.ro/modules/http/http_filters.c:1376
#9 0x5578ad98f7bd in ap_pass_brigade
/home/yle/src/apache/httpd/trunk.ro/server/util_filter.c:783
#10 0x5578ad9a67f3 in ap_content_length_filter
/home/yle/src/apache/httpd/trunk.ro/server/protocol.c:2046
#11 0x5578ad98f7bd in ap_pass_brigade
/home/yle/src/apache/httpd/trunk.ro/server/util_filter.c:783
#12 0x5578ad9405ae in ap_byterange_filter
/home/yle/src/apache/httpd/trunk.ro/modules/http/byterange_filter.c:463
#13 0x5578ad98f7bd in ap_pass_brigade
/home/yle/src/apache/httpd/trunk.ro/server/util_filter.c:783
#14 0x7fe7330e398b in ap_headers_output_filter
/home/yle/src/apache/httpd/trunk.ro/modules/metadata/mod_headers.c:891
#15 0x5578ad98f7bd in ap_pass_brigade
/home/yle/src/apache/httpd/trunk.ro/server/util_filter.c:783
#16 0x7fe732e32dba in session_output_filter
/home/yle/src/apache/httpd/trunk.ro/modules/session/mod_session.c:501
#17 0x5578ad98f7bd in ap_pass_brigade
/home/yle/src/apache/httpd/trunk.ro/server/util_filter.c:783
#18 0x5578ad9c8ee5 in default_handler
/home/yle/src/apache/httpd/trunk.ro/server/core.c:5188
#19 0x5578ad9431bb in ap_run_handler
/home/yle/src/apache/httpd/trunk.ro/server/config.c:170
#20 0x5578ad944941 in ap_invoke_handler
/home/yle/src/apache/httpd/trunk.ro/server/config.c:444
#21 0x5578ad92cc23 in ap_process_async_request
/home/yle/src/apache/httpd/trunk.ro/modules/http/http_request.c:463
#22 0x5578ad924d7c in ap_process_http_async_connection
/home/yle/src/apache/httpd/trunk.ro/modules/http/http_core.c:158
#23 0x5578ad925410 in ap_process_http_connection
/home/yle/src/apache/httpd/trunk.ro/modules/http/http_core.c:252
#24 0x5578ad97e04d in ap_run_process_connection
/home/yle/src/apache/httpd/trunk.ro/server/connection.c:42
#25 0x7fe735c7ef79 in process_socket
/home/yle/src/apache/httpd/trunk.ro/server/mpm/event/event.c:1097
#26 0x7fe735c856a0 in worker_thread
/home/yle/src/apache/httpd/trunk.ro/server/mpm/event/event.c:2386
#27 0x7fe738f7cef4 in dummy_worker threadproc/unix/thread.c:145
#28 0x7fe738e3eea6 in start_thread nptl/pthread_create.c:477
#29 0x7fe738d6ed4e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xfdd4e)
0x60600025acf0 is located 48 bytes inside of 64-byte region
[0x60600025acc0,0x60600025ad00)
freed by thread T63 here:
#0 0x7fe7391ed277 in __interceptor_free
(/usr/lib/x86_64-linux-gnu/libasan.so.5+0x107277)
#1 0x7fe738f4e9e5 in pool_clear_debug memory/unix/apr_pools.c:1893
#2 0x7fe738f4ecb2 in pool_destroy_debug memory/unix/apr_pools.c:1956
#3 0x7fe738f4eeeb in apr_pool_destroy_debug memory/unix/apr_pools.c:2002
#4 0x5578ada2534b in ap_queue_info_push_pool
/home/yle/src/apache/httpd/trunk.ro/server/mpm_fdqueue.c:230
#5 0x7fe735c81412 in process_lingering_close
/home/yle/src/apache/httpd/trunk.ro/server/mpm/event/event.c:1686
#6 0x7fe735c7f9bc in process_socket
/home/yle/src/apache/httpd/trunk.ro/server/mpm/event/event.c:1255
#7 0x7fe735c856a0 in worker_thread
/home/yle/src/apache/httpd/trunk.ro/server/mpm/event/event.c:2386
#8 0x7fe738f7cef4 in dummy_worker threadproc/unix/thread.c:145
#9 0x7fe738e3eea6 in start_thread nptl/pthread_create.c:477
