[
https://issues.apache.org/jira/browse/DISPATCH-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324334#comment-17324334
]
Jiri Daněk commented on DISPATCH-2056:
--------------------------------------
Here's a better stacktrace, obtained by using malloc/free instead of poison. It
includes the free stack and the allocation stack is actually true (because the
pool chunks are not being reused multiple times here).
{noformat}
69: ======================================================================
69: ERROR: tearDownClass (system_tests_http2.Http2TestInteriorEdgeRouter)
69: ----------------------------------------------------------------------
69: Traceback (most recent call last):
69: File "/home/jdanek/repos/qpid/qpid-dispatch/tests/system_test.py", line
865, in tearDownClass
69: cls.tester.teardown()
69: File "/home/jdanek/repos/qpid/qpid-dispatch/tests/system_test.py", line
808, in teardown
69: raise RuntimeError("Errors during teardown: \n\n%s" %
"\n\n".join([str(e) for e in errors]))
69: RuntimeError: Errors during teardown:
69:
69: Process 14625 error: exit code 1, expected -1
69: qdrouterd -c interior-router.conf -I
/home/jdanek/repos/qpid/qpid-dispatch/python
69:
/home/jdanek/repos/qpid/qpid-dispatch/cmake-build-debug-asan/tests/system_test.dir/system_tests_http2/Http2TestInteriorEdgeRouter/setUpClass/interior-router-42.cmd
69: >>>>
69: =================================================================
69: ==14625==ERROR: AddressSanitizer: heap-use-after-free on address
0x618000040718 at pc 0x7fb8b66d1508 bp 0x7fb89ee31f40 sp 0x7fb89ee31f38
69: WRITE of size 8 at 0x618000040718 thread T4
69: #0 0x7fb8b66d1507 in qdr_connection_set_context
../src/router_core/connections.c:162
69: #1 0x7fb8b67d78cd in close_connections
../src/adaptors/http2/http2_adaptor.c:2132
69: #2 0x7fb8b67e8c9f in handle_disconnected
../src/adaptors/http2/http2_adaptor.c:2175
69: #3 0x7fb8b6802609 in handle_connection_event
../src/adaptors/http2/http2_adaptor.c:2422
69: #4 0x7fb8b67b0ee6 in handle_event_with_context ../src/server.c:804
69: #5 0x7fb8b67b0f96 in do_handle_raw_connection_event ../src/server.c:810
69: #6 0x7fb8b67be603 in handle ../src/server.c:1090
69: #7 0x7fb8b67bf9bd in thread_run ../src/server.c:1122
69: #8 0x7fb8b66a3deb in _thread_init ../src/posix/threading.c:174
69: #9 0x7fb8b60d7eac in start_thread
(/nix/store/q53f5birhik4dxg3q3r2g5f324n7r5mc-glibc-2.31-74/lib/libpthread.so.0+0x7eac)
69: #10 0x7fb8b5276cae in __GI___clone
(/nix/store/q53f5birhik4dxg3q3r2g5f324n7r5mc-glibc-2.31-74/lib/libc.so.6+0xf7cae)
69:
69: 0x618000040718 is located 664 bytes inside of 832-byte region
[0x618000040480,0x6180000407c0)
69: freed by thread T1 here:
69: #0 0x7fb8b6debb6f in __interceptor_free
(/nix/store/g40sl3zh3nv52vj0mrl4iki5iphh5ika-gcc-10.2.0-lib/lib/libasan.so.6+0xacb6f)
69: #1 0x7fb8b6617dae in qd_dealloc ../src/alloc_pool.c:503
69: #2 0x7fb8b66cfb20 in free_qdr_connection_t
../src/router_core/connections.c:44
69: #3 0x7fb8b66e6c68 in qdr_connection_free
../src/router_core/connections.c:1437
69: #4 0x7fb8b66e919d in qdr_connection_closed_CT
../src/router_core/connections.c:1553
69: #5 0x7fb8b674f4b1 in router_core_thread
../src/router_core/router_core_thread.c:240
69: #6 0x7fb8b66a3deb in _thread_init ../src/posix/threading.c:174
69: #7 0x7fb8b60d7eac in start_thread
(/nix/store/q53f5birhik4dxg3q3r2g5f324n7r5mc-glibc-2.31-74/lib/libpthread.so.0+0x7eac)
69:
69: previously allocated by thread T4 here:
69: #0 0x7fb8b6deca3c in __interceptor_posix_memalign
(/nix/store/g40sl3zh3nv52vj0mrl4iki5iphh5ika-gcc-10.2.0-lib/lib/libasan.so.6+0xada3c)
69: #1 0x7fb8b66148ea in qd_alloc ../src/alloc_pool.c:398
69: #2 0x7fb8b66cfaf0 in new_qdr_connection_t
../src/router_core/connections.c:44
69: #3 0x7fb8b66d01a2 in qdr_connection_opened
../src/router_core/connections.c:89
69: #4 0x7fb8b68018d3 in qdr_http_connection_ingress_accept
../src/adaptors/http2/http2_adaptor.c:2049
69: #5 0x7fb8b6801e98 in handle_connection_event
../src/adaptors/http2/http2_adaptor.c:2383
69: #6 0x7fb8b67b0ee6 in handle_event_with_context ../src/server.c:804
69: #7 0x7fb8b67b0f96 in do_handle_raw_connection_event ../src/server.c:810
69: #8 0x7fb8b67be603 in handle ../src/server.c:1090
69: #9 0x7fb8b67bf9bd in thread_run ../src/server.c:1122
69: #10 0x7fb8b66a3deb in _thread_init ../src/posix/threading.c:174
69: #11 0x7fb8b60d7eac in start_thread
(/nix/store/q53f5birhik4dxg3q3r2g5f324n7r5mc-glibc-2.31-74/lib/libpthread.so.0+0x7eac)
69:
69: Thread T4 created by T0 here:
69: #0 0x7fb8b6d972a2 in __interceptor_pthread_create
(/nix/store/g40sl3zh3nv52vj0mrl4iki5iphh5ika-gcc-10.2.0-lib/lib/libasan.so.6+0x582a2)
69: #1 0x7fb8b66a4744 in sys_thread ../src/posix/threading.c:183
69: #2 0x7fb8b67c1ef0 in qd_server_run ../src/server.c:1485
69: #3 0x40260e in main_process ../router/src/main.c:115
69: #4 0x403f4b in main ../router/src/main.c:369
69: #5 0x7fb8b51a2cbc in __libc_start_main
(/nix/store/q53f5birhik4dxg3q3r2g5f324n7r5mc-glibc-2.31-74/lib/libc.so.6+0x23cbc)
69:
69: Thread T1 created by T0 here:
69: #0 0x7fb8b6d972a2 in __interceptor_pthread_create
(/nix/store/g40sl3zh3nv52vj0mrl4iki5iphh5ika-gcc-10.2.0-lib/lib/libasan.so.6+0x582a2)
69: #1 0x7fb8b66a4744 in sys_thread ../src/posix/threading.c:183
69: #2 0x7fb8b6737b76 in qdr_core ../src/router_core/router_core.c:122
69: #3 0x7fb8b67ac756 in qd_router_setup_late ../src/router_node.c:2122
69: #4 0x7fb8b0c1aabc in ffi_call_unix64
(/nix/store/m8y5mz1f0al3rg3b56rq5bza49jjxnc0-libffi-3.3/lib/libffi.so.7+0x7abc)
69: #5 0x7ffe19ca302f ([stack]+0x1e02f)
69:
69: SUMMARY: AddressSanitizer: heap-use-after-free
../src/router_core/connections.c:162 in qdr_connection_set_context
69: Shadow bytes around the buggy address:
69: 0x0c3080000090: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
69: 0x0c30800000a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
69: 0x0c30800000b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
69: 0x0c30800000c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
69: 0x0c30800000d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
69: =>0x0c30800000e0: fd fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd
69: 0x0c30800000f0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
69: 0x0c3080000100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
69: 0x0c3080000110: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
69: 0x0c3080000120: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
69: 0x0c3080000130: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
69: Shadow byte legend (one shadow byte represents 8 application bytes):
69: Addressable: 00
69: Partially addressable: 01 02 03 04 05 06 07
69: Heap left redzone: fa
69: Freed heap region: fd
69: Stack left redzone: f1
69: Stack mid redzone: f2
69: Stack right redzone: f3
69: Stack after return: f5
69: Stack use after scope: f8
69: Global redzone: f9
69: Global init order: f6
69: Poisoned by user: f7
69: Container overflow: fc
69: Array cookie: ac
69: Intra object redzone: bb
69: ASan internal: fe
69: Left alloca redzone: ca
69: Right alloca redzone: cb
69: Shadow gap: cc
69: ==14625==ABORTING
{noformat}
> AddressSanitizer: use-after-poison in qdr_connection_set_context during
> system_tests_http2
> ------------------------------------------------------------------------------------------
>
> Key: DISPATCH-2056
> URL: https://issues.apache.org/jira/browse/DISPATCH-2056
> Project: Qpid Dispatch
> Issue Type: Bug
> Affects Versions: 1.16.0
> Reporter: Jiri Daněk
> Priority: Major
>
> The pool poison PR is new and untried, so this report needs to be taken with
> a portion of healthy scepticism.
> https://travis-ci.com/github/apache/qpid-dispatch/jobs/498888397#L30319
> {noformat}
> 72: =================================================================
> 3216172: ==18570==ERROR: AddressSanitizer: use-after-poison on address
> 0x61800006fb18 at pc 0x7ffa2c7dab05 bp 0x7ffa226d1190 sp 0x7ffa226d1188
> 3216272: WRITE of size 8 at 0x61800006fb18 thread T4
> 3216372: #0 0x7ffa2c7dab04 in qdr_connection_set_context
> /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:162:28
> 3216472: #1 0x7ffa2c6de93c in handle_disconnected
> /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:364:9
> 3216572: #2 0x7ffa2c6de93c in handle_connection_event
> /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:655:9
> 3216672: #3 0x7ffa2c908291 in handle
> /home/travis/build/apache/qpid-dispatch/src/server.c
> 3216772: #4 0x7ffa2c901c6f in thread_run
> /home/travis/build/apache/qpid-dispatch/src/server.c:1122:23
> 3216872: #5 0x7ffa2c363608 in start_thread
> (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
> 3216972: #6 0x7ffa2bb8e292 in clone
> (/lib/x86_64-linux-gnu/libc.so.6+0x122292)
> 3217072:
> 3217172: 0x61800006fb18 is located 664 bytes inside of 832-byte region
> [0x61800006f880,0x61800006fbc0)
> 3217272: allocated by thread T4 here:
> 3217372: #0 0x496f97 in posix_memalign
> (/home/travis/build/apache/qpid-dispatch/build/router/qdrouterd+0x496f97)
> 3217472: #1 0x7ffa2c6eff9e in qd_alloc
> /home/travis/build/apache/qpid-dispatch/src/alloc_pool.c:398:13
> 3217572: #2 0x7ffa2c7d4c8e in new_qdr_connection_t
> /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:44:1
> 3217672: #3 0x7ffa2c7d4c8e in qdr_connection_opened
> /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:89:32
> 3217772: #4 0x7ffa2c6e16f7 in qdr_tcp_open_server_side_connection
> /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:761:30
> 3217872: #5 0x7ffa2c6df1c0 in handle_connection_event
> /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:625:17
> 3217972: #6 0x7ffa2c908291 in handle
> /home/travis/build/apache/qpid-dispatch/src/server.c
> 3218072: #7 0x7ffa2c901c6f in thread_run
> /home/travis/build/apache/qpid-dispatch/src/server.c:1122:23
> 3218172: #8 0x7ffa2c363608 in start_thread
> (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
> 3218272:
> 3218372: Thread T4 created by T0 here:
> 3218472: #0 0x480f0a in pthread_create
> (/home/travis/build/apache/qpid-dispatch/build/router/qdrouterd+0x480f0a)
> 3218572: #1 0x7ffa2c7a7b9d in sys_thread
> /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:183:5
> 3218672: #2 0x7ffa2c90152e in qd_server_run
> /home/travis/build/apache/qpid-dispatch/src/server.c:1485:22
> 3218772: #3 0x4c7bbb in main_process
> /home/travis/build/apache/qpid-dispatch/router/src/main.c:115:5
> 3218872: #4 0x4c6876 in main
> /home/travis/build/apache/qpid-dispatch/router/src/main.c:369:9
> 3218972: #5 0x7ffa2ba930b2 in __libc_start_main
> (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> 3219072:
> 3219172: SUMMARY: AddressSanitizer: use-after-poison
> /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:162:28
> in qdr_connection_set_context
> 3219272: Shadow bytes around the buggy address:
> 3219372: 0x0c3080005f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 3219472: 0x0c3080005f20: 00 00 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3219572: 0x0c3080005f30: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3219672: 0x0c3080005f40: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3219772: 0x0c3080005f50: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3219872: =>0x0c3080005f60: f7 f7 f7[f7]f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3219972: 0x0c3080005f70: f7 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
> 3220072: 0x0c3080005f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
> 3220172: 0x0c3080005f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 3220272: 0x0c3080005fa0: 00 00 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3220372: 0x0c3080005fb0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
> 3220472: Shadow byte legend (one shadow byte represents 8 application bytes):
> 3220572: Addressable: 00
> 3220672: Partially addressable: 01 02 03 04 05 06 07
> 3220772: Heap left redzone: fa
> 3220872: Freed heap region: fd
> 3220972: Stack left redzone: f1
> 3221072: Stack mid redzone: f2
> 3221172: Stack right redzone: f3
> 3221272: Stack after return: f5
> 3221372: Stack use after scope: f8
> 3221472: Global redzone: f9
> 3221572: Global init order: f6
> 3221672: Poisoned by user: f7
> 3221772: Container overflow: fc
> 3221872: Array cookie: ac
> 3221972: Intra object redzone: bb
> 3222072: ASan internal: fe
> 3222172: Left alloca redzone: ca
> 3222272: Right alloca redzone: cb
> 3222372: Shadow gap: cc
> 3222472: ==18570==ABORTING
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
