Ken Giusti created DISPATCH-2130:
------------------------------------
Summary: python lock inversion deadlock
Key: DISPATCH-2130
URL: https://issues.apache.org/jira/browse/DISPATCH-2130
Project: Qpid Dispatch
Issue Type: Bug
Components: Router Node
Affects Versions: 1.16.0
Reporter: Ken Giusti
Assignee: Ken Giusti
Fix For: 1.17.0
The python lock is a top-level lock: no other locks should be held when taking
the python lock.
The python lock is held for as long as the thread is running in python. The
python code often calls back into C, which takes locks as necessary (timers,
server lock, whatever).
Debug code should be added to enforce this.
TSAN deadlock trace:
21: Mutex M253 acquired here while holding mutex M13 in main thread:
21: #0 pthread_mutex_lock <null> (libtsan.so.0+0x528ac)
<M253 == timer lock == qd_server->lock>
21: #1 sys_mutex_lock
/home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:57
(libqpid-dispatch.so+0xacee5)
21: #2 qd_timer_schedule
/home/kgiusti/work/dispatch/qpid-dispatch/src/timer.c:216
(libqpid-dispatch.so+0x1350fe)
21: #3 qd_router_setup_late
/home/kgiusti/work/dispatch/qpid-dispatch/src/router_node.c:2144
(libqpid-dispatch.so+0x128656)
21: #4 ffi_call_unix64 <null> (libffi.so.6+0x6aef)
21: #5 main_process
/home/kgiusti/work/dispatch/qpid-dispatch/router/src/main.c:97
(qdrouterd+0x402697)
21: #6 main /home/kgiusti/work/dispatch/qpid-dispatch/router/src/main.c:369
(qdrouterd+0x403557)
21:
21: Mutex M13 previously acquired by the same thread here:
21: #0 pthread_mutex_lock <null> (libtsan.so.0+0x528ac)
<M13 == python ilock>
21: #1 sys_mutex_lock
/home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:57
(libqpid-dispatch.so+0xacee5)
21: #2 qd_python_lock
/home/kgiusti/work/dispatch/qpid-dispatch/src/python_embedded.c:902
(libqpid-dispatch.so+0xb070e)
21: #3 qd_dispatch_load_config
/home/kgiusti/work/dispatch/qpid-dispatch/src/dispatch.c:129
(libqpid-dispatch.so+0x80ad2)
21: #4 main_process
/home/kgiusti/work/dispatch/qpid-dispatch/router/src/main.c:97
(qdrouterd+0x402697)
21: #5 main /home/kgiusti/work/dispatch/qpid-dispatch/router/src/main.c:369
(qdrouterd+0x403557)
---------------------------
21: Mutex M13 acquired here while holding mutex M253 in thread T4:
21: #0 pthread_mutex_lock <null> (libtsan.so.0+0x528ac)
<M13 == python ilock>
21: #1 sys_mutex_lock
/home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:57
(libqpid-dispatch.so+0xacee5)
21: #2 qd_python_lock
/home/kgiusti/work/dispatch/qpid-dispatch/src/python_embedded.c:902
(libqpid-dispatch.so+0xb070e)
21: #3 qd_policy_socket_close
/home/kgiusti/work/dispatch/qpid-dispatch/src/policy.c:296
(libqpid-dispatch.so+0xa3a8e)
21: #4 qd_connection_free
/home/kgiusti/work/dispatch/qpid-dispatch/src/server.c:925
(libqpid-dispatch.so+0x12f703)
21: #5 thread_run /home/kgiusti/work/dispatch/qpid-dispatch/src/server.c:1127
(libqpid-dispatch.so+0x13069e)
21: #6 _thread_init
/home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:172
(libqpid-dispatch.so+0xad37a)
21: #7 <null> <null> (libtsan.so.0+0x2d33f)
21:
21: Mutex M253 previously acquired by the same thread here:
21: #0 pthread_mutex_lock <null> (libtsan.so.0+0x528ac)
<M253 == qd_server->lock>
21: #1 sys_mutex_lock
/home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:57
(libqpid-dispatch.so+0xacee5)
21: #2 qd_connection_free
/home/kgiusti/work/dispatch/qpid-dispatch/src/server.c:920
(libqpid-dispatch.so+0x12f48f)
21: #3 thread_run /home/kgiusti/work/dispatch/qpid-dispatch/src/server.c:1127
(libqpid-dispatch.so+0x13069e)
21: #4 _thread_init
/home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:172
(libqpid-dispatch.so+0xad37a)
21: #5 <null> <null> (libtsan.so.0+0x2d33f)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
