Joris Van Remoortere created MESOS-3201:
-------------------------------------------
Summary: Libev handle_async can deadlock with run_in_event_loop
Key: MESOS-3201
URL: https://issues.apache.org/jira/browse/MESOS-3201
Project: Mesos
Issue Type: Bug
Components: libprocess
Affects Versions: 0.23.0
Reporter: Joris Van Remoortere
Assignee: Joris Van Remoortere
Due to the arbitrary nature of the functions that are executed in handle_async,
invoking them under the (A) {{watchers_mutex}} can lead to deadlocks if (B) is
acquired before calling {{run_in_event_loop}} and (B) is also acquired within
the arbitrary function.
{code}
==82679== Thread #10: lock order "0x60774F8 before 0x60768C0" violated
==82679==
==82679== Observed (incorrect) order is: acquisition of lock at 0x60768C0
==82679== at 0x4C32145: pthread_mutex_lock (in
/usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==82679== by 0x692C9B: __gthread_mutex_lock(pthread_mutex_t*)
(gthr-default.h:748)
==82679== by 0x6950BF: std::mutex::lock() (mutex:134)
==82679== by 0x696219: Synchronized<std::mutex>
synchronize<std::mutex>(std::mutex*)::{lambda(std::mutex*)#1}::operator()(std::mutex*)
const (synchronized.hpp:58)
==82679== by 0x696238: Synchronized<std::mutex>
synchronize<std::mutex>(std::mutex*)::{lambda(std::mutex*)#1}::_FUN(std::mutex*)
(synchronized.hpp:58)
==82679== by 0x6984CF: Synchronized<std::mutex>::Synchronized(std::mutex*,
void (*)(std::mutex*), void (*)(std::mutex*)) (synchronized.hpp:35)
==82679== by 0x6962DE: Synchronized<std::mutex>
synchronize<std::mutex>(std::mutex*) (synchronized.hpp:60)
==82679== by 0x728FE1: process::handle_async(ev_loop*, ev_async*, int)
(libev.cpp:48)
==82679== by 0x761384: ev_invoke_pending (ev.c:2994)
==82679== by 0x7643C4: ev_run (ev.c:3394)
==82679== by 0x728E37: ev_loop (ev.h:826)
==82679== by 0x729469: process::EventLoop::run() (libev.cpp:135)
==82679==
==82679== followed by a later acquisition of lock at 0x60774F8
==82679== at 0x4C32145: pthread_mutex_lock (in
/usr/lib/valgrind/vgpreload_helgrind-amd64-linux.so)
==82679== by 0x4C6F9D: __gthread_mutex_lock(pthread_mutex_t*)
(gthr-default.h:748)
==82679== by 0x4C6FED: __gthread_recursive_mutex_lock(pthread_mutex_t*)
(gthr-default.h:810)
==82679== by 0x4F5D3D: std::recursive_mutex::lock() (mutex:175)
==82679== by 0x516513: Synchronized<std::recursive_mutex>
synchronize<std::recursive_mutex>(std::recursive_mutex*)::{lambda(std::recursive_mutex*)#1}::operator()(std::recursive_mutex*)
const (synchronized.hpp:58)
==82679== by 0x516532: Synchronized<std::recursive_mutex>
synchronize<std::recursive_mutex>(std::recursive_mutex*)::{lambda(std::recursive_mutex*)#1}::_FUN(std::recursive_mutex*)
(synchronized.hpp:58)
==82679== by 0x52E619:
Synchronized<std::recursive_mutex>::Synchronized(std::recursive_mutex*, void
(*)(std::recursive_mutex*), void (*)(std::recursive_mutex*))
(synchronized.hpp:35)
==82679== by 0x5165D4: Synchronized<std::recursive_mutex>
synchronize<std::recursive_mutex>(std::recursive_mutex*) (synchronized.hpp:60)
==82679== by 0x6BF4E1: process::ProcessManager::use(process::UPID const&)
(process.cpp:2127)
==82679== by 0x6C2B8C: process::ProcessManager::terminate(process::UPID
const&, bool, process::ProcessBase*) (process.cpp:2604)
==82679== by 0x6C6C3C: process::terminate(process::UPID const&, bool)
(process.cpp:3107)
==82679== by 0x692B65: process::Latch::trigger() (latch.cpp:53)
{code}
This was introduced in
https://github.com/apache/mesos/commit/849fc4d361e40062073324153ba97e98e294fdf2
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
