[
https://issues.apache.org/jira/browse/QPID-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033898#comment-13033898
]
Cliff Jansen commented on QPID-3256:
------------------------------------
I can reproduce the problem with a program that is not linked against
qpidclientd but loads and unloads it directly. A deadlock occurs
processing the global static destructors.
Thread 1 blocks waiting for thread 2 to exit/die. It also owns the
infamous loader lock (thanks to the use of FreeLibrary())
ntdll!NtWaitForSingleObject+0xa
KERNELBASE!WaitForSingleObjectEx+0x79
qpidcommond!qpid::sys::Thread::join+0x66 [thread.cpp @ 82]
qpidclientd!qpid::client::`anonymous namespace'::IOThread::~IOThread+0x141
qpidclientd!`qpid::client::`anonymous namespace'::theIO'::`2'::`dynamic atexit
destructor for 'io''+0x26
qpidclientd!_CRT_INIT+0x2dc
qpidclientd!__DllMainCRTStartup+0x11f
qpidclientd!_DllMainCRTStartup+0x31
ntdll!LdrpUnloadDll+0x27d
ntdll!LdrUnloadDll+0x4a
KERNELBASE!FreeLibrary+0x1d
foo!main+0x112
Thread 2 is a zombie trying to die, if only it could get the loader lock
ntdll!NtWaitForSingleObject+0xa
ntdll!RtlpWaitOnCriticalSection+0xe8
ntdll!RtlEnterCriticalSection+0xd1
ntdll!LdrShutdownThread+0x72
ntdll!RtlExitUserThread+0x38
MSVCR90D!_endthreadex+0x33
qpidcommond!`anonymous namespace'::runRunnable+0x3b [thread.cpp @ 34]
MSVCR90D!_callthreadstartex+0x25
MSVCR90D!_threadstartex+0xbd
kernel32!BaseThreadInitThunk+0xd
ntdll!RtlUserThreadStart+0x1d
Deadlock: Thread 1 does not wakeup until Thread 2 really dies, hence
neither thread can make progress.
Thread 2 has nothing left to do except the Windows DLL_THREAD_DETACH
handshake in all associated DLLs. It has clearly finished from the
point of view of the Windows implementation of the Qpid Thread class.
One way to fix this would be to use a private synchronization
mechanism that completes before the call to endthreadex(). I will put
a patch together for review along these lines unless someone objects.
> Application which uses Qpid (in my case Excel) hangs on shutdown
> ----------------------------------------------------------------
>
> Key: QPID-3256
> URL: https://issues.apache.org/jira/browse/QPID-3256
> Project: Qpid
> Issue Type: Bug
> Components: C++ Client
> Affects Versions: 0.8, 0.10
> Environment: OS: Windows.
> Qpid is assembled as DLL.
> Reporter: Eugene
>
> Hi All
> I encountered with strange behavior on shutdown when using qpid 0-8 and 0-10.
> When I use qpid in standalone console-application everything is ok. But when
> I use qpid in DLL which is loaded into Excel (as RTD module), Excel hangs on
> shutdown.
> I found out that in standalone application on shutdown I have next stack:
> qpidclientd.dll!qpid::client::`anonymous
> namespace'::IOThread::~IOThread() Line 138 C++
> qpidclientd.dll!`qpid::client::`anonymous
> namespace'::theIO'::`2'::`dynamic atexit destructor for 'io''() + 0xd bytes
> C++
> qpidclientd.dll!_CRT_INIT(void * hDllHandle=0x60080000, unsigned long
> dwReason=0, void * lpreserved=0x00000001) Line 449 C
> qpidclientd.dll!__DllMainCRTStartup(void * hDllHandle=0x60080000,
> unsigned long dwReason=0, void * lpreserved=0x00000001) Line 560 + 0x11
> bytes C
> qpidclientd.dll!_DllMainCRTStartup(void * hDllHandle=0x60080000,
> unsigned long dwReason=0, void * lpreserved=0x00000001) Line 510 + 0x11
> bytes C
> ntdll.dll!77b79960()
> [Frames below may be incorrect and/or missing, no symbols loaded for
> ntdll.dll]
> ntdll.dll!77b9a516()
> ntdll.dll!77b9a3b8()
> kernel32.dll!77657363()
> msvcr90d.dll!__crtExitProcess(int status=0) Line 732 C
> msvcr90d.dll!doexit(int code=0, int quick=0, int retcaller=0) Line 644
> + 0x9 bytes C
> msvcr90d.dll!exit(int code=0) Line 412 + 0xd bytes C
> Test.exe!__tmainCRTStartup() Line 599 C
> Test.exe!mainCRTStartup() Line 403 C
> kernel32.dll!77653677()
> ntdll.dll!77b79f02()
> ntdll.dll!77b79ed5()
> And in this state all threads of application have been already terminated.
> The only thread is:
> 1 > 21720 Main Thread Main Thread
> qpid::client::`anonymous namespace'::IOThread::~IOThread Normal 0
> so code from file ConnectionImpl.cpp works well:
> ~IOThread() {
> std::vector<Thread> threads;
> {
> ScopedLock<Mutex> l(threadLock);
> if (poller_)
> poller_->shutdown();
> t.swap(threads);
> }
> for (std::vector<Thread>::iterator i = threads.begin(); i !=
> threads.end(); ++i) {
> i->join();
> }
> }
> BUT in Excel I get stack:
> qpidclientd.dll!qpid::client::`anonymous namespace'::IOThread::~IOThread()
> Line 130 C++
> qpidclientd.dll!`qpid::client::`anonymous
> namespace'::theIO'::`2'::`dynamic atexit destructor for 'io''() + 0xd bytes
> C++
> qpidclientd.dll!_CRT_INIT(void * hDllHandle=0x07700000, unsigned long
> dwReason=0, void * lpreserved=0x00000000) Line 449 C
> qpidclientd.dll!__DllMainCRTStartup(void * hDllHandle=0x07700000,
> unsigned long dwReason=0, void * lpreserved=0x00000000) Line 560 + 0x11
> bytes C
> qpidclientd.dll!_DllMainCRTStartup(void * hDllHandle=0x07700000,
> unsigned long dwReason=0, void * lpreserved=0x00000000) Line 510 + 0x11
> bytes C
> ntdll.dll!77b79960()
> [Frames below may be incorrect and/or missing, no symbols loaded for
> ntdll.dll]
> ntdll.dll!77ba1525()
> ntdll.dll!77b81231()
> KernelBase.dll!77281da7()
> ole32.dll!75bb9562()
> ole32.dll!75bb9593()
> ole32.dll!75bb95a7()
> ole32.dll!75bb98bf()
> ole32.dll!75bb9805()
> ole32.dll!75bb9a8c()
> EXCEL.EXE!2f3811e9()
> EXCEL.EXE!2f6933e8()
> EXCEL.EXE!2f32a5af()
> EXCEL.EXE!2f34894a()
> EXCEL.EXE!2f670001()
> MSO.DLL!65bc6ed5()
> MSO.DLL!65c26a34()
> MSO.DLL!65c30305()
> MSO.DLL!65bc910c()
> MSO.DLL!65c4f420()
> MSO.DLL!65bbf161()
> comctl32.dll!7233463d()
> user32.dll!762971be()
> user32.dll!76297d31()
> user32.dll!76297dfa()
> EXCEL.EXE!2f324572()
> EXCEL.EXE!2f324534()
> EXCEL.EXE!2f324441()
> MSO.DLL!65b78116()
> MSO.DLL!65ba1fd0()
> EXCEL.EXE!2f30424b()
> msvcr90.dll!749936c5()
> msvcr90.dll!749938b3()
> msvcr90.dll!749938c5()
> msvcr90.dll!749ac40c()
> msvcr90.dll!749b028d()
> msvcr90.dll!749b04f3()
> EXCEL.EXE!2f303f0a()
> kernel32.dll!77653677()
> ntdll.dll!77b79f02()
> ntdll.dll!77b79ed5()
> And threads:
> 0 24016 Worker Thread _threadstartex _threadstartex Normal > 0
> 0 > 22928 Main Thread Main Thread
> qpid::client::`anonymous namespace'::IOThread::~IOThread Normal 0
> 0 20224 RPC Thread RPC Callback Thread 77b5fd21
> Normal 0
> 0 16492 Worker Thread Win32 Thread 77b61ed6 Normal > 0
> 0 19948 Worker Thread Win32 Thread 77b600ed Normal > 0
> 0 20524 Worker Thread Win32 Thread 77b61ed6 Normal > 0
> 0 20532 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 21500 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 21848 Worker Thread Win32 Thread 77b61ed6 Normal > 0
> 0 22152 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 22164 Worker Thread Win32 Thread 77b5f8e9 Normal > 0
> 0 22300 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 22360 Worker Thread Win32 Thread 77b61ed6 Normal > 0
> 0 23316 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 23556 Worker Thread Win32 Thread 77b5f8e9 Normal > 0
> 0 23700 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 23912 Worker Thread Win32 Thread 77b5f8e9 Normal > 0
> 0 24276 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 24308 Worker Thread Win32 Thread 77b5f861 Normal > 0
> 0 24424 Worker Thread Win32 Thread 77b600ed Normal > 0
> 0 24452 Worker Thread Win32 Thread 77b5f8e9 Normal > 0
> 0 24520 Worker Thread Win32 Thread 77b61ed6 Normal > 0
> As result this code (below) hangs application(Excel):
> ~IOThread() {
> std::vector<Thread> threads;
> {
> ScopedLock<Mutex> l(threadLock);
> if (poller_)
> poller_->shutdown();
> t.swap(threads);
> }
> for (std::vector<Thread>::iterator i = threads.begin(); i !=
> threads.end(); ++i)
> {
> i->join();-- APPLICATION HANGS HERE !!!!
> }
> }
> I suppose it occurs because of qpid tries to wait of thread completing from
> _DllMainCRTStartup which is called with PROCESS_DETACH.
> To work around it I modified function IOThread::sub():
> void sub() {
> std::vector<Thread> threads;
> {
> ScopedLock<Mutex> l(threadLock);
> --connections;
>
> if (connections == 0){
> if (poller_){
> poller_->shutdown();
> poller_.reset();
> t.swap(threads);
> }
> }
> }
> for (std::vector<Thread>::iterator i = threads.begin(); i !=
> threads.end(); ++i) {
> i->join();
> }
> }
> But I don't think it is a good solution.
> Could you help me to solve this problem?
> Thanks
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project: http://qpid.apache.org
Use/Interact: mailto:[email protected]
