[
https://issues.apache.org/jira/browse/ZOOKEEPER-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15218284#comment-15218284
]
Michael Svendsen commented on ZOOKEEPER-2200:
---------------------------------------------
Came across this when searching for solutions to my own deadlock issues using
the same stack you describe (Net::ZooKeeper and Net::ZooKeeper::Lock v0.03)
My issue was caused by the race condition addressed in this open pull request
on Net::ZooKeeper::Lock -
https://github.com/tadam/Net-ZooKeeper-Lock/pull/2/files
Seems like this could also be your issue.
The pull request fixed the issue I had been encountering. Hope this helps!
> Perl ZooKeeper locks up during heavy load
> -----------------------------------------
>
> Key: ZOOKEEPER-2200
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-2200
> Project: ZooKeeper
> Issue Type: Bug
> Components: c client
> Affects Versions: 3.4.6
> Environment: CentOS 5.8
> Reporter: KARA VAN HORN
>
> We are using Perl Net::ZooKeeper (0.38) and Net::ZooKeeper::Lock (0.03)
> libraries. Deadlock appears to occur at the end during lock cleanup
> activity. Here is a stack dump (sensitive names changed):
> Thread 2 (Thread 0x2ac6fbfa3940 (LWP 13292)):
> #0 0x00002ac6f5aed654 in __lll_lock_wait () from /lib64/libpthread.so.0
> #1 0x00002ac6f5aeb47b in pthread_cond_signal@@GLIBC_2.3.2 () from
> /lib64/libpthread.so.0
> #2 0x00002ac6f835539c in _zk_watcher (handle=<value optimized out>, type=2,
> state=3, path=<value optimized out>, context=0x33f3ce0) at ZooKeeper.xs:179
> #3 0x00002ac6f856d942 in do_foreach_watcher (zh=0x33e4fb0, type=2, state=3,
> path=0x33f3f50 "/lock/cmts/cisco_device1.net-0001851215", list=0x33ed290)
> at /home/myhome/rpm/BUILD/zookeeper-3.4.6/src/c/src/zk_hashtable.c:279
> #4 deliverWatchers (zh=0x33e4fb0, type=2, state=3, path=0x33f3f50
> "/lock/cmts/cisco_device1.net-0001851215", list=0x33ed290)
> at /home/myhome/rpm/BUILD/zookeeper-3.4.6/src/c/src/zk_hashtable.c:321
> #5 0x00002ac6f8564966 in process_completions (zh=0x33e4fb0) at
> /home/myhome/rpm/BUILD/zookeeper-3.4.6/src/c/src/zookeeper.c:2114
> #6 0x00002ac6f856e101 in do_completion (v=<value optimized out>) at
> /home/myhome/rpm/BUILD/zookeeper-3.4.6/src/c/src/mt_adaptor.c:466
> #7 0x00002ac6f5ae683d in start_thread (arg=<value optimized out>) at
> pthread_create.c:301
> #8 0x00002ac6f5dd1fcd in clone () from /lib64/libc.so.6
> Thread 1 (Thread 0x2ac6f6056af0 (LWP 12972)):
> #0 0x00002ac6f5ae7c65 in pthread_join (threadid=47034119371072,
> thread_return=0x0) at pthread_join.c:89
> #1 0x00002ac6f856e7de in adaptor_finish (zh=0x33e4fb0) at
> /home/myhome/rpm/BUILD/zookeeper-3.4.6/src/c/src/mt_adaptor.c:293
> #2 0x00002ac6f8566cdc in zookeeper_close (zh=0x33e4fb0) at
> /home/myhome/rpm/BUILD/zookeeper-3.4.6/src/c/src/zookeeper.c:2536
> #3 0x00002ac6f8357222 in XS_Net__ZooKeeper_DESTROY (my_perl=0x20df010,
> cv=<value optimized out>) at ZooKeeper.xs:885
> #4 0x00002ac6f4b38af6 in Perl_pp_entersub () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #5 0x00002ac6f4adb8d7 in ?? () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #6 0x00002ac6f4adf720 in Perl_call_sv () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #7 0x00002ac6f4b3d3c6 in Perl_sv_clear () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #8 0x00002ac6f4b3db70 in Perl_sv_free () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #9 0x00002ac6f4b6025c in Perl_free_tmps () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #10 0x00002ac6f4adf78a in Perl_call_sv () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #11 0x00002ac6f4b3d3c6 in Perl_sv_clear () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #12 0x00002ac6f4b3db70 in Perl_sv_free () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #13 0x00002ac6f4b3b0e5 in ?? () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #14 0x00002ac6f4b3b141 in Perl_sv_clean_objs () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #15 0x00002ac6f4ae185e in perl_destruct () from
> /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/CORE/libperl.so
> #16 0x0000000000401773 in main ()
> There are about 4 out of 10,000 processes that end up in deadlock, and
> according to our web searches, the only reason pthread_cond_signal would lock
> is due to it waiting on an already destroyed condition.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
