[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12722554#action_12722554 ] Hudson commented on ZOOKEEPER-427: -- Integrated in ZooKeeper-trunk #354 (See [http://hudson.zones.apache.org/hudson/job/ZooKeeper-trunk/354/]) > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti >Assignee: Flavio Paiva Junqueira >Priority: Blocker > Fix For: 3.2.0 > > Attachments: zk_quorum_recv_eof.patch, zoo.cfg, ZOOKEEPER-427.patch, > zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12721454#action_12721454 ] Hadoop QA commented on ZOOKEEPER-427: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12411045/ZOOKEEPER-427.patch against trunk revision 786156. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/124/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/124/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: http://hudson.zones.apache.org/hudson/job/Zookeeper-Patch-vesta.apache.org/124/console This message is automatically generated. > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti >Assignee: Flavio Paiva Junqueira >Priority: Blocker > Fix For: 3.2.0 > > Attachments: zk_quorum_recv_eof.patch, zoo.cfg, ZOOKEEPER-427.patch, > zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715731#action_12715731 ] Mahadev konar commented on ZOOKEEPER-427: - no worries satish, I hope we could have gotten that trace though... We had seen similar behavior in another jira ZOOKEEPER-287, and wanted to make sure if both are caused by similar problems. I have linked these two together. Do update the jira in case you run into this again. > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti > Attachments: zoo.cfg, zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715723#action_12715723 ] Satish Bhatti commented on ZOOKEEPER-427: - oops! Sorry, Mahadev, I already bounced that zookeeper server, (it's a production server, so I didn't want to leave it flapping for too long) and it's been behaving well since. If I can reproduce the problem I will run strace as you have suggested. > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti > Attachments: zoo.cfg, zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715714#action_12715714 ] Mahadev konar commented on ZOOKEEPER-427: - satish for strace you will have to do an strace on all the pid's in /proc/6883/tasks/ to see which of the threads is spinning and on what... > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti > Attachments: zoo.cfg, zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715698#action_12715698 ] Satish Bhatti commented on ZOOKEEPER-427: - I ran: strace -o zookeeper-strace.log -p6883 So far nothing in the logfile. less zookeeper-strace.log futex(0x404079d0, FUTEX_WAIT, 6884, NULL > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti > Attachments: zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (ZOOKEEPER-427) ZooKeeper server unexpectedly high CPU utilisation
[ https://issues.apache.org/jira/browse/ZOOKEEPER-427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12715693#action_12715693 ] Mahadev konar commented on ZOOKEEPER-427: - satish, can you just try strace to see which one of thread is spinning? also, with the log files it seems like you are doing a lot of trasactions through some other zookeeper server? > ZooKeeper server unexpectedly high CPU utilisation > -- > > Key: ZOOKEEPER-427 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-427 > Project: Zookeeper > Issue Type: Bug >Affects Versions: 3.1.1 > Environment: Linux: 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST > 2008 x86_64 x86_64 x86_64 GNU/Linux > java version "1.6.0_03" > Java(TM) SE Runtime Environment (build 1.6.0_03-b05) > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0_03-b05, mixed mode) >Reporter: Satish Bhatti > Attachments: zookeeper-jstack.log, zookeeper.log > > > I am running a 5 node ZooKeeper cluster and I noticed that one of them has > very high CPU usage: > PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND > 6883 infact 22 0 725m 41m 4188 S 95 0.5 > 5671:54 java > It is not "doing anything" application-wise at this point, so I was wondering > why the heck it's using up so much CPU. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
