Turns out there is a bug in the JDK:
https://issues.apache.org/jira/browse/ZOOKEEPER-427?focusedCommentId=12716020&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12716020
We are looking to see if we can solve this via some workaround in 3.2.
Currently the only so
[inf...@df3-8 infact]$ echo stats | nc localhost 2181
Zookeeper version: 3.1.1-755636, built on 03/18/2009 16:52 GMT
Clients:
/127.0.0.1:33139[1](queued=0,recved=0,sent=0)
/172.16.0.178:34283[1](queued=0,recved=925371,sent=925371)
Latency min/avg/max: 0/0/220
Received: 925371
Sent: 925382
Outsta
According to your trace I see you are using jvm 1.6.0_03-b05
One of the bugs fixed in:
http://java.sun.com/javase/6/webnotes/6u4.html
specifically:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6403933
seems to have a description very close to what you are seeing.
Perhaps you can try runnin
Created a Jira and attached logile + jstack file to it.
https://issues.apache.org/jira/secure/ManageAttachments.jspa?id=12426974
On Tue, Jun 2, 2009 at 1:50 PM, Mahadev Konar wrote:
> Hi Satish,
> Can you attach this trace to a jira? Please open one for this. Also, can
> you do the following -
Are the sent and received numbers going up quickly?
What is the second client doing?
On Tue, Jun 2, 2009 at 1:49 PM, Satish Bhatti wrote:
> stat
> Zookeeper version: 3.1.1-755636, built on 03/18/2009 16:52 GMT
> Clients:
> /127.0.0.1:42460[1](queued=0,recved=0,sent=0)
> /172.16.0.178:34283[1]
Hi Satish,
Can you attach this trace to a jira? Please open one for this. Also, can
you do the following -
For all the threads for the zookeeper server you are seeing the problem on,
Can you do an strace on all the threads and see which thread is spinning?
Also, can you upload the configs of t
stat
Zookeeper version: 3.1.1-755636, built on 03/18/2009 16:52 GMT
Clients:
/127.0.0.1:42460[1](queued=0,recved=0,sent=0)
/172.16.0.178:34283[1](queued=0,recved=925109,sent=925109)
Latency min/avg/max: 0/0/220
Received: 925109
Sent: 925109
Outstanding: 0
Zxid: 0x50051c7b3
Mode: follower
Node co
I am sooo glad I put in the fourth item. This is clearly not overload in
that sense.
Do you have logs from the ZK?
What does the stat command return (telnet to the ZK inquestion, type "stat"
(without the quotes)).
On Tue, Jun 2, 2009 at 1:35 PM, Satish Bhatti wrote:
> I have a grand total of
Hey Ted,
I have a grand total of 3 files, each holding a single long! Could it
really be in gc hell because of this? I passed in -Xmx256m on the command
line.
Satish
On Tue, Jun 2, 2009 at 12:52 PM, Ted Dunning wrote:
> I have seen this when I was over-loading our zookeeper instance. When
>
Mailing lists like this often strip attachments.
On Tue, Jun 2, 2009 at 1:29 PM, Satish Bhatti wrote:
> Hey Ben,
> Strange you didn't get the attachment, my gmail is showing the paper clip
> thingy for that message. ANyway, I have pasted the whole jstack output
> into
> this email, since it's p
Hey Ben,
Strange you didn't get the attachment, my gmail is showing the paper clip
thingy for that message. ANyway, I have pasted the whole jstack output into
this email, since it's pretty small.
Satish
2009-06-02 11:56:26
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_03-b05 mixed
mo
can you attach the jstack output? it seems to be missing from your email.
ben
Satish Bhatti wrote:
I am running a 5 node ZooKeeper cluster and I noticed that one of them
has very high CPU usage:
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
6883 infact 22
I have seen this when I was over-loading our zookeeper instance. When total
data or total number of files gets large, the system can wind up in GC
almost permanently. Zookeeper being Zookeeper, it does an amazing job of
keeping on, but eventually things go bad.
To test if this is your problem, y
13 matches
Mail list logo
