[
https://issues.apache.org/jira/browse/ZOOKEEPER-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609829#action_12609829
]
Johannes Zillmann commented on ZOOKEEPER-57:
--------------------------------------------
Might be related, we're observing the following on the client side of a running
zookeeper system:
{noformat}
2008-07-01 05:16:12,643 WARN com.yahoo.zookeeper.server.NIOServerCnxn: Creating
new session 11ade8d31110000
2008-07-01 05:16:12,686 WARN com.yahoo.zookeeper.server.NIOServerCnxn: Finished
init of 11ade8d31110000: true
2008-07-01 05:16:13,646 WARN com.yahoo.zookeeper.server.ZooKeeperServer: Trying
to connect to hadoop.host/100.211.8.204:2181
2008-07-01 05:16:13,647 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Priming connection to java.nio.channels.SocketChannel[connected
local=/100.211.8.204:40166 remote=hadoop.host/100.211.8.204:2181]
2008-07-01 05:16:13,651 WARN com.yahoo.zookeeper.server.NIOServerCnxn:
Connected to /100.211.8.204:40166 lastZxid 0
2008-07-01 05:16:13,652 WARN
com.yahoo.zookeeper.serhadoop.hostver.NIOServerCnxn: Creating new session
11ade8d31110001
2008-07-01 05:16:13,665 WARN com.yahoo.zookeeper.server.NIOServerCnxn: Finished
init of 11ade8d31110001: true
2008-07-01 05:16:24,002 WARN com.yahoo.zookeeper.server.SessionTrackerImpl:
Expiring 11ad84b6b650001
2008-07-01 05:16:24,002 WARN com.yahoo.zookeeper.server.SessionTrackerImpl:
Expiring 11ad84b6b650000
2008-07-01 05:16:24,003 WARN com.yahoo.zookeeper.server.PrepRequestProcessor:
Processed session termination request for id: 11ad84b6b650001
2008-07-01 05:16:24,003 WARN com.yahoo.zookeeper.server.PrepRequestProcessor:
Processed session termination request for id: 11ad84b6b650000
2008-07-01 05:16:42,140 WARN com.yahoo.zookeeper.server.ZooKeeperServer: Trying
to connect to hadoop.host/100.211.8.204:2181
2008-07-01 05:16:42,160 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Priming connection to java.nio.channels.SocketChannel[connected
local=/100.211.8.204:45830 remote=hadoop.host/100.211.8.204:2181]
2008-07-01 05:16:43,480 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
Initializing RPC Metrics with hostName=Node, port=50000
2008-07-01 05:16:46,953 INFO org.apache.hadoop.ipc.Server: IPC Server
Responder: starting
2008-07-01 05:16:46,955 INFO org.apache.hadoop.ipc.Server: IPC Server listener
on 50000: starting
2008-07-01 05:16:46,955 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0
on 50000: starting
2008-07-01 06:07:25,641 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Closing:
java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 lim=4
cap=4]
at com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:484)
at com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
2008-07-01 06:07:27,775 WARN com.yahoo.zookeeper.server.ZooKeeperServer: Trying
to connect to hadoop.host/100.211.8.204:2181
2008-07-01 06:07:27,775 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Priming connection to java.nio.channels.SocketChannel[connected
local=/100.211.8.207:47535 remote=hadoop.host/100.211.8.204:2181]
2008-07-01 06:07:27,776 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Closing:
java.io.IOException: Session Expired
at
com.yahoo.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:406)
at com.yahoo.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:492)
at com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:705)
2008-07-01 06:07:33,294 ERROR com.yahoo.zookeeper.server.ZooKeeperServer: from
EventThread
java.lang.RuntimeException: Unable to re subscribe to child change notification
for: /katta/node-to-shard/hadoop1.host:50000
at net.sf.katta.zk.ZKClient.process(ZKClient.java:364)
at com.yahoo.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:264)
Caused by: com.yahoo.zookeeper.KeeperException: KeeperErrorCode = SessionExpired
at com.yahoo.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:634)
at net.sf.katta.zk.ZKClient.process(ZKClient.java:359)
... 1 more
{noformat}
At the same time on the server side you can see:
{noformat}
2008-07-01 05:16:24,002 WARN com.yahoo.zookeeper.server.SessionTrackerImpl:
Expiring 11ad84b6b650001
2008-07-01 05:16:24,002 WARN com.yahoo.zookeeper.server.SessionTrackerImpl:
Expiring 11ad84b6b650000
2008-07-01 05:16:24,003 WARN com.yahoo.zookeeper.server.PrepRequestProcessor:
Processed session termination request for id: 11ad84b6b650001
2008-07-01 05:16:24,003 WARN com.yahoo.zookeeper.server.PrepRequestProcessor:
Processed session termination request for id: 11ad84b6b650000
{noformat}
Some hours later this is in the server logs. Don't know if its related...
{noformat}
2008-07-01 15:52:59,798 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Closing:
java.io.IOException: TIMED OUT
at com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
2008-07-01 15:52:59,798 WARN com.yahoo.zookeeper.server.ZooKeeperServer:
Closing:
java.io.IOException: TIMED OUT
at com.yahoo.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:666)
{noformat}
> Connection times out when idle
> ------------------------------
>
> Key: ZOOKEEPER-57
> URL: https://issues.apache.org/jira/browse/ZOOKEEPER-57
> Project: Zookeeper
> Issue Type: Bug
> Components: java client
> Reporter: Flavio Paiva Junqueira
>
> I have observed several connection timeouts with one single client, during
> periods of inactivity. From reports of other users, it seems that others are
> observing a similar issue.
> In my case, tickTime is 2000, and the client sets session timeout to be
> 10000. Client and servers are on different sites. I suspect there is
> something wrong with the mechanism that issues ping messages.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.