It seems that you ZK session has expired. This typically means that your ZK client is GC-ing. You can either fix the GC or increase the ZK session timeout.
Thanks, Jun On Mon, Jun 11, 2012 at 5:35 AM, 刘明敏 <diveintotomor...@gmail.com> wrote: > The topology of our kafka cluster is 1 producer,3 broker servers and 2 > consumers > The whole system runs well for several days,and yesterday we found that the > producer kept connect and disconnect and then re-connect to the zookeeper. > > below are the log that I thought might be relevant > * > * > *producer(10.18.10.4):* > [2012-06-11 17:25:55,315] INFO Socket connection established to zk-node-1/ > 10.18.10.1:2181, initiating session (org.apache.zookeeper.ClientCnxn) > [2012-06-11 17:25:55,339] INFO Session establishment complete on > serverzk-node-1/ > 10.18.10.1:2181, sessionid = 0x1379e52e0de154d, negotiated timeout = 6000 > (org.apache.zookeeper.ClientCnxn) > [2012-06-11 17:25:55,339] INFO zookeeper state changed (SyncConnected) > (org.I0Itec.zkclient.ZkClient) > [2012-06-11 17:27:00,220] INFO Client session timed out, have not heard > from server in 46827ms for sessionid 0x1379e52e0de154d, closing socket > connection and attempting reconnect (org.apache.zookeeper. ClientCnxn) > [2012-06-11 17:27:00,320] INFO zookeeper state changed (Disconnected) > (org.I0Itec.zkclient.ZkClient) > [2012-06-11 17:27:00,988] INFO Unable to reconnect to ZooKeeper service, > session 0x1379e52e0de154d has expired, closing socket connection > (org.apache.zookeeper.ClientCnxn) > > > *zk node 1(10.18.10.1):* > 25230 2012-06-11 17:25:55,365 - INFO [CommitProcessor:1:NIOServerCnxn@1580 > ] > - Established session 0x1379e52e0de154d with negotiated timeout 6000 for > client /10.18.10.4:36134 // 10.18.10.4 is producer > 25231 2012-06-11 17:26:22,004 - INFO [CommitProcessor:1:NIOServerCnxn@1435 > ] > - Closed socket connection for client /10.18.10.4:36134 which had > sessionid > 0x1379e52e0de154d > > > *zk node 2:* > 2012-06-11 17:26:22,001 - INFO [SessionTracker:ZooKeeperServer@316] - > Expiring session 0x1379e52e0de154d, timeout of 6000ms exceeded > 2012-06-11 17:26:22,001 - INFO [ProcessThread:-1:PrepRequestProcessor@399 > ] > - Processed session termination for sessionid: 0x1379e52e0de154d > 2012-06-11 17:27:01,011 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory@251] - Accepted socket > connection from /10.18.10.4:25923 > 2012-06-11 17:27:01,012 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@770] - Client attempting to renew > session 0x1379e52e0de154d at /10.18.10.4:25923 > 2012-06-11 17:27:01,012 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1573] - Invalid session > 0x1379e52e0de154dfor client > /10.18.10.4:25923, probably expired > 2012-06-11 17:27:01,013 - INFO [NIOServerCxn.Factory: > 0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1435] - Closed socket connection for > client /10.18.10.4:25923 which had sessionid 0x1379e52e0de154d > > > *zk node 3:* > (content has been rotated when I remembered to copy the content from > log,while it is similar with zk node 1 and 2) > > -- > Best Regards > > ---------------------- > 刘明敏 | mmLiu >
