Hi Rahul, one possibility could be system time updations: Can you check , System time changed in your system? Since the heartbeats will depends on System times, that will effect sending the heartbeats to NN.
Whihc version of hadoop are you using? approximately how many blocks will be there in DN? Regards, Uma ****************************************************************************************** This email and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained here in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this email in error, please notify the sender by phone or email immediately and delete it! ***************************************************************************************** ----- Original Message ----- From: Rahul Das <rahul.h...@gmail.com> Date: Wednesday, August 3, 2011 11:53 am Subject: Dananode not sending the Hearbeat messages to Namenode To: hdfs-user@hadoop.apache.org > Hi, > > I found a strange behavior in my cluster. The data nodes stop > sending any > information randomly (no logs coming). So the namenode thinks its > down. But > after some time ( approx 30 mints) the datanode nodes comes up and > startbehaving properly. I tried finding any error log, but the > datanode node is > not writing any error message during this time. > > The Namenode shows some warning similar to > > 2011-07-28 20:59:35,275 WARN > org.apache.hadoop.hdfs.server.namenode.FSNamesystem: > PendingReplicationMonitor timed out block > blk_8370263993564715002_23947922 > I checked this is not happening due to network outage or some > other process > eating up the CPU. > > Please help me with this. > -- > Rahul >
