Re: HDFS fsck command giving health as corrupt for '/'

2017-02-16 Thread Ravi Prakash
Hi Nishant! I'd suggest reading the HDFS user guide to begin with and becoming familiar with the architecture. https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html . Where are the blocks stored on the datanodes? Were they on persistent storage on the EC2

Re: HDFS fsck command giving health as corrupt for '/'

2017-02-16 Thread Nishant Verma
Hi Philippe Yes, I did. I restarted NameNode and other daemons multiple times. I found that all my files had got corrupted somehow. I was able to fix the issue by running below command: hdfs fsck / | egrep -v '^\.+$' | grep -v replica | grep -v Replica But it deleted all the files from my

Re: HDFS fsck command giving health as corrupt for '/'

2017-02-15 Thread Philippe Kernévez
Hi Nishant, You namenode are probably unable to comunicate with your datanode. Did you restart all the HDFS services ? Regards, Philipp On Tue, Feb 14, 2017 at 10:43 AM, Nishant Verma wrote: > Hi > > I have open source hadoop version 2.7.3 cluster (2 Masters + 3

HDFS fsck command giving health as corrupt for '/'

2017-02-14 Thread Nishant Verma
Hi I have open source hadoop version 2.7.3 cluster (2 Masters + 3 Slaves) installed on AWS EC2 instances. I am using the cluster to integrate it with Kafka Connect. The setup of cluster was done last month and setup of kafka connect was completed last fortnight. Since then, we were able to