Correct me if I am wrong but the command you ran on the cluster seems to be doing a CRC check as well. I am still a novice to hadoop but that is the most obvious thing i see in the output below.
--- Regards, Jonathan Aquilina Founder Eagle Eye T On 2015-08-07 12:34, Shashi Vishwakarma wrote: > Hi > > I have a small confusion regarding checksum verification.Lets say , i have a > file abc.txt and I transferred this file to hdfs. How do I ensure about data > integrity? > > I followed below steps to check that file is correctly transferred. > > ON LOCAL FILE SYSTEM: > > md5sum abc.txt > > 276fb620d097728ba1983928935d6121 TestFile > > ON HADOOP CLUSTER : > > hadoop fs -checksum /abc.txt > > /abc.txt MD5-of-0MD5-of-512CRC32C > 000002000000000000000000911156a9cf0d906c56db7c8141320df0 > > Both output looks different to me. Let me know if I am doing anything wrong. > > How do I verify if my file is transferred properly into HDFS? > > Thanks > Shashi
