Hi Scott- On Wed, Mar 02, 2011 at 11:05:04AM -0600, Scott Golby wrote: > How can I do a md5sum/sha1sum directly against files on HDFS ?
I'm not sure if you meant to exclude this option, but mounting HDFS via FUSE would allow you to call md5sum(1) as usual. I've used this approach to migrate several hundred TB from our old distributed storage system to HDFS. FUSE certainly adds overhead, but it's the best way to meet your 'standard linux command' requirement. -- Will Maier - UW High Energy Physics cel: 608.438.6162 tel: 608.263.9692 web: http://www.hep.wisc.edu/~wcmaier/
