[ https://issues.apache.org/jira/browse/HADOOP-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13887280#comment-13887280 ]
Hudson commented on HADOOP-10295: --------------------------------- SUCCESS: Integrated in Hadoop-trunk-Commit #5077 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5077/]) HADOOP-10295. Allow distcp to automatically identify the checksum type of source files and use it for the target. Contributed by Jing Zhao and Laurent Goujon. (jing9: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563019) * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileChecksum.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/MD5MD5CRC32FileChecksum.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java * /hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java > Allow distcp to automatically identify the checksum type of source files and > use it for the target > -------------------------------------------------------------------------------------------------- > > Key: HADOOP-10295 > URL: https://issues.apache.org/jira/browse/HADOOP-10295 > Project: Hadoop Common > Issue Type: Improvement > Components: tools/distcp > Affects Versions: 2.2.0 > Reporter: Jing Zhao > Assignee: Jing Zhao > Fix For: 3.0.0, 2.4.0 > > Attachments: HADOOP-10295.000.patch, HADOOP-10295.002.patch, > hadoop-10295.patch > > > Currently while doing distcp, users can use "-Ddfs.checksum.type" to specify > the checksum type in the target FS. This works fine if all the source files > are using the same checksum type. If files in the source cluster have mixed > types of checksum, users have to either use "-skipcrccheck" or have checksum > mismatching exception. Thus we may need to consider adding a new option to > distcp so that it can automatically identify the original checksum type of > each source file and use the same checksum type in the target FS. -- This message was sent by Atlassian JIRA (v6.1.5#6160)