[
https://issues.apache.org/jira/browse/HADOOP-10295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13887651#comment-13887651
]
Hudson commented on HADOOP-10295:
---------------------------------
SUCCESS: Integrated in Hadoop-Yarn-trunk #467 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/467/])
HADOOP-10295. Allow distcp to automatically identify the checksum type of
source files and use it for the target. Contributed by Jing Zhao and Laurent
Goujon. (jing9:
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1563019)
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileChecksum.java
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/MD5MD5CRC32FileChecksum.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptionSwitch.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpOptions.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/OptionsParser.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyMapper.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/DistCpUtils.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/TestOptionsParser.java
*
/hadoop/common/trunk/hadoop-tools/hadoop-distcp/src/test/java/org/apache/hadoop/tools/mapred/TestCopyMapper.java
> Allow distcp to automatically identify the checksum type of source files and
> use it for the target
> --------------------------------------------------------------------------------------------------
>
> Key: HADOOP-10295
> URL: https://issues.apache.org/jira/browse/HADOOP-10295
> Project: Hadoop Common
> Issue Type: Improvement
> Components: tools/distcp
> Affects Versions: 2.2.0
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Fix For: 3.0.0, 2.4.0
>
> Attachments: HADOOP-10295.000.patch, HADOOP-10295.002.patch,
> hadoop-10295.patch
>
>
> Currently while doing distcp, users can use "-Ddfs.checksum.type" to specify
> the checksum type in the target FS. This works fine if all the source files
> are using the same checksum type. If files in the source cluster have mixed
> types of checksum, users have to either use "-skipcrccheck" or have checksum
> mismatching exception. Thus we may need to consider adding a new option to
> distcp so that it can automatically identify the original checksum type of
> each source file and use the same checksum type in the target FS.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
