[
https://issues.apache.org/jira/browse/HADOOP-17932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423838#comment-17423838
]
yinan zhan commented on HADOOP-17932:
-------------------------------------
[[email protected]]
I hope someone will review it. If it is designed like this, I hope to tell the
reason.
Because if the current situation is reasonable, it means that distcp does not
have any checks (because crc is not turned on by default, and the crc
implementation of each file system is different)
> distcp file length comparison have no effect
> --------------------------------------------
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, tools, tools/distcp
> Affects Versions: 3.3.1
> Reporter: yinan zhan
> Priority: Major
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
> FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
> FileSystem targetFS, Path target, boolean skipCrc,
> long targetLen) throws IOException {
> if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
> }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
