[
https://issues.apache.org/jira/browse/HADOOP-17932?focusedWorklogId=666163&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666163
]
ASF GitHub Bot logged work on HADOOP-17932:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 18/Oct/21 07:58
Start Date: 18/Oct/21 07:58
Worklog Time Spent: 10m
Work Description: aajisaka commented on a change in pull request #3519:
URL: https://github.com/apache/hadoop/pull/3519#discussion_r730659799
##########
File path:
hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/RetriableFileCopyCommand.java
##########
@@ -154,7 +154,7 @@ private long doCopy(CopyListingFileStatus source, Path
target,
if (!source.isSplit()) {
DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
sourcePath, sourceChecksum, targetFS,
- targetPath, skipCrc, source.getLen());
+ targetPath, skipCrc, bytesRead);
Review comment:
The test failures are related to the change. Probably we need to add
offset.
```suggestion
targetPath, skipCrc, offset + bytesRead);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 666163)
Time Spent: 40m (was: 0.5h)
> distcp file length comparison have no effect
> --------------------------------------------
>
> Key: HADOOP-17932
> URL: https://issues.apache.org/jira/browse/HADOOP-17932
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, tools, tools/distcp
> Affects Versions: 3.3.1
> Reporter: yinan zhan
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> the params for compareFileLengthsAndChecksums in RetriableFileCopyCommand
> have no effect
> current is
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, source.getLen());{code}
> {code:java}
> public static void compareFileLengthsAndChecksums(long srcLen,
> FileSystem sourceFS, Path source, FileChecksum sourceChecksum,
> FileSystem targetFS, Path target, boolean skipCrc,
> long targetLen) throws IOException {
> if (srcLen != targetLen) {
> throw new IOException(
> DistCpConstants.LENGTH_MISMATCH_ERROR_MSG + source + " (" + srcLen
> + ") and target:" + target + " (" + targetLen + ")");
> }
> {code}
> so, compare source.getLen() with source.getLen()...
> It should be like below in history view
> {code:java}
> DistCpUtils.compareFileLengthsAndChecksums(source.getLen(), sourceFS,
> sourcePath, sourceChecksum, targetFS,
> targetPath, skipCrc, bytesRead);
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
