[
https://issues.apache.org/jira/browse/HADOOP-18723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17718766#comment-17718766
]
ASF GitHub Bot commented on HADOOP-18723:
-----------------------------------------
symious commented on PR #5603:
URL: https://github.com/apache/hadoop/pull/5603#issuecomment-1532347000
The issue we met was not "source file needs copying again", but source file
uploaded with a different checksum type.
Because it's distcp, so the temp file was removed after the exception. We
need to first skip the crc check, then run "hadoop fs -checksum" on the source
file and target file to find out the root cause of this one.
The initial idea of this addition of log was to help others with the same
issue to save to process of redundant works and just check the log for mismatch
reasons.
@steveloughran @ayushtkn
> Add detail logs if distcp checksum mismatch
> -------------------------------------------
>
> Key: HADOOP-18723
> URL: https://issues.apache.org/jira/browse/HADOOP-18723
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Janus Chow
> Assignee: Janus Chow
> Priority: Major
> Labels: pull-request-available
>
> We encountered some errors of mismatch checksum during Distcp jobs. It took
> us some time to figure out that checksum type is different.
> Adding error logs shall help us to figure out such problems faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
