xiaojunxiang created HDFS-17498:
-----------------------------------
Summary: Distcp the concat files error, because sourceFS's
chesksum is not equals to targetFS's chesksum.
Key: HDFS-17498
URL: https://issues.apache.org/jira/browse/HDFS-17498
Project: Hadoop HDFS
Issue Type: Improvement
Components: distcp
Affects Versions: 3.3.4
Reporter: xiaojunxiang
Attachments: image-2024-04-24-15-54-16-253.png,
image-2024-04-24-15-54-58-047.png, image-2024-04-24-15-55-25-519.png,
image-2024-04-24-15-55-48-752.png
When we use distcp, the sourceFS's checksum and targetFS's checksum are checked
for consistency after the file transfer is complete.
However, for some files produced by ClientProcotol's concat(RPC method) on the
source side, the Block Size is less than 128MB(such as sourceFS file
=10MB+10MB, targetFS file = 20MB), so the checksum of the source and
destination side will be inconsistent, So It waill cause distcp failed
!image-2024-04-24-15-54-16-253.png!
!image-2024-04-24-15-54-58-047.png!
!image-2024-04-24-15-55-25-519.png!
!image-2024-04-24-15-55-48-752.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
