[
https://issues.apache.org/jira/browse/HDFS-13764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuexin Zhang resolved HDFS-13764.
---------------------------------
Resolution: Won't Fix
it turns out -skipcrccheck needs to be used along with -update:
hadoop distcp -skipcrccheck /tmp1 /tmp2
18/07/25 23:37:06 ERROR tools.DistCp: Invalid arguments:
java.lang.IllegalArgumentException: Skip CRC is valid only with update options
at org.apache.hadoop.tools.DistCpOptions.validate(DistCpOptions.java:691)
> [DOC] update flag is not necessary to avoid verifying checksums
> ---------------------------------------------------------------
>
> Key: HDFS-13764
> URL: https://issues.apache.org/jira/browse/HDFS-13764
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
> Affects Versions: 2.7.0, 2.7.3
> Reporter: Yuexin Zhang
> Priority: Major
> Attachments: HDFS-13764_DOC1.patch
>
>
> We mentioned to use "-update" option to avoid checksum in the following doc:
> [https://hadoop.apache.org/docs/r2.7.3/hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html#Copying_between_encrypted_and_unencrypted_locations]
> {code:java}
> Copying between encrypted and unencrypted locations
> By default, distcp compares checksums provided by the filesystem to verify
> that the data was successfully copied to the destination. When copying
> between an unencrypted and encrypted location, the filesystem checksums will
> not match since the underlying block data is different. In this case, specify
> the -skipcrccheck and -update distcp flags to avoid verifying checksums.
> {code}
>
> But actually, "-update" option is not necessary, only "-skipcrccheck" is
> needed. Can we change it to:
>
> {code:java}
> Copying between encrypted and unencrypted locations
> By default, distcp compares checksums provided by the filesystem to verify
> that the data was successfully copied to the destination. When copying
> between an unencrypted and encrypted location, the filesystem checksums will
> not match since the underlying block data is different. In this case, specify
> the -skipcrccheck flags to avoid verifying checksums.
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
