[
https://issues.apache.org/jira/browse/HADOOP-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liuxiaolong updated HADOOP-17256:
---------------------------------
Attachment: image-2020-09-10-17-33-50-505.png
> DistCp -update option will be invalid when distcp files from hdfs to S3
> -----------------------------------------------------------------------
>
> Key: HADOOP-17256
> URL: https://issues.apache.org/jira/browse/HADOOP-17256
> Project: Hadoop Common
> Issue Type: Bug
> Components: tools/distcp
> Reporter: liuxiaolong
> Priority: Major
> Attachments: image-2020-09-10-17-25-46-354.png,
> image-2020-09-10-17-33-50-505.png
>
>
> We use distcp with -update option to copy a dir from hdfs to S3. When we run
> distcp job once more, it will overwrite S3 dir directly, rather than skip the
> same files.
>
> Test Case:
> 1. Run twice distcp cmd, the modify time of S3 files will be modified
> hadoop distcp -update /testA/ s3a://tiered-storage-bigdata-1251625956/testA/
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
