[
https://issues.apache.org/jira/browse/HDFS-13940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HDFS-13940:
----------------------------------
Summary: Implement Multipart-aware distcp (was: Implement Multipart-aware
discp-equivalent)
> Implement Multipart-aware distcp
> --------------------------------
>
> Key: HDFS-13940
> URL: https://issues.apache.org/jira/browse/HDFS-13940
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Major
>
> the per block upload would permit a high-performance distcp
> * within the same FS
> * when writing to object stores with multipart uploads
> Implement this, if initially as a proof of concept to validate the
> completeness of the API
> One thing to consider is throttling upload bandwidth better: there's the
> potential here to overload the long-haul links more here, even when uploading
> a few files (as the peark bandwidth which a single file being copied ma use
> isO(blocks))
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
