Rajesh Balamohan created HADOOP-13432:
-----------------------------------------
Summary: S3A: Consider using TransferManager.download for
copyToLocalFile
Key: HADOOP-13432
URL: https://issues.apache.org/jira/browse/HADOOP-13432
Project: Hadoop Common
Issue Type: Improvement
Components: fs/s3
Reporter: Rajesh Balamohan
Priority: Minor
Currently it relies on the default implementation of FileSystem. But it would
be good to explore using TransferManager.download() (Ref:
https://java.awsblog.com/post/Tx3Z7NO7C2TVLB/Parallelizing-Large-Downloads-for-Optimal-Speed
for recent aws-sdk-java). When aws-sdk version is bumped it,it would
automatically get the benefit of parallel download sas well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
