noslowerdna commented on a change in pull request #645: HADOOP-16132 Support
multipart download in S3AFileSystem
URL: https://github.com/apache/hadoop/pull/645#discussion_r269698213
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -402,6 +416,66 @@ public void initialize(URI name, Configuration
originalConf)
long authDirTtl = conf.getLong(METADATASTORE_AUTHORITATIVE_DIR_TTL,
DEFAULT_METADATASTORE_AUTHORITATIVE_DIR_TTL);
ttlTimeProvider = new S3Guard.TtlTimeProvider(authDirTtl);
+
+ S3Downloader rawS3Downloader = new S3Downloader() {
+ @Override
+ public AbortableInputStream download(
+ String requestBucket, String key, long rangeStart, long rangeEnd,
+ ChangeTracker changeTracker, String operation) throws IOException {
+ String serverSideEncryptionKey = getServerSideEncryptionKey(
+ requestBucket, getConf());
+ GetObjectRequest request = new GetObjectRequest(requestBucket, key);
+ if (S3AEncryptionMethods.SSE_C
+ .equals(getServerSideEncryptionAlgorithm()) &&
+ StringUtils.isNotBlank(serverSideEncryptionKey)) {
+ request.setSSECustomerKey(
+ new SSECustomerKey(serverSideEncryptionKey));
+ }
+
+ changeTracker.maybeApplyConstraint(request);
+
+ String text = String.format("%s %s at %d",
+ operation, uri, rangeStart);
+
+ S3Object object = Invoker.once(text, "s3a://" + key + "/" + bucket,
Review comment:
Do the `key` and `bucket` need to be switched here? Also, use
`requestBucket` instead of `bucket`?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
