[
https://issues.apache.org/jira/browse/HADOOP-19368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ahmar Suhail updated HADOOP-19368:
----------------------------------
Description:
Currently we initialise a new CRT client when analytics-accelerator is enabled.
This is done in S3AFileSystem, and is also not configured with S3A configs as
used by other clients.
We should move this creation to S3AStoreImpl, and also use suitable hadoop
configs where it makes sense to do so.
Also to use the S3AsynClient, we have temporarily done
this.isMultipartUploadEnabled = false, as without this you cannot make ranged
GETs. Believe this issue may have been resolved in the later versions of the
SDK. We will need to revisit this code.
was:
Currently we initialise a new CRT client when analytics-accelerator is enabled.
This is done in S3AFileSystem, and is also not configured with S3A configs as
used by other clients.
We should move this creation to S3AStoreImpl, and also use suitable hadoop
configs where it makes sense to do so.
> S3A Analytics-Accelerator: Update SDK client creation code
> -----------------------------------------------------------
>
> Key: HADOOP-19368
> URL: https://issues.apache.org/jira/browse/HADOOP-19368
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Ahmar Suhail
> Priority: Major
>
> Currently we initialise a new CRT client when analytics-accelerator is
> enabled. This is done in S3AFileSystem, and is also not configured with S3A
> configs as used by other clients.
> We should move this creation to S3AStoreImpl, and also use suitable hadoop
> configs where it makes sense to do so.
> Also to use the S3AsynClient, we have temporarily done
> this.isMultipartUploadEnabled = false, as without this you cannot make ranged
> GETs. Believe this issue may have been resolved in the later versions of the
> SDK. We will need to revisit this code.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
