[
https://issues.apache.org/jira/browse/HADOOP-19399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17921477#comment-17921477
]
ASF GitHub Bot commented on HADOOP-19399:
-----------------------------------------
ahmarsuhail commented on PR #7333:
URL: https://github.com/apache/hadoop/pull/7333#issuecomment-2616535949
@steveloughran before I do anymore work here, could you please validate
these changes. Here is what I want to achieve from this PR:
* Minimum required configuration
- Proxy is not configured
- Region configuration logic is highly simplified (no endpoint parsing)
- A lot of HTTP configurations we apply cannot be applied to CRT
* Add a new `getOrCreateCRTClient()`. We previously discussed
`getOrCreateAsyncClient(crtRequired)`. But I want to keep CRT client separate
from the Java async client.
I am initially aiming for this behaviour:
* When using analytic accelerator (AAL), use CRT as default. This will be
controlled by `ANALYTICS_ACCELERATOR_CRT_ENABLED`. If you want to use it
without CRT, just disable this.
* Transfer manager to continue to use Java async client.
So, CRT default on with AAL, can be disabled, in which case
`getOrCreateAsyncClient()` will get called. No changes to existing
functionality of transfer manager.
The other option is to have a `CRT_ENABLED` flag for all of S3A, which will
be default OFF. This will control if CRT is used in both AAL and the transfer
manager. So will mean AAL is used with Java async as default.
Our benchmarks have been with the CRT client, and so would really like CRT
to be the default client with AAL.
What do you think?
> S3A: Add support for CRT client
> -------------------------------
>
> Key: HADOOP-19399
> URL: https://issues.apache.org/jira/browse/HADOOP-19399
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.1
> Reporter: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.2
>
>
> * Allow ClientManager to initialise a CRT client
> * Should be optional, java async client to still be supported as default
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
