ahmarsuhail commented on PR #7333:
URL: https://github.com/apache/hadoop/pull/7333#issuecomment-2616535949
@steveloughran before I do anymore work here, could you please validate
these changes. Here is what I want to achieve from this PR:
* Minimum required configuration
- Proxy is not configured
- Region configuration logic is highly simplified (no endpoint parsing)
- A lot of HTTP configurations we apply cannot be applied to CRT
* Add a new `getOrCreateCRTClient()`. We previously discussed
`getOrCreateAsyncClient(crtRequired)`. But I want to keep CRT client separate
from the Java async client.
I am initially aiming for this behaviour:
* When using analytic accelerator (AAL), use CRT as default. This will be
controlled by `ANALYTICS_ACCELERATOR_CRT_ENABLED`. If you want to use it
without CRT, just disable this.
* Transfer manager to continue to use Java async client.
So, CRT default on with AAL, can be disabled, in which case
`getOrCreateAsyncClient()` will get called. No changes to existing
functionality of transfer manager.
The other option is to have a `CRT_ENABLED` flag for all of S3A, which will
be default OFF. This will control if CRT is used in both AAL and the transfer
manager. So will mean AAL is used with Java async as default.
Our benchmarks have been with the CRT client, and so would really like CRT
to be the default client with AAL.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
