>From Rithwik Koul <[email protected]>: Rithwik Koul has uploaded a new patch set (#2). ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21235?usp=email )
Change subject: MB-71767: Honor HTTP idle/lifetime config in S3 and GCS cloud clients ...................................................................... MB-71767: Honor HTTP idle/lifetime config in S3 and GCS cloud clients The cluster-config knobs cloudRequestsHttpConnectionMaxIdleSeconds and cloudRequestsHttpConnectionMaxLifetimeSeconds were declared in CloudProperties and exposed via ICloudProperties but never read by S3ClientConfig.of or GCSClientConfig.of, so S3 and GCS cloud clients silently ignored them - leaving operators unable to force periodic connection refresh against S3-compatible LBs/appliances that drop pooled sockets by idle or age. The Azure client already honors them via Reactor Netty's ConnectionProvider.maxIdleTime/maxLifeTime; this mirrors that precedent for the two remaining cloud clients. S3 sync (S3CloudClient.buildClient): apply ApacheHttpClient.Builder.connectionMaxIdleTime(Duration) and connectionTimeToLive(Duration) when the respective knob is > 0. S3 async (S3ParallelDownloader.createS3AsyncClient): apply the same two methods to NettyNioAsyncHttpClient.Builder. The CRT parallel-downloader path is not similarly configurable and is left untouched. GCS (GCSCloudClient.buildClient): google-cloud-storage's StorageOptions does not expose connection pool eviction or TTL directly, so plug a custom ApacheHttpTransport via HttpTransportOptions.setHttpTransport- Factory when either knob is set, with evictIdleConnections + setConn- ectionTimeToLive on its Apache HttpClientBuilder. The google-http- client-apache-v2 artifact is already transitive on the classpath, so no pom edits. S3ClientConfig and GCSClientConfig get maxIdleSeconds / maxLifetime- Seconds fields, ctor plumbing, of(ICloudProperties) wiring, and getters. Coordinated with cbas-core MB-71767 which extends AzureCloudSharedIT's existing config-change pattern into S3MockCloudSharedIT and MinIOCloudSharedIT (S3RoundRobinCloudSharedIT inherits transitively), and adds a printEvictionEvidence step that snapshots S3 source ports before/after an idle window so the eviction is visible in the IT log. S3ConnectionLifecycleTest and GCSConnectionLifecycleTest are new unit tests verifying the eviction + TTL wiring end-to-end against an in- process mock (no Docker required). Ext-ref: MB-71767 Change-Id: I52bfe3e855b654b48f4294a9cd52e59406555aeb --- M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3ClientConfig.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3CloudClient.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/aws/s3/S3ParallelDownloader.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSClientConfig.java M asterixdb/asterix-cloud/src/main/java/org/apache/asterix/cloud/clients/google/gcs/GCSCloudClient.java A asterixdb/asterix-cloud/src/test/java/org/apache/asterix/cloud/clients/aws/s3/S3ConnectionLifecycleTest.java A asterixdb/asterix-cloud/src/test/java/org/apache/asterix/cloud/clients/google/gcs/GCSConnectionLifecycleTest.java 7 files changed, 705 insertions(+), 8 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/35/21235/2 -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21235?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: newpatchset Gerrit-Project: asterixdb Gerrit-Branch: lumina Gerrit-Change-Id: I52bfe3e855b654b48f4294a9cd52e59406555aeb Gerrit-Change-Number: 21235 Gerrit-PatchSet: 2 Gerrit-Owner: Rithwik Koul <[email protected]>
