xiaoxuandev commented on PR #12868: URL: https://github.com/apache/iceberg/pull/12868#issuecomment-3304992198
Hi @danielcweeks, thanks for the review! Currently, the mapping is already Table -> S3FileIO -> S3Client -> HttpClient. (1 to 1) The problem is that multiple HttpClient instances can still end up sharing the same underlying connection pool. For Apache HttpClient, that shared pool [gets closed when close() is called](https://github.com/aws/aws-sdk-java-v2/blob/cc6f80e5b8cfd84f23c2171e53d773932a895089/http-clients/apache-client/src/main/java/software/amazon/awssdk/http/apache/ApacheHttpClient.java#L254), which is where this issue comes from. I thought sharing pool is a common expected behavior for ApacheHttpClient, but maybe we are config it wrong. I will take a closer look later. I believe the same applies to the JVM-managed pool used by UrlConnectionHttpClient, but in that case it [doesn’t get closed when close() is called](https://github.com/aws/aws-sdk-java-v2/blob/master/http-clients/url-connection-client/src/main/java/software/amazon/awssdk/http/urlconnection/UrlConnectionHttpClient.java#L144) . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
