nssalian opened a new pull request, #18919:
URL: https://github.com/apache/iceberg/pull/18919

   This updates `S3FileIO` to reuse the client factory created in 
`initialize()` for the root prefix.
   
   Since [#12799](https://github.com/apache/iceberg/pull/12799), `initialize()` 
creates the client factory only to read its credential, and each 
`PrefixedS3Client` creates two more factories, one for `s3` and one for 
`s3Async`. The prefix map is transient and is rebuilt where it is first used, 
so after deserialization every executor creates new factories. Before #12799, 
the factory created by `initialize()` was serialized with the `S3FileIO`. 
Factories that depend on where the `S3FileIO` was created, like configuration 
resolved on the driver, now fail or redo their work on executors.
   
   This passes the factory from `initialize()` to the root `PrefixedS3Client` 
and creates one factory per `PrefixedS3Client` for both clients. Storage 
credential prefixes still create their own factory from their own properties, 
and suppliers passed to the constructor are still used. 
[#18757](https://github.com/apache/iceberg/pull/18757) also changes the 
`PrefixedS3Client` constructor.
   
   Test plan:
   - In `TestS3FileIOClientFactory`:
       - Root clients use the factory from `initialize()`, for both 
`S3FileIOAwsClientFactory` and `AwsClientFactory`
       - Root clients of a deserialized `S3FileIO` use the serialized factory, 
with Java and Kryo serialization
       - A storage credential prefix creates one factory for both clients
       - Preload creates the root client from the factory from `initialize()`
       - All of these fail without this change
   
   ---
   **AI Disclosure**
   - Model: Claude Opus 5.5
   - Platform/Tool: Claude Code
   - Human Oversight: fully reviewed with tests run locally
   - Prompt Summary: Reuse the S3FileIO client factory for the root prefix and 
create one factory per PrefixedS3Client, with tests; the code and tests are 
generated.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to