binarybrain-009 commented on issue #17485: URL: https://github.com/apache/iceberg/issues/17485#issuecomment-6283130645
I'd like to work on this. The cache in `AnalyticsAcceleratorUtil` is bounded at 100 entries and its removal listener closes evicted factories, so the 101st `S3FileIO` closes a factory that a live `S3FileIO` is still reading through. Each entry already has an owner that removes it: `PrefixedS3Client.close()` calls `cleanupCache`. My plan is to drop the size bound so a factory is only closed when its `S3FileIO` is closed, and add a test that opening more than 100 FileIOs doesn't close a factory still in use. The trade-off is that factories of FileIOs that are never closed stay cached, but those FileIOs already keep their S3 clients open. Does that sound right? cc @SanjayMarreddi @nastra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
