924060929 opened a new pull request, #66913: URL: https://github.com/apache/doris/pull/66913
## Problem The Iceberg table meta cache holds raw Iceberg tables. When an entry is refreshed, invalidated, or evicted, the old table's per-table FileIO (e.g. S3FileIO and its AWS clients/thread pools) is not deterministically closed. It is only reclaimed when GC collects the old table, which can leave native/off-heap resources resident for a long time when old-gen GC is infrequent. This can make FE RSS/thread count grow until cgroup OOM. ## Change - Add optional removal-listener support to the external meta cache framework. - Register a removal listener on the Iceberg table cache entry. - Mark Iceberg table cache values as owning their FileIO only when the catalog creates per-table FileIO (Glue/DLF, and REST when the table does not share the catalog-level FileIO). - Close the owned FileIO when the cache entry is removed. ## Test - Added unit test verifying that invalidating an Iceberg table cache entry closes a table-owned FileIO. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
