thswlsqls opened a new issue, #17095: URL: https://github.com/apache/iceberg/issues/17095
**Apache Iceberg version** main @ 3038fde68 **Query engine** N/A (catalog-level, engine-agnostic) **Please describe the bug** `SnowflakeCatalog.newTableOps(TableIdentifier)` creates a new `FileIO` on every table load and unconditionally registers it with `closeableGroup` (`snowflake/src/main/java/org/apache/iceberg/snowflake/SnowflakeCatalog.java` line 255-256, before this fix). `CloseableGroup` backs its closeable list with a `Deque` that only grows until the whole group is closed, so a long-lived `SnowflakeCatalog` instance that repeatedly loads or refreshes tables (a REST catalog server, a long-running Spark session) accumulates one `FileIO` per load, holding native resources (S3/Azure/GCS clients, connection pools) until the catalog itself is closed. `GlueCatalog.newTableOps(TableIdentifier)` (`aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java`) has the same "new FileIO per table load" design but already solves this with core's `FileIOTracker` (`core/src/main/java/org/apache/iceberg/io/FileIOTracker.java`), which tracks each `FileIO` against its `TableOperations` with a weak-keyed cache and closes it once the `TableOperations` is no longer reachable. **Steps to reproduce** 1. Create one `SnowflakeCatalog` instance. 2. Call `loadTable()` (or `newTableOps()`) repeatedly, including reloading the same table. 3. Expected: unreferenced `FileIO` instances get reclaimed independently of catalog lifetime, matching `GlueCatalog`. Actual: every call adds a `FileIO` to `closeableGroup` that is never released until `catalog.close()`. **Additional context** N/A, covered above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
