dejii commented on PR #38149: URL: https://github.com/apache/beam/pull/38149#issuecomment-4256174866
I think we can go with (1). No real downside to not closing for a shared static catalog. Taking Rest catalog as an example, the resources are: - [One HTTP client](https://github.com/apache/iceberg/blob/304c00f3c17a67a7eb2306cf018fe8ba5647e4b1/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L240) - [One FileIO](https://github.com/apache/iceberg/blob/304c00f3c17a67a7eb2306cf018fe8ba5647e4b1/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L259) - [One auth manager](https://github.com/apache/iceberg/blob/304c00f3c17a67a7eb2306cf018fe8ba5647e4b1/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L206) (handles its own token refresh) - [Per-table FileIOs](https://github.com/apache/iceberg/blob/304c00f3c17a67a7eb2306cf018fe8ba5647e4b1/core/src/main/java/org/apache/iceberg/rest/RESTSessionCatalog.java#L260) cleaned by `FileIOTracker` weak-key GC All are designed to be long-lived. The only "leak" is that these resources aren't explicitly released when the pipeline finishes, but the JVM/container dies shortly after anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
