yuqi1129 opened a new pull request, #11431: URL: https://github.com/apache/gravitino/pull/11431
### What changes were proposed in this pull request? When `IsolatedClassLoader` resolves Hadoop/AWS/Azure classes via parent delegation (e.g. `hadoop-common.jar` or `gravitino-aws-bundle.jar` appears on the AppClassLoader's classpath), `Class.forName(..., targetClassLoader)` returns the parent-loaded class. The static fields on that class (`MutableQuantiles.scheduler`, `FileSystem.CACHE`, `AwsSdkMetrics` MBean, `AbfsClientThrottlingIntercept` timers) are JVM-global and shared across every catalog. Calling `scheduler.shutdownNow()` on the global instance permanently terminates Hadoop metrics scheduling, causing `RejectedExecutionException` on any subsequent catalog that creates a `MutableQuantiles` instance (e.g. S3A FileSystem instrumentation). This PR adds `isOwnedByClassLoader(clazz, targetClassLoader)` guards before any static-state mutation in `ClassLoaderResourceCleanerUtils`. If the class was delegated to a parent ClassLoader, cleanup is skipped; if it belongs to the catalog's own ClassLoader (the normal isolated case), cleanup proceeds as before. The same guard is applied to: - `closeStatsDataClearerInFileSystem`: `FileSystem.closeAll()`, scheduler shutdown, `STATS_DATA_CLEANER` thread interrupt. - `closeResourceInAWS`: `AwsSdkMetrics` MBean unregistration. - `closeResourceInGCP`: shaded `LogFactory` release. - `closeResourceInAzure`: `AbfsClientThrottlingIntercept` timer cancellation. Additionally, `closeClassLoaderResource` calls are added to `HiveCatalogOperations` and `FilesetCatalogOperations`, which use Hadoop FileSystem extensively but had no classloader cleanup on close, risking thread and memory leaks. ### Why are the changes needed? Fix: #11301 After an Iceberg or Paimon catalog is evicted from the cache and re-created, any subsequent S3A FileSystem initialization fails with: ``` RejectedExecutionException: Task ... rejected from ScheduledThreadPoolExecutor@...[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 23664] ``` because `closeStatsDataClearerInFileSystem` shut down the JVM-global `MutableQuantiles.scheduler` when Hadoop happened to be on the AppClassLoader's classpath. ### Does this PR introduce _any_ user-facing change? No functional behavior change for correctly isolated deployments. For deployments where Hadoop is on the server's main classpath, the broken scheduler-shutdown is now skipped, restoring correct behavior after catalog re-creation. ### How was this patch tested? - `./gradlew :catalogs:catalog-common:spotlessApply :catalogs:catalog-hive:spotlessApply :catalogs:catalog-fileset:spotlessApply` - `./gradlew :catalogs:catalog-hive:test :catalogs:catalog-fileset:test -PskipDockerTests=true` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
