acvictor commented on PR #11002: URL: https://github.com/apache/incubator-gluten/pull/11002#issuecomment-3479282458
> As Spark has been shutdown already, creating the local directory is successfully but register the hook is failed, this will cause Gluten unavailable. Could explain the meaning to catch the IllegalStateException? When Spark's shutdown is in progress (SparkShutdownHookManager.shuttingDown = true https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala#L212), Gluten initialization still succeeds because directory creation (line 38) happens regardless of hook registration. The hook is only for cleanup - its failure doesn't affect Gluten's functionality. If we're in shutdown, Gluten either already executed or won't be used. Catching the exception allows graceful shutdown instead of crashing. Gluten should remain fully available during normal execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
