acvictor opened a new issue, #11001:
URL: https://github.com/apache/incubator-gluten/issues/11001
### Backend
VL (Velox)
### Bug description
[Expected behavior] and [actual behavior].
We have seen cases where Spark context shuts down due to some exception
while Gluten plugin is being initialized, Gluten plugin tries to register
shutdown hooks for cleaning up temporary directories. However, adding new
shutdown hooks once the shutdown sequence has started is not allowed leading to
an IllegalStateException.
### Gluten version
Gluten-1.3
### Spark version
Spark-3.5.x
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
```bash
java.lang.IllegalStateException: Shutdown hooks cannot be modified during
shutdown.
at
org.apache.spark.util.SparkShutdownHookManager.add(ShutdownHookManager.scala:195)
at
org.apache.spark.util.ShutdownHookManager$.addShutdownHook(ShutdownHookManager.scala:153)
at
org.apache.spark.util.SparkShutdownManagerUtil$.addHookForTempDirRemoval(SparkShutdownManagerUtil.scala:29)
at
org.apache.spark.util.SparkDirectoryUtil.$anonfun$ROOTS$1(SparkDirectoryUtil.scala:40)
at
scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at
scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at
scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:198)
at
org.apache.spark.util.SparkDirectoryUtil.<init>(SparkDirectoryUtil.scala:35)
at
org.apache.spark.util.SparkDirectoryUtil$.init(SparkDirectoryUtil.scala:79)
at
org.apache.spark.util.SparkDirectoryUtil$.init(SparkDirectoryUtil.scala:74)
at
org.apache.gluten.backendsapi.velox.VeloxListenerApi.onDriverStart(VeloxListenerApi.scala:95)
at
org.apache.gluten.backendsapi.SubstraitBackend.onDriverStart(SubstraitBackend.scala:27)
at
org.apache.gluten.backendsapi.SubstraitBackend.onDriverStart$(SubstraitBackend.scala:26)
at
org.apache.gluten.backendsapi.velox.VeloxBackend.onDriverStart(VeloxBackend.scala:62)
at
org.apache.gluten.GlutenDriverPlugin.$anonfun$init$1(GlutenPlugin.scala:72)
at
org.apache.gluten.GlutenDriverPlugin.$anonfun$init$1$adapted(GlutenPlugin.scala:72)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at org.apache.gluten.GlutenDriverPlugin.init(GlutenPlugin.scala:72)
at
org.apache.spark.internal.plugin.DriverPluginContainer.$anonfun$driverPlugins$1(PluginContainer.scala:53)
at
scala.collection.TraversableLike.$anonfun$flatMap$1(TraversableLike.scala:293)
at
scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:62)
at
scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:55)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:49)
at scala.collection.TraversableLike.flatMap(TraversableLike.scala:293)
at scala.collection.TraversableLike.flatMap$(TraversableLike.scala:290)
at scala.collection.AbstractTraversable.flatMap(Traversable.scala:108)
at
org.apache.spark.internal.plugin.DriverPluginContainer.<init>(PluginContainer.scala:46)
at
org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:210)
at
org.apache.spark.internal.plugin.PluginContainer$.apply(PluginContainer.scala:193)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
