Juliusz Sompolski created SPARK-45206:
-----------------------------------------
Summary: Shutting down SparkConnectClient can have outstanding
ReleaseExecute
Key: SPARK-45206
URL: https://issues.apache.org/jira/browse/SPARK-45206
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 4.0.0
Reporter: Juliusz Sompolski
In Spark Connect scala client, there can be outstanding asynchronous
ReleaseExecute calls when the client is shutdown.
In ExecutePlanResponseReattachableIterator we (ab)use a grpc thread in
createRetryingReleaseExecuteResponseObserver to run it.
When we do {{{}SparkConnectClient.shutdown{}}}, which does
{{channel.shutdownNow()}}
it kills it before it finishes the ReleaseExecute. Maybe a more graceful
shutdown would work, or we should move to a more explicit threadpool that we
manage, like in python?
See discussion in https://github.com/apache/spark/pull/42929/files#r1329071826
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
