Li Xian created SPARK-36842:
-------------------------------
Summary: Stop task result getter properly on spark context stopping
Key: SPARK-36842
URL: https://issues.apache.org/jira/browse/SPARK-36842
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.3.0
Reporter: Li Xian
org.apache.spark.scheduler.TaskSchedulerImpl#stop doesn't handle exception
properly. If one component throws exceptions on stopping, the exception is
thrown and TaskSchedulerImpl.stop() will not be executed completely.
For example if backend.stop() fails, then taskResultGetter.stop() won't be
executed. The result is that after a couple of restart of the spark context,
there will be a lot of '
task-result-getter' threads retained.
!image-2021-09-24-18-50-57-072.png!
!image-2021-09-24-18-51-03-837.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
