This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 1e17a5b [SPARK-32920][FOLLOW-UP][CORE] Shutdown
shuffleMergeFinalizeScheduler when DAGScheduler stop
1e17a5b is described below
commit 1e17a5bc1960989507900eb270ddfaef57455700
Author: yi.wu <[email protected]>
AuthorDate: Sat Jul 24 17:40:47 2021 -0700
[SPARK-32920][FOLLOW-UP][CORE] Shutdown shuffleMergeFinalizeScheduler when
DAGScheduler stop
### What changes were proposed in this pull request?
Call `shuffleMergeFinalizeScheduler.shutdownNow()` in `DAGScheduler.stop()`.
### Why are the changes needed?
Avoid the thread leak.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass existing tests.
Closes #33495 from Ngone51/SPARK-32920-followup.
Authored-by: yi.wu <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 21450b325457573265b2bf66e3ef7dd27453978e)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala | 1 +
1 file changed, 1 insertion(+)
diff --git a/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
b/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
index 1f37638..9709ec1 100644
--- a/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
+++ b/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
@@ -2512,6 +2512,7 @@ private[spark] class DAGScheduler(
def stop(): Unit = {
messageScheduler.shutdownNow()
+ shuffleMergeFinalizeScheduler.shutdownNow()
eventProcessLoop.stop()
taskScheduler.stop()
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
