tgravescs commented on a change in pull request #27050: [SPARK-30388][Core]
Mark running map stages of finished job as finished, and cancel running tasks
URL: https://github.com/apache/spark/pull/27050#discussion_r366596341
##########
File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
##########
@@ -608,8 +608,9 @@ private[spark] class DAGScheduler(
*
* @param job The job whose state to cleanup.
*/
- private def cleanupStateForJobAndIndependentStages(job: ActiveJob): Unit = {
+ private def cleanupStateForJobAndIndependentStages(job: ActiveJob):
HashSet[Stage] = {
val registeredStages = jobIdToStageIds.get(job.jobId)
+ val removedStages = new HashSet[Stage]
Review comment:
I think you are only returning removed stages that were running so variables
should be named removedRunningStages
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
