dongjoon-hyun commented on code in PR #38441:
URL: https://github.com/apache/spark/pull/38441#discussion_r1024243192
##########
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala:
##########
@@ -2193,11 +2193,13 @@ private[spark] class DAGScheduler(
* Return true when:
* 1. Waiting for decommission start
* 2. Under decommission process
- * Return false when:
- * 1. Stopped or terminated after finishing decommission
- * 2. Under decommission process, then removed by driver with other reasons
+ * 3. Stopped or terminated after finishing decommission
+ * 4. Under decommission process, then removed by driver with other reasons
+ * Return false in case 3 and 4 when removed executors info are not retained.
+ * The max size of removed executors is controlled by
+ * spark.scheduler.maxRetainedRemovedExecutors
*/
- private[scheduler] def isExecutorDecommissioningOrDecommissioned(
+ private def isExecutorDecommissioningOrDecommissioned(
Review Comment:
Shall we revert this change because this PR doesn't use
`isExecutorDecommissioningOrDecommissioned`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
