Ngone51 commented on PR #43954: URL: https://github.com/apache/spark/pull/43954#issuecomment-1847130056
To fix the tests, I have to move (https://github.com/apache/spark/pull/43954/commits/fe70ba95a67800b798443b8fa873d2b24efa2067) the "abort stage" call back into `cancelTasks()` with the control flag rather than call "abort stage" after `cancelTasks()`. After the move, the timing of "abort stage" call strictly follows the original behavior. The probelm of calling "abort stage" after `cancelTasks()` is, e.g., test `DAGSchedulerSuite` has its own implementation of `TaskScheduler` which overrides the `cancelTasks()` with no "abort stage". But our change has extracted "abort stage" out side of `cancelTasks()`. Thus, the intention to not call "abort stage" in `cancelTask()` is broken and so test fails. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
