linzebing commented on issue #27223: [SPARK-30511][SPARK-28403][CORE] Don't treat failed/killed speculative tasks as pending in Spark scheduler URL: https://github.com/apache/spark/pull/27223#issuecomment-578931499 @tgravescs I have addressed your commented, and improved the test case to include cases where speculative/original tasks can fail. Regarding the JIRA you'd like me to create, are you referring to improve the logic of speculation task tracking in `TaskSetManager.scala`? I don't fully understand the details you want me to fill in. Currently I can only think of it's possible for us to launch two speculative tasks for the same task index, which adds complexity to speculative task tracking in `ExecutorAllocationManager`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
