Github user squito commented on the issue:

    https://github.com/apache/spark/pull/20987
  
    > Things I'm concerned about is that does there exists a situation like 'a 
task gets killed after it gets a FetchFailure, but re-run later and gets a 
FetchFailure too without TaskKilledException' (or this fix against speculative 
tasks only).
    
    I don't think its worth trying to be fancy here -- in almost all 
situations, we don't care about the fetch failure handling when the task is 
killed.  Even if this task is not speculative, it could be that its killed 
because *another* speculative task finished, and so this one gets aborted.
    
    Suppose there is a real fetch failure, and you just happen to kill the task 
*just* after that.  Since the task was killed, that means you don't really care 
about the input shuffle data at the moment in any case.  You *might* run 
another job later on which tries to read the same shuffle data, and then it'll 
have to rediscover the missing shuffle data.  But, thats about it.  oh well.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to