Github user squito commented on the issue:
    > Things I'm concerned about is that does there exists a situation like 'a 
task gets killed after it gets a FetchFailure, but re-run later and gets a 
FetchFailure too without TaskKilledException' (or this fix against speculative 
tasks only).
    I don't think its worth trying to be fancy here -- in almost all 
situations, we don't care about the fetch failure handling when the task is 
killed.  Even if this task is not speculative, it could be that its killed 
because *another* speculative task finished, and so this one gets aborted.
    Suppose there is a real fetch failure, and you just happen to kill the task 
*just* after that.  Since the task was killed, that means you don't really care 
about the input shuffle data at the moment in any case.  You *might* run 
another job later on which tries to read the same shuffle data, and then it'll 
have to rediscover the missing shuffle data.  But, thats about it.  oh well.


To unsubscribe, e-mail:
For additional commands, e-mail:

Reply via email to