Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/21898
> Also, if there shouldn't exist two active attempts at the same time for a
barrier stage, maybe we should store attemptId as a state variable. Basically,
if we see a new attempt ID, we should abort the old attempts.
Actually I'm not sure whether we can guarantee that, since kill tasks may
take some time, it's always possible that a new stage attempt is launched, and
then a zombie task that haven't been killed send a barrier sync message.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
