Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19145
@klion26 , this is not a problem related to Spark Streaming and Structured
Streaming. For any Spark application it will run into this problem. This is
basically a YARN problem and looks hard to address in Spark. You might just had
this point fix worked, but what if other behavior happened during RM/NM
restart? May be this is a one case of inconsistency during RM/NM restart, I'm
not sure how to well fix it. Can you please check MR/TEZ if they have a proper
fix about this problem, I assume they may also suffer this problem.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
