Github user andrewor14 commented on the pull request:
https://github.com/apache/spark/pull/1940#issuecomment-55342956
@xuzhongxing I followed the conversation on the JIRA and it looks like we
still don't have a good idea of why Spark driver is hanging. Although we have a
fix that makes the problem go away, the root cause is probably deeper, and the
behavior you observed on fine-grained mode is just a symptom.
My guess is that when an `UnsupportedOperationException` is thrown when we
try to `killTask`, we never end up posting the job end event to the listeners.
This may be a behavior introduced in #1219. @kayousterhout Can you comment on
this? Any thoughts on why `DAGScheduler` hangs if we don't post a job end event?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
