Github user aray commented on the issue:
https://github.com/apache/spark/pull/18306
@zsxwing @marmbrus @rxin This is ready for review. I have changed the
approach so that queries from all SparkSession's are stopped. I was not able to
use a SparkListener as @zsxwing suggested or even make a new event type (as I
suggested) as all of that is handled asynchronously and thus queries are not
stopped before the SparkContext is stopped. This new approach adds a way to
register stop hooks to the SparkContext that are called in a blocking fashion
on stop. Does this work for everyone?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
