GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/22674
[SPARK-25680][SQL] SQL execution listener shouldn't happen on execution
thread
## What changes were proposed in this pull request?
The SQL execution listener framework was created from scratch(see
https://github.com/apache/spark/pull/9078). It didn't leverage what we already
have in the spark listener framework, and one major problem is, the listener
runs on the spark execution thread, which means a bad listener can block
spark's query processing.
This PR re-implements the SQL execution listener framework. Now
`ExecutionListenerManager` is just a normal spark listener, which watches the
`SparkListenerSQLExecutionEnd` events and post events to the
SQL execution listeners.
## How was this patch tested?
existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark listener
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22674.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22674
----
commit 1701f3b9df78babef60d3f0cad12a332e3f24ec8
Author: Wenchen Fan <wenchen@...>
Date: 2018-10-08T17:18:44Z
SQL execution listener shouldn't happen on execution thread
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
