cfmcgrady commented on a change in pull request #31156:
URL: https://github.com/apache/spark/pull/31156#discussion_r557093329
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala
##########
@@ -91,14 +91,25 @@ object SQLExecution {
var ex: Option[Throwable] = None
val startTime = System.nanoTime()
try {
- sc.listenerBus.post(SparkListenerSQLExecutionStart(
+
+ // queryExecution.executedPlan may throw an exception and made the
send event message
+ // failed. we should send SparkListenerSQLExecutionStart event to
listener bus when we
+ // catch an exception.
+ val sparkPlanInfo = try {
+ SparkPlanInfo.fromSparkPlan(queryExecution.executedPlan)
+ } catch {
+ case e: Exception =>
+ null
+ }
+
+ sparkSession.listenerManager.post(SparkListenerSQLExecutionStart(
Review comment:
We send `SparkListenerSQLExecutionStart` event through
`sparkSession.listenerManager.post`, this method will only send SQL executions
event to the listener manager created by the SQL launched session, not all of
the listener managers.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
