Ngone51 commented on a change in pull request #31156:
URL: https://github.com/apache/spark/pull/31156#discussion_r556399043
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/util/QueryExecutionListener.scala
##########
@@ -124,13 +131,27 @@ class ExecutionListenerManager private[sql](session:
SparkSession, loadExtension
}
}
-private[sql] class ExecutionListenerBus(session: SparkSession)
Review comment:
IIUC, `ExecutionListenerBus` is created per SparkSession intentionally.
With your change, it seems to break the assumption of "Only catch SQL execution
with a name, and triggered by the same spark session that this listener manager
belongs."
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala
##########
@@ -91,14 +91,25 @@ object SQLExecution {
var ex: Option[Throwable] = None
val startTime = System.nanoTime()
try {
- sc.listenerBus.post(SparkListenerSQLExecutionStart(
+
+ // queryExecution.executedPlan may throw an exception and made the
send event message
+ // failed. we should send SparkListenerSQLExecutionStart event to
listener bus when we
+ // catch an exception.
+ val sparkPlanInfo = try {
Review comment:
This seems to fix a separate issue?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
