HeartSaVioR commented on a change in pull request #29372:
URL: https://github.com/apache/spark/pull/29372#discussion_r469655682
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala
##########
@@ -54,6 +55,8 @@ class QueryExecution(
val logical: LogicalPlan,
val tracker: QueryPlanningTracker = new QueryPlanningTracker) extends
Logging {
+ val id: Long = QueryExecution.nextExecutionId
Review comment:
We have a listener which parses the logical & physical plan from the
query execution and reflects the changes to Apache Atlas. While it works well
in general, there's a pain point on the listener due to the fact Spark can
callback the listener multiple times on the same query execution, whereas the
instance of query execution doesn't have the unique ID to distinct. That's
already I wrote in PR description as well as JIRA description.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
