Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/20282#discussion_r161884338
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala
---
@@ -210,15 +214,17 @@ class ContinuousExecution(
lastExecution.executedPlan // Force the lazy generation of execution
plan
}
- sparkSession.sparkContext.setLocalProperty(
+ sparkSessionForQuery.sparkContext.setLocalProperty(
ContinuousExecution.START_EPOCH_KEY, currentBatchId.toString)
- sparkSession.sparkContext.setLocalProperty(
- ContinuousExecution.RUN_ID_KEY, runId.toString)
+ val epochCoordinatorId = UUID.randomUUID.toString
--- End diff --
could you add `run_id + random_uuid` so that it's easy to tell which query
this epoch coordinator belongs to?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
