[GitHub] [hudi] KnightChess commented on a diff in pull request #8233: [HUDI-5956] Simple repair spark sql dag ui display problem

[via GitHub]

KnightChess commented on code in PR #8233:
URL: https://github.com/apache/hudi/pull/8233#discussion_r1141330869


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieSparkSqlWriter.scala:
##########
@@ -112,6 +113,27 @@ object HoodieSparkSqlWriter {
             asyncClusteringTriggerFn: Option[SparkRDDWriteClient[_] => Unit] = 
Option.empty,
             extraPreCommitFn: Option[BiConsumer[HoodieTableMetaClient, 
HoodieCommitMetadata]] = Option.empty):
   (Boolean, HOption[String], HOption[String], HOption[String], 
SparkRDDWriteClient[_], HoodieTableConfig) = {
+    //TODO reuse DataWritingCommand sparkPlan, reduce the number of sql list 
in SPARK UI SQL tag, rendering raw DAG
+    val executionId = getExecutionId(sqlContext.sparkContext, 
df.queryExecution)

Review Comment:
   new qe is create by DataSet.ofRows, is diff from raw sparkPlan, can not 
udpate the new dag ui.
   before:
   <img width="608" alt="image" 
src="https://user-images.githubusercontent.com/20125927/226170429-f12efb73-31fb-4c4a-8157-27515442580f.png";>
   
   after:
   <img width="673" alt="image" 
src="https://user-images.githubusercontent.com/20125927/226170462-d024bd5f-5aec-46d5-a7a4-47c019ab42ea.png";>
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org