Github user LantaoJin commented on a diff in the pull request:
https://github.com/apache/spark/pull/20803#discussion_r176102381
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -166,20 +168,28 @@ private[sql] object Dataset {
class Dataset[T] private[sql](
@transient val sparkSession: SparkSession,
@DeveloperApi @InterfaceStability.Unstable @transient val
queryExecution: QueryExecution,
- encoder: Encoder[T])
+ encoder: Encoder[T],
+ val sqlText: String = "")
--- End diff --
I have decoupled the sqlText with sql execution. In current implementation,
when user invoke spark.sql(xx), it will create a new
SparkListenerSQLTextCaptured event to listenerbus. Then in
SQLAppStatusListener, the information will be stored and all the sql sentences
will display in AllExecutionPage in order with submission time, instead of in
each ExecutionPage. I will upload the commit after testing.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
