This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-4.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.0 by this push:
new fc626b95def8 [SPARK-50872][SQL][UI] Makes ToPrettyString expression
not affect UI presentation
fc626b95def8 is described below
commit fc626b95def869c73da5d48c26d0098a5de87ecb
Author: Kent Yao <[email protected]>
AuthorDate: Sat Jan 18 14:48:12 2025 -0800
[SPARK-50872][SQL][UI] Makes ToPrettyString expression not affect UI
presentation
### What changes were proposed in this pull request?
The Plan graph and plan detail are affected by the injected ToPrettyString
expression. They are not consistent with the output of our `explain` API. If
there are a lot of output columns, the UI looks malformed.
### Why are the changes needed?
It's not good to keep noise on the user's daily debug tool
### Does this PR introduce _any_ user-facing change?
No, this only changes the ongoing 4.0
### How was this patch tested?
Added UT and UI manual test
### Was this patch authored or co-authored using generative AI tooling?
no
Closes #49551 from yaooqinn/SPARK-50872.
Authored-by: Kent Yao <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit c3f6c58b890337914f4efb66b73aa0b58f81d2dc)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.../org/apache/spark/sql/catalyst/expressions/ToPrettyString.scala | 2 ++
.../apache/spark/sql/catalyst/expressions/ToPrettyStringSuite.scala | 6 ++++++
2 files changed, 8 insertions(+)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyString.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyString.scala
index f6fc9b3abd65..d18fa7c13892 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyString.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyString.scala
@@ -73,4 +73,6 @@ case class ToPrettyString(child: Expression, timeZoneId:
Option[String] = None)
|""".stripMargin
ev.copy(code = finalCode, isNull = FalseLiteral)
}
+
+ override def sql: String = child.sql
}
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyStringSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyStringSuite.scala
index 2a5f76cab361..64529bf54bd2 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyStringSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ToPrettyStringSuite.scala
@@ -128,4 +128,10 @@ class ToPrettyStringSuite extends SparkFunSuite with
ExpressionEvalHelper {
val v = new VariantVal(Array[Byte](1, 2, 3), Array[Byte](1, 1))
checkEvaluation(ToPrettyString(Literal(v)),
UTF8String.fromString(v.toString))
}
+
+ test("sql method is equalivalent to child's sql") {
+ val child = Literal(1)
+ val prettyString = ToPrettyString(child)
+ assert(prettyString.sql === child.sql)
+ }
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
