Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19971#discussion_r156914630
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/TPCDSQuerySuite.scala ---
@@ -348,13 +351,37 @@ class TPCDSQuerySuite extends QueryTest with
SharedSQLContext with BeforeAndAfte
"q81", "q82", "q83", "q84", "q85", "q86", "q87", "q88", "q89", "q90",
"q91", "q92", "q93", "q94", "q95", "q96", "q97", "q98", "q99")
+ private def checkGeneratedCode(plan: SparkPlan): Unit = {
+ val codegenSubtrees = new
collection.mutable.HashSet[WholeStageCodegenExec]()
+ plan foreach {
+ case s: WholeStageCodegenExec =>
+ codegenSubtrees += s
+ case s => s
+ }
+ codegenSubtrees.toSeq.foreach { subtree =>
+ val code = subtree.doCodeGen()._2
+ try {
+ // Just check the generated code can be properly compiled
+ CodeGenerator.compile(code)
+ } catch {
+ case e: Exception =>
+ logError(s"failed to compile: $e", e)
--- End diff --
ya, I know though, the logging is there because compilation errors lead to
the fallbacks and then the exception messages are not passed to users. But, in
this test case, we explicitly throw an exception below with the message. So, I
said a bit meaningless.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
