sarutak opened a new pull request #25274: [SPARK-28537][SQL] DebugExec cannot debug broadcast or columnar related queries. URL: https://github.com/apache/spark/pull/25274 DebugExec does not implement doExecuteBroadcast and doExecuteColumnar so we can't debug broadcast or columnar related query. One example for broadcast is here. ``` val df1 = Seq(1, 2, 3).toDF val df2 = Seq(1, 2, 3).toDF val joined = df1.join(df2, df1("value") === df2("value")) joined.debug() java.lang.UnsupportedOperationException: Debug does not implement doExecuteBroadcast ... ``` Another for columnar is here. ``` val df = Seq(1, 2, 3).toDF df.persist df.debug() java.lang.IllegalStateException: Internal Error class org.apache.spark.sql.execution.debug.package$DebugExec has column support mismatch: ... ``` ## How was this patch tested? Additional test cases in DebuggingSuite.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
