HyukjinKwon commented on a change in pull request #25008: [SPARK-28213][SQL]
Replace ColumnarBatchScan with equivilant from Columnar
URL: https://github.com/apache/spark/pull/25008#discussion_r307788625
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/test/SQLTestUtils.scala
##########
@@ -392,7 +392,7 @@ private[sql] trait SQLTestUtilsBase
*/
protected def stripSparkFilter(df: DataFrame): DataFrame = {
val schema = df.schema
- val withoutFilters = df.queryExecution.sparkPlan.transform {
+ val withoutFilters = df.queryExecution.executedPlan.transform {
Review comment:
I don't know the correct way to fix it for now - it needs some
investigations. Can you clarify why it didn't work well?
The problem seems like by the internal behaviour changes - we now rely on
`ApplyColumnarRulesAndInsertTransitions`. Can we make some investigation to
confirm that it doesn't affect anything, and list up what changes were made in
this PR description?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
