WangGuangxin commented on a change in pull request #23942:
[SPARK-27033][SQL]Add Optimize rule RewriteArithmeticFiltersOnIntOrLongColumn
URL: https://github.com/apache/spark/pull/23942#discussion_r263191418
##########
File path:
sql/hive/src/test/scala/org/apache/spark/sql/hive/OptimizeHiveMetadataOnlyQuerySuite.scala
##########
@@ -65,11 +65,11 @@ class OptimizeHiveMetadataOnlyQuerySuite extends QueryTest
with TestHiveSingleto
// verify the matching partitions
val partitions = spark.internalCreateDataFrame(Distinct(Filter(($"x" <
5).expr,
- Project(Seq(($"part" + 1).as("x").expr.asInstanceOf[NamedExpression]),
+ Project(Seq(($"part" * 1).as("x").expr.asInstanceOf[NamedExpression]),
Review comment:
Because with this optimizer in this PR, `part + 1 < 5` will be optimized to
'part < 4' , where `part` is a partition column, so it only need to fetch 4
partitions instead of 11, so the last assert
`assert(HiveCatalogMetrics.METRIC_PARTITIONS_FETCHED.getCount - startCount ==
11)` will fail.
From the comments in this test, it wants to test the case where `verify that
the partition predicate was not pushed down to the metastore`, so I changed to
`part * 1`, which will not be optimized.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
