Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/22684#discussion_r224005596
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcFilterSuite.scala
---
@@ -382,5 +382,40 @@ class OrcFilterSuite extends OrcTest with
SharedSQLContext {
))
)).get.toString
}
+
+ // Can not remove unsupported `StringContains` predicate since it is
under `Or` operator.
+ assert(OrcFilters.createFilter(schema, Array(
+ Or(
+ LessThan("a", 10),
+ And(
+ StringContains("b", "prefix"),
+ GreaterThan("a", 1)
+ )
+ )
+ )).isEmpty)
+
+ // Safely remove unsupported `StringContains` predicate and push down
`LessThan`
+ assertResult("leaf-0 = (LESS_THAN a 10), expr = leaf-0") {
+ OrcFilters.createFilter(schema, Array(
+ And(
+ LessThan("a", 10),
+ StringContains("b", "prefix")
+ )
+ )).get.toString
+ }
+
+ // Safely remove unsupported `StringContains` predicate and push down
`LessThan`
--- End diff --
There is another predicate pushed down. We shall mention it too?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
