leaves12138 commented on code in PR #9306:
URL: https://github.com/apache/paimon/pull/9306#discussion_r3813148063
##########
paimon-spark/paimon-spark-ut/src/test/scala/org/apache/paimon/spark/sql/PrimaryKeySortedIndexTest.scala:
##########
@@ -18,17 +18,83 @@
package org.apache.paimon.spark.sql
+import org.apache.paimon.data.BinaryString
import org.apache.paimon.globalindex.IndexedSplit
+import org.apache.paimon.predicate.{ArrayContains, ArrayContainsAll,
ArraysOverlap, LeafPredicate, PredicateBuilder}
import org.apache.paimon.spark.PaimonSparkTestBase
import org.apache.paimon.table.source.DataSplit
import org.apache.spark.sql.Row
import scala.collection.JavaConverters._
-/** End-to-end Spark SQL tests for source-backed primary-key BTree and Bitmap
indexes. */
+/** End-to-end Spark SQL tests for source-backed primary-key sorted indexes. */
class PrimaryKeySortedIndexTest extends PaimonSparkTestBase {
+ test("Spark array predicates use multivalue index") {
Review Comment:
This suite is shared by all Spark version modules and is also executed for
Spark 3.2 in the Spark 3.x workflow. However, this test unconditionally expects
array predicate pushdown, while `PaimonSparkSessionExtensions` deliberately
disables `PushDownArrayPredicates` on Spark 3.2. I reproduced the failure in
`paimon-spark-3.2`: `pushedDataFilters` is only `Array(IsNotNull(tags))` and
does not contain `ArrayContains(tags, red)`. Please gate this test for Spark >=
3.3 or move the pushdown-specific coverage to version-specific suites.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
