mazhengxuan created HBASE-30510:
-----------------------------------
Summary: Make predicate pushdown type-aware in Spark connectors
Key: HBASE-30510
URL: https://issues.apache.org/jira/browse/HBASE-30510
Project: HBase
Issue Type: Sub-task
Components: hbase-connectors
Reporter: mazhengxuan
Some comparisons are accepted for pushdown even though NaiveEncoder cannot
encode their values correctly; byte comparisons and timestamp literals are
examples that can produce incorrect results or exceptions. Since these filters
are reported as handled, Spark may not evaluate them again.
We should check both the data type and operator before accepting a filter,
leave unsupported comparisons to Spark, and avoid unsafe row-key pruning for
them, with tests comparing results with pushdown enabled and disabled in both
Spark 3 and Spark 4.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
