[
https://issues.apache.org/jira/browse/HBASE-30510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mazhengxuan reassigned HBASE-30510:
-----------------------------------
Assignee: mazhengxuan
> Make predicate pushdown type-aware in Spark connectors
> ------------------------------------------------------
>
> Key: HBASE-30510
> URL: https://issues.apache.org/jira/browse/HBASE-30510
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
> Reporter: mazhengxuan
> Assignee: mazhengxuan
> Priority: Major
>
> Some comparisons are accepted for pushdown even though NaiveEncoder cannot
> encode their values correctly; byte comparisons and timestamp literals are
> examples that can produce incorrect results or exceptions. Since these
> filters are reported as handled, Spark may not evaluate them again.
> We should check both the data type and operator before accepting a filter,
> leave unsupported comparisons to Spark, and avoid unsafe row-key pruning for
> them, with tests comparing results with pushdown enabled and disabled in both
> Spark 3 and Spark 4.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
