[
https://issues.apache.org/jira/browse/HBASE-30178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18136100#comment-18136100
]
mazhengxuan commented on HBASE-30178:
-------------------------------------
Thanks for confirming, Wellington! I'll create the seven sub-tasks under
HBASE-30178 and start with the first two, since they can affect query results.
> Add support to spark 4 in hbase-connectors
> ------------------------------------------
>
> Key: HBASE-30178
> URL: https://issues.apache.org/jira/browse/HBASE-30178
> Project: HBase
> Issue Type: Umbrella
> Components: hbase-connectors
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> With spark 4.0.0 GA for around a year now, we should start working on making
> hbase-connectors integrate with spark 4.
> Since we would still need to support spark 3 for a while and due to spark 4
> requiring minimum jdk17 and scala 2.13, it would be easier to define a
> specific spark 4 module under the hbase-connectors root, so that
> spark.version, scala.version, compiler flags, and optional spark-4 only APIs
> never collide with spark 3 in a single parent pom.
> We should keep feature parity as much as possible. Datasources, context, rdd
> and connection cache related logic can be duplicated under this new
> hbase-spark4 module, in order to compile and package with jdk17 and scala
> 2.13. This module build should be isolated on a specific maven profile, so it
> wouldn't be build by default.
> Pushdown filter related classes and filter protocol would remain on a common
> module (no code duplication needed).
>
>
>
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
