[
https://issues.apache.org/jira/browse/HBASE-30189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-30189:
-----------------------------------------
Description:
The spark4 module currently has basic HBase RDD operations (put, get, delete,
foreachPartition, mapPartitions). The next step is porting the SparkSQL
datasource layer from the spark3 module, which enables reading/writing HBase
tables via Spark SQL (spark.read.format("hbase")...). The spark3 module uses
the deprecated DataSource V1 API (RelationProvider, BaseRelation,
PrunedFilteredScan).
For spark4, we'll implement the modern DataSource V2 API, which provides better
performance (columnar reads, push-down at the connector level) and a cleaner
contract.
was:This is to deal with the work of bringing the hbase-spark datasource
package to the spark4 module.
> Port the whole data source stack to spark4 module.
> --------------------------------------------------
>
> Key: HBASE-30189
> URL: https://issues.apache.org/jira/browse/HBASE-30189
> Project: HBase
> Issue Type: Sub-task
> Components: hbase-connectors
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> The spark4 module currently has basic HBase RDD operations (put, get, delete,
> foreachPartition, mapPartitions). The next step is porting the SparkSQL
> datasource layer from the spark3 module, which enables reading/writing HBase
> tables via Spark SQL (spark.read.format("hbase")...). The spark3 module uses
> the deprecated DataSource V1 API (RelationProvider, BaseRelation,
> PrunedFilteredScan).
> For spark4, we'll implement the modern DataSource V2 API, which provides
> better performance (columnar reads, push-down at the connector level) and a
> cleaner contract.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
