[
https://issues.apache.org/jira/browse/HBASE-28214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated HBASE-28214:
--------------------------------
Description:
The README for the Spark connector details the classpath requirements for the
HBase server side, but does not talk about how to set up the Spark classpath
for HBase.
The Cloudera docs
[https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html]
suggest using "hbase mapredcp" It is, however inconsistent, as "hbase
mapredcp" includes the unshaded hadoop libraries, while the example command
line omits the hadoop libraries (and seem to depend on the on the existing
Hadoop JARs on the Spark classpath).
Figure this out, and update the documentation.
UPDATE:
SPARK-33618 has reverted to using unshaded Hadoop.
We only have to document the `hbase mapredcp` option.
was:
The README for the Spark connector details the classpath requirements for the
HBase server side, but does not talk about how to set up the Spark classpath
for HBase.
The Cloudera docs
[https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html]
suggest using "hbase mapredcp" It is, however inconsistent, as "hbase
mapredcp" includes the unshaded hadoop libraries, while the example command
line omits the hadoop libraries (and seem to depend on the on the existing
Hadoop JARs on the Spark classpath).
Figure this out, and update the documentation.
> Document Spark classpath requirements for the Spark connector
> -------------------------------------------------------------
>
> Key: HBASE-28214
> URL: https://issues.apache.org/jira/browse/HBASE-28214
> Project: HBase
> Issue Type: Bug
> Components: spark
> Reporter: Istvan Toth
> Priority: Major
>
> The README for the Spark connector details the classpath requirements for the
> HBase server side, but does not talk about how to set up the Spark classpath
> for HBase.
> The Cloudera docs
> [https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html]
> suggest using "hbase mapredcp" It is, however inconsistent, as "hbase
> mapredcp" includes the unshaded hadoop libraries, while the example command
> line omits the hadoop libraries (and seem to depend on the on the existing
> Hadoop JARs on the Spark classpath).
> Figure this out, and update the documentation.
> UPDATE:
> SPARK-33618 has reverted to using unshaded Hadoop.
> We only have to document the `hbase mapredcp` option.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
