[
https://issues.apache.org/jira/browse/HBASE-28214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Istvan Toth updated HBASE-28214:
--------------------------------
Description:
The README for the Spark connector details the classpath requirements for the
HBase server side, but does not talk about how to set up the Spark classpath
for HBase.
The Cloudera docs
[https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html]
suggest using "hbase mapredcp" It is, however inconsistent, as "hbase
mapredcp" includes the unshaded hadoop libraries, while the example command
line omits the hadoop libraries (and seem to depend on the on the existing
Hadoop JARs on the Spark classpath).
Figure this out, and update the documentation.
was:
The README for the Spark connector details the classpath requirements for the
HBase server side, but does not talk about how to set up the Spark classpath
for HBase.
The Cloudera docs
[https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html]
suggest using "hbase mapredcp" It is, however inconsistent, as "hbase
mapredcp" includes the unshaded hadoop libraries, while the example command
line omits the hadoop libraries.
(and seem to depend on the Hadoop JARs included in the hbase-shaded-mapreduce
JAR, or perhaps on the existing Hadoop JARs on the Spark classpath, depending
on the classpath ordering and the phase of the moon).
Figure this out, and update the documentation.
> Document Spark classpath requirements for the Spark connector
> -------------------------------------------------------------
>
> Key: HBASE-28214
> URL: https://issues.apache.org/jira/browse/HBASE-28214
> Project: HBase
> Issue Type: Bug
> Components: spark
> Reporter: Istvan Toth
> Priority: Major
>
> The README for the Spark connector details the classpath requirements for the
> HBase server side, but does not talk about how to set up the Spark classpath
> for HBase.
> The Cloudera docs
> [https://docs.cloudera.com/cdp-private-cloud-base/7.1.9/accessing-hbase/topics/hbase-configure-spark-connector.html]
> suggest using "hbase mapredcp" It is, however inconsistent, as "hbase
> mapredcp" includes the unshaded hadoop libraries, while the example command
> line omits the hadoop libraries (and seem to depend on the on the existing
> Hadoop JARs on the Spark classpath).
> Figure this out, and update the documentation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
