[
https://issues.apache.org/jira/browse/HBASE-16179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15529743#comment-15529743
]
Jonathan Hsieh commented on HBASE-16179:
----------------------------------------
Review on v7.
This fails for me:
{code}
mvn clean dependency:tree
mvn clean dependency:tree -Dspark.profile=1.6
{code}
This works
{code}
mvn clean dependency:tree -Dspark.profile=2.0
{code}
hbase-spark/pom.xml
- currently no profile is activated -- both say !spark.profile. You want
default with !spark.profile and the othres with actual values the values -- 1.6
and 2.0. Look at how hadoop.profile is handled.
- Based on sean's comments can you make the default scala 2.11? We don't have
to update as much going forward.
hbasetablescanrdd
- why is the "//with Logging" comment there? Remove?
pom.xml
- consider calling the modules hbase-spark1.6-compat and hbase-spark2.0-compat
instead of hbase-spark1.6-compat and hbase-spark-compat. This way when new
spark versions come out and we need do a ship, it will be obvious what to do.
DataTypeParserWrapper
- add comments about why this exists (e.g. necessary for spark2.0, not present
in 1.6 etc)
Logging
- add comments saying why this is present instead of using the spark1.6 Logging
class.
nits: fix space/tabs issues in hbase-spark/pom.xml
> Fix compilation errors when building hbase-spark against Spark 2.0
> ------------------------------------------------------------------
>
> Key: HBASE-16179
> URL: https://issues.apache.org/jira/browse/HBASE-16179
> Project: HBase
> Issue Type: Bug
> Components: spark
> Reporter: Ted Yu
> Assignee: Ted Yu
> Fix For: 2.0.0
>
> Attachments: 16179.v0.txt, 16179.v1.txt, 16179.v1.txt, 16179.v4.txt,
> 16179.v5.txt, 16179.v7.txt
>
>
> I tried building hbase-spark module against Spark-2.0 snapshot and got the
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
