Sean Busbey commented on HBASE-16179:

bq. I am back in US. There should be no ambiguity who would be doing the work 
for this JIRA.

Good to hear.

Looks like we are going to support Scala 2.10 and 2.11 with Spark 1.6 and 2.x 
How many artifacts would be produced by the build process ?

we support Scala 2.10 and 2.11 with and spark 2.0 to spark 2.2 combinations, no 
more spark1.6 needed

We don't need to work with Spark 1.6, agreed. Agreed we need to run on Spark 
2.z. currently the [spark dist 
area|https://dist.apache.org/repos/dist/release/spark/] has a 2.0, 2.1, and 2.2 
version. (presuming we get HBase 2.0 out the door before there's a Spark 2.3)

We definitely need a build process that makes appropriate artifacts for runtime 
of the set of scala x spark versions. I don't know if we need a different 
compilation for the different spark minor versions. Presuming worse case 
(needing a different artifact for each minor spark version), that'd be 6 

bq.  Maybe for this jira, we do not need to consider artifacts, and open an 
other jira to talk about the artifacts? I am not so sure about this part.

I think Ted's already on track to generate multiple artifacts, presuming we 
need them. That said, if he'd rather not deal with the Spark 2.z minor version 
runtime compatibility, I'm fine with just making the current module work with 
Spark 2 for this jira so long as we have some jira that tracks getting 
everything done for the set of scala x spark versions.

> Fix compilation errors when building hbase-spark against Spark 2.0
> ------------------------------------------------------------------
>                 Key: HBASE-16179
>                 URL: https://issues.apache.org/jira/browse/HBASE-16179
>             Project: HBase
>          Issue Type: Bug
>          Components: spark
>            Reporter: Ted Yu
>            Assignee: Ted Yu
>            Priority: Critical
>              Labels: build
>             Fix For: 2.0.0
>         Attachments: 16179.v0.txt, 16179.v10.txt, 16179.v11.txt, 
> 16179.v12.txt, 16179.v12.txt, 16179.v12.txt, 16179.v13.txt, 16179.v15.txt, 
> 16179.v16.txt, 16179.v18.txt, 16179.v19.txt, 16179.v19.txt, 16179.v1.txt, 
> 16179.v1.txt, 16179.v20.txt, 16179.v22.txt, 16179.v23.txt, 16179.v24.txt, 
> 16179.v25.txt, 16179.v26.txt, 16179.v27.txt, 16179.v4.txt, 16179.v5.txt, 
> 16179.v7.txt, 16179.v8.txt, 16179.v9.txt
> I tried building hbase-spark module against Spark-2.0 snapshot and got the 
> following compilation errors:
> http://pastebin.com/bg3w247a
> Some Spark classes such as DataTypeParser and Logging are no longer 
> accessible to downstream projects.
> hbase-spark module should not depend on such classes.

This message was sent by Atlassian JIRA

Reply via email to