Hi Jordan! How do you currently pull in the dependency? Do you need us to publish an artifact to maven central? Would a convenience binary built against spark-3 on downloads.apache.org suffice?
On Thu, Oct 7, 2021 at 7:55 PM Jordan Hambleton <[email protected]> wrote: > Hi Peter, > > We're seeing an uptick in usage with Spark 3. While hadoop-connectors have > started supporting Spark 3 (HBASE-25326 > <https://issues.apache.org/jira/browse/HBASE-25326>) and while it's fine > pulling down the repo and building from source, it would be a lot easier if > we had a release for supporting Spark 3. > > Is it possible to get a next release of the hadoop-connectors out or is > there another way others are using for Spark3 integration with HBase? > > Below is what I'm referring to in building. > > To build the connector with Spark 3.0, compile it with scala 2.12. > Additional configurations that you can customize are the Spark version, > HBase version, and Hadoop version. Example: > > $ mvn -Dspark.version=3.0.1 -Dscala.version=2.12.10 > -Dscala.binary.version=2.12 -Dhbase.version=2.2.4 -Dhadoop.profile=3.0 > -Dhadoop-three.version=3.2.0 -DskipTests clean package > > best regards, > Jordan Hambleton >
