busbey commented on issue #42: HBASE-22817 Use hbase-shaded dependencies in hbase-spark URL: https://github.com/apache/hbase-connectors/pull/42#issuecomment-535180290 Okay I figured this out. the easiest way to reproduce the problem is to just go to the top level and do `mvn --batch-mode clean test-compile -DskipTests=true` the problem is this bit in hbase-spark ``` <dependency> <groupId>org.apache.hbase.connectors.spark</groupId> <artifactId>hbase-spark-protocol-shaded</artifactId> <exclusions> <exclusion> <groupId>org.apache.hbase.connectors.spark</groupId> <artifactId>hbase-spark-protocol</artifactId> </exclusion> </exclusions> </dependency> ``` So long as we run though a `package` phase this does the correct thing. We get a dependency on the classes in `hbase-spark-protocol-shaded` that have our generated classes from `hbase-spark-protocol` rewritten to use the shaded protobuf and avoid an extra copy of those classes without the rewrite. The problem presents when we don't do a package phase. At that point maven relies on the classes output by modules included in a reactor build (rather than e.g. jars from a prior installed build) and if we look at the `hbase-spark-protocol-shaded` module we'll see there are none: ``` Busbey-MBA:hbase-connectors busbey$ find spark/hbase-spark-protocol-shaded/target/ spark/hbase-spark-protocol-shaded/target/ spark/hbase-spark-protocol-shaded/target//test-classes spark/hbase-spark-protocol-shaded/target//test-classes/META-INF spark/hbase-spark-protocol-shaded/target//test-classes/META-INF/LICENSE spark/hbase-spark-protocol-shaded/target//test-classes/META-INF/DEPENDENCIES spark/hbase-spark-protocol-shaded/target//test-classes/META-INF/NOTICE spark/hbase-spark-protocol-shaded/target//maven-shared-archive-resources spark/hbase-spark-protocol-shaded/target//maven-shared-archive-resources/META-INF spark/hbase-spark-protocol-shaded/target//maven-shared-archive-resources/META-INF/LICENSE spark/hbase-spark-protocol-shaded/target//maven-shared-archive-resources/META-INF/DEPENDENCIES spark/hbase-spark-protocol-shaded/target//maven-shared-archive-resources/META-INF/NOTICE spark/hbase-spark-protocol-shaded/target//classes spark/hbase-spark-protocol-shaded/target//classes/META-INF spark/hbase-spark-protocol-shaded/target//classes/META-INF/LICENSE spark/hbase-spark-protocol-shaded/target//classes/META-INF/DEPENDENCIES spark/hbase-spark-protocol-shaded/target//classes/META-INF/NOTICE spark/hbase-spark-protocol-shaded/target//.plxarc Busbey-MBA:hbase-connectors busbey$ ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
