I was more on the thinking to avoid/reduce pulling in hbase dependencies into hbase-spark, and that maybe even hbase-spark can depend on shaded client and server -- it will be easier and more feasible if the shaded client becomes the default as you mentioned.
Your idea that hbase-spark itself becomes a shaded artifact sounds better if I understand you correctly. The spark/scala dependencies are 'provided' already. Jerry On Wed, Feb 8, 2017 at 6:14 PM, Nick Dimiduk <ndimi...@gmail.com> wrote: > On Wed, Feb 8, 2017 at 10:24 AM Jerry He <jerry...@gmail.com> wrote: > > > Yeah. Talking about the dependency, the hbase-spark module already has > > dependency on hbase-server (coming from the spark bulk load producing > > hfiles). > > This is not very good. We have to be careful not entangling it more. > > Also, there is already problem running the hbase-spark due to dependency > > conflict, and one has to be careful about the order of the classpath to > > make it work. > > We own the hbase-spark module, do we not? In that case, we control our own > destiny. An explicit goal of this effort would be to make use of that > module agnostic to classpath load order. As per my earlier reply, > hbase-spark could itself be an artifact shaded over hbase-server and all of > its dependencies. That way the user doesn't need to think about it at all. > > It further seems to me that the maven-shade-plugin could gain a new analyze > goal, similar to the that of the dependency-plugin, which would audit the > classes packaged in a jar vs the contract defined in configuration. This > could further be used to fail the build of there's any warnings reported. > I've found myself wanting this very functionality as I consume ES and > Phoenix, shaded in downstream projects. >
