While not 100% correct, IMO making Hadoop an optional dependency it may be the way to go for Hadoop downstream projects. By doing this, the users of the downstream project can set the version of Hadoop they need and do the exclusions for that version they use without having to worry about all exclusions for all possible Hadoop versions.
Thanks. Alejandro On Tue, Nov 15, 2011 at 7:25 PM, Roman Shaposhnik <[email protected]> wrote: > Sorry for reviving the dead thread, but I have, like, a real problem > to solve ;-) > > On Fri, Nov 11, 2011 at 2:09 PM, Alejandro Abdelnur <[email protected]> wrote: >> Eric, >> >> Do you mean that the HBASE published POM won't have a Hadoop artifact >> as a dependency? > > Ok, so lets see what the problem here really is (and also verify that > my understanding is correct): > 1. we have a binary Maven artifact for HBase that is pretty well > insulated from the underlying Hadoop by a layer of shims and thus > can work with *any* version of Hadoop selected at run time. Basically, > my reading of Todd's reply is that there's no *reason* to have multiple > versions of hbase-0.92.jar > > 2. we have a test binary artifact that is NOT insulated. Case in point: > org.apache.hadoop.hbase.mapreduce.NMapInputFormat is failing because > org.apache.hadoop.mapreduce.JobContext is either class or interface > depending on which version of Hadoop you compile it against. > > Now given the above it sounds like a proper way to fix this is to provide > a level of shims for tests and make them run against any version of Hadoop. > > Agreed? > > That leaves us with a single problem -- when both artifacts become hadoop > agnostic we still have to put *some* version of Hadoop into our POM file > as a dependency AND we have to either: > 1. make that dependency optional > 2. make all of the downstream exclude it > > Personally, I'd go with #1 since it makes things much more explicit. But what > do others think? > > Thanks, > Roman. >
