FWIW it does get landed in lib_managed... i beleive there's a post in CDH group with the same very problem i had , still unresolved/unadvised by Cloudera.
On Tue, Jul 30, 2013 at 10:07 PM, Konstantin Boudnik <[email protected]> wrote: > Ok, that makes sense, although I thought examples might be a good part of > the > documentation for the beginners' references, etc. But I have problems > modifying > the assembly accordingly. Will do it the first thing in the morning. > > Thanks for the input. > Cos > > On Tue, Jul 30, 2013 at 09:57PM, Matei Zaharia wrote: > > Basically the way to think of the assembly is that it should have > libraries > > that users' client programs need to run. These are core, repl (needed if > > they use the shell), and likely bagel and streaming and mllib, though > > originally we'd opted to leave those out. We are still deciding on that > -- > > could go either way. > > > > Matei > > > > On Jul 30, 2013, at 9:56 PM, Matei Zaharia <[email protected]> > wrote: > > > > > Yeah, that is true. But the assembly shouldn't include the examples > project at all IMO -- if it does now, we should remove it. > > > > > > Matei > > > > > > On Jul 30, 2013, at 9:47 PM, Konstantin Boudnik <[email protected]> > wrote: > > > > > >> Matei, > > >> > > >> Hbase dependencies aren't actually included into the Maven assembly > as of this > > >> moment, because scope of hbase dependency in examples' module is > "compile"; but > > >> the assembly is only includes those with "runtime". Hence it is > automatically > > >> excluded. > > >> > > >> I believe, hbase is needed for examples during the execution time, > and if so - > > >> it would have to be fixed in the module. This will lead to need to > exclude it > > >> from the assembly, in turn. > > >> > > >> And of course... :) > > >> > > >> s/putt/pull/ > > >> > > >> Cos > > >> > > >> On Tue, Jul 30, 2013 at 09:08PM, Matei Zaharia wrote: > > >>> Yeah, and maybe we will want to change to Maven as the recommended > tool for > > >>> assembly building. I want to look into this more for the 0.8 release. > > >>> > > >>> Matei > > >>> > > >>> On Jul 30, 2013, at 9:04 PM, Konstantin Boudnik <[email protected]> > wrote: > > >>> > > >>>> On Tue, Jul 30, 2013 at 08:44PM, Matei Zaharia wrote: > > >>>>> Let's at the very least make it configurable, but an even better > thing will > > >>>>> be to make sbt assembly not include it. I think the only thing > that depends > > >>>>> on HBase is the examples project, but unfortunately SBT puts all > its JARs in > > >>>>> the lib_managed folder and just stupidly creates an assembly by > grouping > > >>>>> those. The Maven build, for example, should not do that. > > >>>> > > >>>> It is very easy to exclude dependencies in Maven assembly, like it > is done for > > >>>> Hadoop. Lemme send out a putt request - a good finding indeed, > Dmitriy, thank > > >>>> you! > > >>>> > > >>>> Cos > > >>>> > > >>>>> Matei > > >>>>> > > >>>>> On Jul 30, 2013, at 7:40 PM, Dmitriy Lyubimov <[email protected]> > wrote: > > >>>>> > > >>>>>> Hello, > > >>>>>> > > >>>>>> after couple of days(!) of trying to understand where i get the > > >>>>>> "NoSuchMethod" error, i traced it down to the fact that 0.8 now > includes > > >>>>>> hbase. > > >>>>>> > > >>>>>> While it is assumed that hadoop version is specified, hbase > version is > > >>>>>> fixed. This seem to create problem if hbase is used with a > particular > > >>>>>> version of CDH hadoop client in the backend. (there's a known > compatibility > > >>>>>> bug). > > >>>>>> > > >>>>>> wouldn't it make sense in this case to allow to declare hbase > version as > > >>>>>> well, perhaps even tie it to the CDH version? > > >>>>>> > > >>>>>> At the very least i think it deserves a specific mention in the > header > > >>>>>> section to provide opportunity to override, just like hadoop > version does? > > >>>>>> > > >>>>>> Thanks. > > >>>>>> -D > > >>>>> > > >>> > > > > > >
