On Jan 25, 2013, at 9:18 AM, Travis Crawford wrote:
> On Thu, Jan 24, 2013 at 7:36 PM, Sushanth Sowmyan <khorg...@gmail.com> wrote:
> Hmm, looks like the pom.xml used in that bigtop build is different
> from the one we have in our incubating candidate (probably older?) -
> it was bringing in hadoop-core, hadoop-test and hadoop-tools as
> dependencies from the root pom.xml with version ${hadoop20.version}.
>
> However, I do still see that webhcat-svr has a dependency on
> hadoop-core-${hadoop20.version} in the incubating candidate. We may
> have to wrap that inside a profile.
>
> This is intentional because webhcat does not yet build with hadoop 23. I
> believe there are a couple other submodules with this issue as well.
>
> Do we see this as a blocker for the 0.5 release? I don't believe full hadoop
> 23 support is expected yet, and I'd prefer not to block the release on this.
> I totally agree full 23 support is important, but think we should save it for
> a future release.
+1. We have some great new functionality. Let's get it out there.
Alan.
>
> Thoughts?
>
> --travis
>
>
>
>
> On Thu, Jan 24, 2013 at 7:09 PM, Travis Crawford
> <traviscrawf...@gmail.com> wrote:
> > Huh that's very interesting. If you do a verbose build maven will print out
> > a dep tree and we can see why it's pulled in. Can you reproduce?
> >
> > --Travis
> >
> >
> > On Thursday, January 24, 2013, Sushanth Sowmyan wrote:
> >>
> >> And also, in this case, in the core/build/lib/compile/ directory, I
> >> see the following versions of hadoop jars (notice hadoop-core-1.0.3 in
> >> the midst of the others):
> >>
> >> --
> >> hadoop-annotations-2.0.2.22-alpha.jar
> >> hadoop-archives-2.0.2.22-alpha.jar
> >> hadoop-auth-2.0.2.22-alpha.jar
> >> hadoop-common-2.0.2.22-alpha.jar
> >> hadoop-core-1.0.3.jar
> >> hadoop-hdfs-2.0.2.22-alpha.jar
> >> hadoop-mapreduce-client-core-2.0.2.22-alpha.jar
> >> hadoop-yarn-api-2.0.2.22-alpha.jar
> >> hadoop-yarn-common-2.0.2.22-alpha.jar
> >> --
> >>
> >> On Thu, Jan 24, 2013 at 5:03 PM, Sushanth Sowmyan <khorg...@gmail.com>
> >> wrote:
> >> > Interestingly, I'm hitting the same issue with a bigtop build for
> >> > hadoop-2. And this one isn't solved by doing a clean.
> >> >
> >> > On Thu, Jan 24, 2013 at 4:26 PM, Travis Crawford
> >> > <traviscrawf...@gmail.com> wrote:
> >> >> On Thu, Jan 24, 2013 at 4:20 PM, Timothy Potter <thelabd...@gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> Hi Travis,
> >> >>>
> >> >>> Thank you! Using your ant command, I have a build.
> >> >>
> >> >>
> >> >> Do you know what changed? You might have had 0.20 hadoop jars already
> >> >> resolved, which would cause a build failure without cleaning.
> >> >>
> >> >>
> >> >>>
> >> >>> Are there any instructions or known pitfalls when upgrading from 0.4
> >> >>> to
> >> >>> 0.5?
> >> >>>
> >> >>> Or, even better - can I just replace the 0.4 JARS in
> >> >>> /usr/lib/hive/share/hcatalog with the new 0.5 JARs and restart the
> >> >>> metastore?
> >> >>
> >> >>
> >> >> The upgrade procedure will differ based on how you have things
> >> >> deployed,
> >> >> primarily if you run the HiveMetaStore as a service or you directly
> >> >> connect
> >> >> to the HMS. You might try running a job with just the HCatalog 0.5
> >> >> client
> >> >> jars against your existing metastore and see if that works for you. As
> >> >> the
> >> >> vast majority of HCatalog is client-side, its possible to run jobs with
> >> >> different versions while you work out the upgrade procedure for your
> >> >> site.
> >> >>
> >> >> --travis
> >> >>
> >> >>
> >> >>
> >> >>>
> >> >>> Cheers,
> >> >>> Tim
> >> >>>
> >> >>> On Thu, Jan 24, 2013 at 5:00 PM, Travis Crawford
> >> >>> <traviscrawf...@gmail.com> wrote:
> >> >>>>
> >> >>>> Hey Timothy -
> >> >>>>
> >> >>>> On a clean master, this command works for me:
> >> >>>>
> >> >>>> ant clean package -Dmvn.hadoop.profile=hadoop23
> >> >>>>
> >> >>>> After doing so, we see the following jars were used in the build:
> >> >>>>
> >> >>>> fynn-2:hcatalog travis$ ls -1 core/build/lib/compile/*hadoop*
> >> >>>> core/build/lib/compile/hadoop-annotations-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-archives-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-auth-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-common-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-hdfs-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-mapreduce-client-core-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-yarn-api-0.23.3.jar
> >> >>>> core/build/lib/compile/hadoop-yarn-common-0.23.3.jar
> >> >>>> fynn-2:hcatalog travis$
> >> >>>>
> >> >>>> I'm not familiar with how CDH4 packages things. Let's see what jars
> >> >>>> you're pulling in first. Also, are you changing this line in the
> >> >>>> top-level
> >> >>>> pom.xml?
> >> >>>>
> >> >>>> <hadoop23.version>0.23.3</hadoop23.version>
> >> >>>>
> >> >>>> If not, I'm surprised you see issues as you wouldn't even be
> >> >>>> compiling
> >> >>>> against CDH4.
> >> >>>>
> >> >>>> --travis
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> On Thu, Jan 24, 2013 at 3:40 PM, Timothy Potter
> >> >>>> <thelabd...@gmail.com>
> >> >>>> wrote:
> >> >>>>>
> >> >>>>> Would like to upgrade to 0.5 as we're maintaining 0.4 + many of the
> >> >>>>> patches supplied since the 0.4 release.
> >> >>>>>
> >> >>>>> However, I need HCatalog to run against CDH4.1.2 which based on this
> >> >>>>> link
> >> >>>>>
> >> >>>>> (https://ccp.cloudera.com/display/DOC/CDH+Version+and+Packaging+Information#CDHVersionandPackagingInformation-id511647)
> >> >>>>> is hadoop-2.0.0+5
>
