Hmm, looks like the pom.xml used in that bigtop build is different
from the one we have in our incubating candidate (probably older?) -
it was bringing in hadoop-core, hadoop-test and hadoop-tools as
dependencies from the root pom.xml with version ${hadoop20.version}.
However, I do still see that webhcat-svr has a dependency on
hadoop-core-${hadoop20.version} in the incubating candidate. We may
have to wrap that inside a profile.
On Thu, Jan 24, 2013 at 7:09 PM, Travis Crawford
<traviscrawf...@gmail.com> wrote:
> Huh that's very interesting. If you do a verbose build maven will print out
> a dep tree and we can see why it's pulled in. Can you reproduce?
>
> --Travis
>
>
> On Thursday, January 24, 2013, Sushanth Sowmyan wrote:
>>
>> And also, in this case, in the core/build/lib/compile/ directory, I
>> see the following versions of hadoop jars (notice hadoop-core-1.0.3 in
>> the midst of the others):
>>
>> --
>> hadoop-annotations-2.0.2.22-alpha.jar
>> hadoop-archives-2.0.2.22-alpha.jar
>> hadoop-auth-2.0.2.22-alpha.jar
>> hadoop-common-2.0.2.22-alpha.jar
>> hadoop-core-1.0.3.jar
>> hadoop-hdfs-2.0.2.22-alpha.jar
>> hadoop-mapreduce-client-core-2.0.2.22-alpha.jar
>> hadoop-yarn-api-2.0.2.22-alpha.jar
>> hadoop-yarn-common-2.0.2.22-alpha.jar
>> --
>>
>> On Thu, Jan 24, 2013 at 5:03 PM, Sushanth Sowmyan <khorg...@gmail.com>
>> wrote:
>> > Interestingly, I'm hitting the same issue with a bigtop build for
>> > hadoop-2. And this one isn't solved by doing a clean.
>> >
>> > On Thu, Jan 24, 2013 at 4:26 PM, Travis Crawford
>> > <traviscrawf...@gmail.com> wrote:
>> >> On Thu, Jan 24, 2013 at 4:20 PM, Timothy Potter <thelabd...@gmail.com>
>> >> wrote:
>> >>>
>> >>> Hi Travis,
>> >>>
>> >>> Thank you! Using your ant command, I have a build.
>> >>
>> >>
>> >> Do you know what changed? You might have had 0.20 hadoop jars already
>> >> resolved, which would cause a build failure without cleaning.
>> >>
>> >>
>> >>>
>> >>> Are there any instructions or known pitfalls when upgrading from 0.4
>> >>> to
>> >>> 0.5?
>> >>>
>> >>> Or, even better - can I just replace the 0.4 JARS in
>> >>> /usr/lib/hive/share/hcatalog with the new 0.5 JARs and restart the
>> >>> metastore?
>> >>
>> >>
>> >> The upgrade procedure will differ based on how you have things
>> >> deployed,
>> >> primarily if you run the HiveMetaStore as a service or you directly
>> >> connect
>> >> to the HMS. You might try running a job with just the HCatalog 0.5
>> >> client
>> >> jars against your existing metastore and see if that works for you. As
>> >> the
>> >> vast majority of HCatalog is client-side, its possible to run jobs with
>> >> different versions while you work out the upgrade procedure for your
>> >> site.
>> >>
>> >> --travis
>> >>
>> >>
>> >>
>> >>>
>> >>> Cheers,
>> >>> Tim
>> >>>
>> >>> On Thu, Jan 24, 2013 at 5:00 PM, Travis Crawford
>> >>> <traviscrawf...@gmail.com> wrote:
>> >>>>
>> >>>> Hey Timothy -
>> >>>>
>> >>>> On a clean master, this command works for me:
>> >>>>
>> >>>> ant clean package -Dmvn.hadoop.profile=hadoop23
>> >>>>
>> >>>> After doing so, we see the following jars were used in the build:
>> >>>>
>> >>>> fynn-2:hcatalog travis$ ls -1 core/build/lib/compile/*hadoop*
>> >>>> core/build/lib/compile/hadoop-annotations-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-archives-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-auth-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-common-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-hdfs-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-mapreduce-client-core-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-yarn-api-0.23.3.jar
>> >>>> core/build/lib/compile/hadoop-yarn-common-0.23.3.jar
>> >>>> fynn-2:hcatalog travis$
>> >>>>
>> >>>> I'm not familiar with how CDH4 packages things. Let's see what jars
>> >>>> you're pulling in first. Also, are you changing this line in the
>> >>>> top-level
>> >>>> pom.xml?
>> >>>>
>> >>>> <hadoop23.version>0.23.3</hadoop23.version>
>> >>>>
>> >>>> If not, I'm surprised you see issues as you wouldn't even be
>> >>>> compiling
>> >>>> against CDH4.
>> >>>>
>> >>>> --travis
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Thu, Jan 24, 2013 at 3:40 PM, Timothy Potter
>> >>>> <thelabd...@gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Would like to upgrade to 0.5 as we're maintaining 0.4 + many of the
>> >>>>> patches supplied since the 0.4 release.
>> >>>>>
>> >>>>> However, I need HCatalog to run against CDH4.1.2 which based on this
>> >>>>> link
>> >>>>>
>> >>>>> (https://ccp.cloudera.com/display/DOC/CDH+Version+and+Packaging+Information#CDHVersionandPackagingInformation-id511647)
>> >>>>> is hadoop-2.0.0+5
