[
https://issues.apache.org/jira/browse/HIVE-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834519#action_12834519
]
John Sichi commented on HIVE-984:
---------------------------------
I spoke with Zheng about this and here's what we came up with. Carl, let me
know if this works for you.
* If at all possible, we want to keep building all supported shims as part of
ant package to make sure that when a change breaks one, the developer finds out
early (before even submitting a bad patch)
* The long term plan does involve deprecating and eventually dropping support
for older Hadoop versions. The fact that Facebook still has some dependencies
on 0.17 probably explains why that is currently the oldest version, but the
standard voting procedure can be used at the project level for initiating a
deprecation process going forward.
* Regardless of how many Hadoop versions we support, the current Hadoop+ivy
situation is definitely broken, and we need to fix it ASAP since it can be a
major impediment to new or existing contributors.
* Before doing anything else, I'm going to see if a more reliable source than
archive.apache.org would address the problem. I'll test this with my home
network tomorrow, which usually fails with archive.apache.org.
* If a more reliable source would help, then we'll see if we can get
mirror.facebook.net to provide all supported Hadoop versions (currently only
apache.archive.org has the old ones), and if that's the case, then we'll check
in a change to build.properties to make it the default source.
* If either of the above is not the case, then we can do what you proposed in
HIVE-1171 (check the Hadoop dependencies into svn instead).
> Building Hive occasionally fails with Ivy error:
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> -------------------------------------------------------------------------------------------------------
>
> Key: HIVE-984
> URL: https://issues.apache.org/jira/browse/HIVE-984
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-984.2.patch, HIVE-984.patch
>
>
> Folks keep running into this problem when building Hive from source:
> {noformat}
> [ivy:retrieve]
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve] :::: WARNINGS
> [ivy:retrieve] [FAILED ]
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> expected=hadoop-0.20.1.tar.gz: computed=719e169b7760c168441b49f405855b72
> (138662ms)
> [ivy:retrieve] [FAILED ]
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> expected=hadoop-0.20.1.tar.gz: computed=719e169b7760c168441b49f405855b72
> (138662ms)
> [ivy:retrieve] ==== hadoop-resolver: tried
> [ivy:retrieve]
> http://archive.apache.org/dist/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: FAILED DOWNLOADS ::
> [ivy:retrieve] :: ^ see resolution messages for details ^ ::
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: hadoop#core;0.20.1!hadoop.tar.gz(source)
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]
> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> {noformat}
> The problem appears to be either with a) the Hive build scripts, b) ivy, or
> c) archive.apache.org
> Besides fixing the actual bug, one other option worth considering is to add
> the Hadoop jars to the
> Hive source repository.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
