[
https://issues.apache.org/jira/browse/HIVE-984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834544#action_12834544
]
Carl Steinbach commented on HIVE-984:
-------------------------------------
bq. If at all possible, we want to keep building all supported shims as part
of ant package to make sure that when a change breaks one, the developer finds
out early (before even submitting a bad patch)
Unless your change specifically mucks with the shim code I think it's unlikely
that you're going to introduce a compile time error. It seems more likely that
you would cause a test error, and that's something you will only catch if you
run the full test suite against all supported versions -- something that we
only expect Hudson to do.
Which brings up another point. How do we configure JIRA/Hudson to automatically
test submitted patches? The Hadoop and Pig projects are both setup to do this,
but I can't find any references to how it was done. Do either of you know how
to set this up, or have objections to doing so?
bq. Before doing anything else, I'm going to see if a more reliable source than
archive.apache.org would address the problem. I'll test this with my home
network tomorrow, which usually fails with archive.apache.org.
Over the weekend I figured out that there are actually two different reasons
why people are encountering errors during the download process, and wanted to
make sure that everyone else is aware of this as well:
# Unable to connect to archive.apache.org: We can fix this by adding additional
apache mirrors (see http://www.apache.org/mirrors/) to the hadoop-source
resolver in ivysettings, and also by letting people know that they can
explicitly set the mirror location using the hadoop.mirror property.
# -Dhadoop.version=0.20.1: When people set hadoop.version to 0.20.1 it causes
ant to download both 0.20.0 *and* 0.20.1, which is unnecessary since the API
does not change between patch releases. But the bigger problem is that 0.20.1's
md5 checksum file on archive.apache.org contains an md5 hash along with a bunch
of other garbage that breaks ivy. We can fix this either by disabling checksums
for archive.apache.org (set ivy.checksums="" on that resolver), or by enhancing
the build script so that it ignores patch release numbers and maps 0.20.1 to
0.20.0.
> Building Hive occasionally fails with Ivy error:
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> -------------------------------------------------------------------------------------------------------
>
> Key: HIVE-984
> URL: https://issues.apache.org/jira/browse/HIVE-984
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Carl Steinbach
> Assignee: Carl Steinbach
> Attachments: HIVE-984.2.patch, HIVE-984.patch
>
>
> Folks keep running into this problem when building Hive from source:
> {noformat}
> [ivy:retrieve]
> [ivy:retrieve] :: problems summary ::
> [ivy:retrieve] :::: WARNINGS
> [ivy:retrieve] [FAILED ]
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> expected=hadoop-0.20.1.tar.gz: computed=719e169b7760c168441b49f405855b72
> (138662ms)
> [ivy:retrieve] [FAILED ]
> hadoop#core;0.20.1!hadoop.tar.gz(source): invalid md5:
> expected=hadoop-0.20.1.tar.gz: computed=719e169b7760c168441b49f405855b72
> (138662ms)
> [ivy:retrieve] ==== hadoop-resolver: tried
> [ivy:retrieve]
> http://archive.apache.org/dist/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: FAILED DOWNLOADS ::
> [ivy:retrieve] :: ^ see resolution messages for details ^ ::
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve] :: hadoop#core;0.20.1!hadoop.tar.gz(source)
> [ivy:retrieve] ::::::::::::::::::::::::::::::::::::::::::::::
> [ivy:retrieve]
> [ivy:retrieve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
> {noformat}
> The problem appears to be either with a) the Hive build scripts, b) ivy, or
> c) archive.apache.org
> Besides fixing the actual bug, one other option worth considering is to add
> the Hadoop jars to the
> Hive source repository.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
