GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/12215
[SPARK-14435] Shade Kryo in our custom Hive 1.2.1 fork
This patch updates our custom Hive 1.2.1 fork in order to shade Kryo in
Hive. This is a blocker for upgrading Spark to use Kryo 3 (see #12076).
The source for this new fork of Hive can be found at
https://github.com/JoshRosen/hive/tree/release-1.2.1-spark2
Here's the complete diff from the official Hive 1.2.1 release:
https://github.com/apache/hive/compare/release-1.2.1...JoshRosen:release-1.2.1-spark2
Here's the diff from the sources that @pwendell used to publish the current
`1.2.1.spark` release of Hive:
https://github.com/pwendell/hive/compare/release-1.2.1-spark...JoshRosen:release-1.2.1-spark2.
This diff looks large because his branch used a shell script to rewrite the
groupId, whereas I had to commit the groupId changes in order to prevent the
find-and-replace from affecting the package names in our relocated Kryo
classes:
https://github.com/pwendell/hive/compare/release-1.2.1-spark...JoshRosen:release-1.2.1-spark2#diff-6ada9aaec70e069df8f2c34c5519dd1e
Using these changes, I was able to publish a local version of Hive and
verify that this change fixes the test failures which are blocking #12076. Note
that this PR will not compile until we complete the review of the Hive POM
changes and stage and publish a release.
/cc @vanzin, @steveloughran, and @pwendell for review.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark shade-kryo-in-hive
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12215.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12215
----
commit 8f74c7485886cbe3af4963115a2e9d8225f11693
Author: Josh Rosen <[email protected]>
Date: 2016-04-06T19:07:47Z
[SPARK-14435] Shade Kryo in our custom Hive 1.2.1 fork
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
