[
https://issues.apache.org/jira/browse/HIVE-16391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16501408#comment-16501408
]
ASF GitHub Bot commented on HIVE-16391:
---------------------------------------
GitHub user jerryshao opened a pull request:
https://github.com/apache/hive/pull/364
HIVE-16391: Add a new classifier for hive-exec to be used by Spark
This fix adding a new classifier for hive-exec artifact (`core-spark`),
which is specifically used for Spark. Details in
[SPARK-20202](https://issues.apache.org/jira/browse/SPARK-20202).
This is because original hive-exec packages many transitive dependencies
into shaded jar without relocation, this makes conflicts in Spark. Spark only
needs to relocate protobuf and kryo jar. So here propose to add a new
classifier to generate a new artifact only for Spark.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/hive 1.2-spark-fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/hive/pull/364.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #364
----
commit bb27b260d82fa0a77d9fea3c123f2af8f1ea88aa
Author: jerryshao <sshao@...>
Date: 2018-06-05T06:59:37Z
HIVE-16391: Add a new classifier for hive-exec to be used by Spark
----
> Publish proper Hive 1.2 jars (without including all dependencies in uber jar)
> -----------------------------------------------------------------------------
>
> Key: HIVE-16391
> URL: https://issues.apache.org/jira/browse/HIVE-16391
> Project: Hive
> Issue Type: Task
> Components: Build Infrastructure
> Reporter: Reynold Xin
> Priority: Major
> Labels: pull-request-available
>
> Apache Spark currently depends on a forked version of Apache Hive. AFAIK, the
> only change in the fork is to work around the issue that Hive publishes only
> two sets of jars: one set with no dependency declared, and another with all
> the dependencies included in the published uber jar. That is to say, Hive
> doesn't publish a set of jars with the proper dependencies declared.
> There is general consensus on both sides that we should remove the forked
> Hive.
> The change in the forked version is recorded here
> https://github.com/JoshRosen/hive/tree/release-1.2.1-spark2
> Note that the fork in the past included other fixes but those have all become
> unnecessary.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
