Github user steveloughran commented on the pull request:
https://github.com/apache/spark/pull/7191#issuecomment-126943233
1. This is a rebased patch; apologies to anyone who has branched off it.
2. It uses our own spark-project/hive version, where hive-exec only
contains the core hive packages (common, hive-exec, shims) *and a shaded
protobuf*. The latter ensures that it works on the spark hadoop-1 branch.
Longer term we'll work with the hive team to sort out "our differences"
3. if you've done the maven build on or since friday, you'll need to purge
any local maven cached version
rm -rf ~/.m2/repository/org/spark-project/hive/
4. If you've done the sbt build, do the same for its copies.
5. Hint: the following finds all the various directories & jars; `rm` the
directories
find ~ | grep "hive-exec-1.2.1.spark"
6. The sql/hive test `org.apache.spark.sql.hive.ClasspathDependenciesSuite`
looks for the various transitive dependencies that need to be shaded (protobuf)
and those which must not be found shaded (kryo & its transitives). If this test
is failing, the wrong version of something is being pulled in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
