GitHub user steveloughran opened a pull request:
https://github.com/apache/spark/pull/6748
WiP: SPARK-8064 Upgrade to Hive 1.2
This is work in progress branch to add Hive 1.2.0 support.
Current status: it compiles, initial instantiations don't throw up obscure
linkage errors. Tests fail fast on ClientWrapper linking
## Build
Some of the POM changes are probably just superstition: I had them in from
my original Spark 1.3 + Hive 1.2 build and haven't gone through to see which
explicit includes/excludes are needed now that
all the spark dependencies have changed.
## Compile
in `org.apache.spark.sql.hive.client.ClientWrapper`; `loadPartition`,
`loadTable` and `loadDynamicPartitions`: need to choose suitable defaults
for the new options, or add them to the signature
`org.apache.spark.sql.hive.client.ClientWrapper.reset()`: should the
`deleteData`
flag be set?
## Kryo versions
HiveShim now has pasted in some of Hive Utilities to ser/deser using
the Spark Kryo version, not Hive's
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/steveloughran/spark
stevel/feature/SPARK-8064-Hive
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6748.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6748
----
commit fb8a1d29c63c853398ec4a1251b0ae01d305d0ea
Author: Steve Loughran <ste...@hortonworks.com>
Date: 2015-06-10T13:25:46Z
SPARK-8064 Upgrade Hive to 1.2. Preamble from initial prototype:
improvements to test setup/teardown & exception reporting.
commit aaba4b9d955c3aea3847396df8d17402aeaa6dea
Author: Steve Loughran <ste...@hortonworks.com>
Date: 2015-06-10T15:53:11Z
SPARK-8064 POM changes to move to 1.2
commit 548f9a2a89f0f4b15ee92c0685bb3fe0051a81d4
Author: Steve Loughran <ste...@hortonworks.com>
Date: 2015-06-10T15:55:05Z
SPARK-8064 changes to make compile work again
commit 90da52ff1c977c067f4699bbfb9ae7e47fce1ba8
Author: Steve Loughran <ste...@hortonworks.com>
Date: 2015-06-10T17:46:54Z
SPARK-8064 POM changes to fix up jackson fasterxml versions consistently
commit 15780c0a6fc79a026c071fe6c5d4cfd79d58727d
Author: Steve Loughran <ste...@hortonworks.com>
Date: 2015-06-10T17:47:59Z
SPARK-8064 source changes for build
commit 7aa2a6da78ae1dac64885d80becdd92e542d8761
Author: Steve Loughran <ste...@hortonworks.com>
Date: 2015-06-10T17:49:10Z
SPARK-8064 minor test source changes; new Java test
(LibraryDependenciesSuite) to fail fast on linkage problems
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
