GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/13611
[SPARK-15887][SQL] Bring back the hive-site.xml support for Spark 2.0
## What changes were proposed in this pull request?
Right now, Spark 2.0 does not load hive-site.xml. Based on users' feedback,
it seems make sense to still load this conf file.
This PR adds a `hadoopConf` API in `SharedState`, which is
`sparkContext.hadoopConfiguration` by default. When users are under hive
context, `SharedState.hadoopConf` will load hive-site.xml and append its
configs to `sparkContext.hadoopConfiguration`.
When we need to read hadoop config in spark sql, we should call
`SessionState.newHadoopConf`, which contains
`sparkContext.hadoopConfiguration`, hive-site.xml and sql configs.
## How was this patch tested?
new test in `HiveDataFrameSuite`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark hive-site
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13611.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13611
----
commit 02e7dcb5f6b48cdd368d7d82198f1a79e47e3105
Author: Wenchen Fan <[email protected]>
Date: 2016-06-10T22:12:56Z
Bring back the hive-site.xml support for Spark 2.0
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
