GitHub user gatorsmile opened a pull request:
https://github.com/apache/spark/pull/13111
[SPARK-15320] [SQL] Spark-SQL Cli Ignores Parameter
hive.metastore.warehouse.dir
#### What changes were proposed in this pull request?
Although overriding `hive.metastore.warehouse.dir` in the spark-sql cli,
the metastore warehouse directory still uses the default value of
`spark.sql.warehouse.dir`. This is a regression. It works in the previous
release.
To reproduce it, we can input the following command in the command line,
```
./bin/spark-sql --hiveconf hive.metastore.warehouse.dir=/Users/xiaoli/a/b
```
However, the log shows the value is overridden by the default value of
`spark.sql.warehouse.dir`.
```
16/05/13 13:43:35 INFO HiveClientImpl: Warehouse location for Hive client
(version 1.2.1) is /Users/xiaoli/IdeaProjects/sparkDelivery/bin/spark-warehouse
```
`CliSuite` also uses `hive.metastore.warehouse.dir` in the Spark-SQL Cli.
This parameter is ineffective. Thus, the whole suite could fail without enough
permission.
https://github.com/apache/spark/blob/890abd1279014d692548c9f3b557483644a0ee32/sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala#L92
This PR is to fix the above issue.
#### How was this patch tested?
Added a test in `CliSuite`
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gatorsmile/spark stopCli
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13111.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13111
----
commit d627a4a6d70b3438f83a6c30c801dc42a033e436
Author: gatorsmile <[email protected]>
Date: 2016-05-14T04:19:56Z
fix
commit f1c5fd74a83676b652fd013081f243ab1fd29f86
Author: gatorsmile <[email protected]>
Date: 2016-05-14T04:29:43Z
revert the changes.
commit 1896e233b26092e8cb848a7b31ed34effdaabbf1
Author: gatorsmile <[email protected]>
Date: 2016-05-14T04:30:36Z
revert the changes.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
