[
https://issues.apache.org/jira/browse/SPARK-10046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981282#comment-14981282
]
Xin Wu commented on SPARK-10046:
--------------------------------
The default HiveConf without hite-site.xml has
"hive.metastore.warehouse.dir=/user/hive/warehouse". and Spark's HiveContext
creates database or tables through Hive Client. So it leaves it to Hive to
decide where to create the database/table directory to.. which is
"/user/hive/warehouse" by default. We can set the desired warehouse location
within spark-shell or spark application, like
{code}
hiveContext.setConf("hive.metastore.warehouse.dir", "/home/spark/warehouse")
{code}
So maybe changing the document to align with the default behavior is an option?
And let user know they may hit directory permission issue with the hive default?
[~yhuai] What do you think? Thanks!
> Hive warehouse dir not set in current directory when not providing
> hive-site.xml
> --------------------------------------------------------------------------------
>
> Key: SPARK-10046
> URL: https://issues.apache.org/jira/browse/SPARK-10046
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, SQL
> Affects Versions: 1.3.1
> Environment: OS X 10.10.4
> Java 1.7.0_79-b15
> Scala 2.10.5
> Spark 1.3.1
> Reporter: Antonio Murgia
> Labels: hive, spark, sparksql
>
> When running spark in local environment (for unit-testing purpose) and
> without providing any `hive-site.xml, databases apart from the default one
> are created in Hive default hive.metastore.warehouse.dir and not in the
> current directory (as stated in [Spark
> docs](http://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables)).
> This code snippet, tested with Spark 1.3.1 demonstrates the issue:
> https://github.com/tmnd1991/spark-hive-bug/blob/master/src/main/scala/Main.scala
> You cane see that the exception is thrown when executing the CREATE DATABASE
> STATEMENT, stating that is `Unable to create database path
> file:/user/hive/warehouse/abc.db, failed to create database abc)` where is
> `/user/hive/warehouse/abc.db`is not the current directory as stated in the
> docs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
