[
https://issues.apache.org/jira/browse/SPARK-24758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670665#comment-16670665
]
Bruce Robbins commented on SPARK-24758:
---------------------------------------
This issue was introduced by commit
[b83b502c4189c571bda776511c6f7541c6067aae|https://github.com/apache/spark/commit/b83b502c4189c571bda776511c6f7541c6067aae].
One can work around the issue by issuing spark-sql as so:
{noformat}
bin/spark-sql --hiveconf
hive.metastore.warehouse.dir=${HOME}/github/spark_clean/spark-warehouse
{noformat}
> Create table wants to use /user/hive/warehouse in clean clone
> -------------------------------------------------------------
>
> Key: SPARK-24758
> URL: https://issues.apache.org/jira/browse/SPARK-24758
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Bruce Robbins
> Priority: Minor
>
> Got a clean clone of repository:
> - git clone [https://github.com/apache/spark.git] spark_clean
> - cd spark_clean/
> - ./build/sbt -Phive -Phive-thriftserver clean package
> Ran spark-sql and tried to create a table:
> - ./bin/spark-sql
> - create table testit as select 1 a, 2 b;
> Got error:
> {noformat}
> 18/07/07 13:33:20 WARN HiveMetaStore: Location:
> file:/user/hive/warehouse/testit specified for non-external table:testit
> 18/07/07 13:33:20 INFO FileUtils: Creating directory if it doesn't exist:
> file:/user/hive/warehouse/testit
> Error in query: org.apache.hadoop.hive.ql.metadata.HiveException:
> MetaException(message:file:/user/hive/warehouse/testit is not a directory or
> unable to create one);
> {noformat}
> To get things working, it seems you need to do something with dataframewriter
> (after removing metastore_db, if it exists):
> {noformat}
> scala> spark.range(0,1).write.saveAsTable("fred")
> spark.range(0,1).write.saveAsTable("fred")
> 18/07/07 14:08:08 WARN ObjectStore: Version information not found in
> metastore. hive.metastore.schema.verification is not enabled so recording the
> schema version 1.2.0
> 18/07/07 14:08:08 WARN ObjectStore: Failed to get database default, returning
> NoSuchObjectException
> scala>
> {noformat}
> After that, create table statements work:
> {noformat}
> spark-sql> create table testit as select 1 a, 2 b;
> create table testit as select 1 a, 2 b;
> 18/07/07 14:14:40 WARN HiveMetaStore: Location:
> file:<work-copy-dir>/spark-warehouse/testit specified for non-external
> table:testit
> 18/07/07 14:14:41 WARN ObjectStore: Failed to get database global_temp,
> returning NoSuchObjectException
> Time taken: 3.387 seconds
> spark-sql> show tables;
> show tables;
> default fred false
> default testit false
> Time taken: 0.07 seconds, Fetched 2 row(s)
> spark-sql>
> {noformat}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
