[ 
https://issues.apache.org/jira/browse/HIVE-23726?focusedWorklogId=453496&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-453496
 ]

ASF GitHub Bot logged work on HIVE-23726:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 01/Jul/20 16:16
            Start Date: 01/Jul/20 16:16
    Worklog Time Spent: 10m 
      Work Description: nrg4878 opened a new pull request #1198:
URL: https://github.com/apache/hive/pull/1198


   …ll with colocation enabled (Naveen Gangam)
   
   ## NOTICE
   
   Please create an issue in ASF JIRA before opening a pull request,
   and you need to set the title of the pull request which starts with
   the corresponding JIRA issue number. (e.g. HIVE-XXXXX: Fix a typo in YYY)
   For more details, please see 
https://cwiki.apache.org/confluence/display/Hive/HowToContribute
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 453496)
    Remaining Estimate: 0h
            Time Spent: 10m

> Create table may throw 
> MetaException(message:java.lang.IllegalArgumentException: Can not create a 
> Path from a null string)
> --------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-23726
>                 URL: https://issues.apache.org/jira/browse/HIVE-23726
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Istvan Fajth
>            Assignee: Naveen Gangam
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> - Given:
>  metastore.warehouse.tenant.colocation is set to true
>  a test database was created as {{create database test location '/data'}}
>  - When:
>  I try to create a table as {{create table t1 (a int) location '/data/t1'}}
>  - Then:
> The create table fails with the following exception:
> {code}
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> MetaException(message:java.lang.IllegalArgumentException: Can not create a 
> Path from a null string)
>       at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1138)
>       at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1143)
>       at 
> org.apache.hadoop.hive.ql.ddl.table.create.CreateTableOperation.createTableNonReplaceMode(CreateTableOperation.java:148)
>       at 
> org.apache.hadoop.hive.ql.ddl.table.create.CreateTableOperation.execute(CreateTableOperation.java:98)
>       at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:80)
>       at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
>       at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) 
>       at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359)
>       at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330)
>       at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
>       at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
>       at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:721)
>       at org.apache.hadoop.hive.ql.Driver.run(Driver.java:488)
>       at org.apache.hadoop.hive.ql.Driver.run(Driver.java:482)
>       at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:422)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:340)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>       at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: 
> java.lang.IllegalArgumentException: Can not create a Path from a null string
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_req_result$create_table_req_resultStandardScheme.read(ThriftHiveMetastore.java:63325)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_req_result$create_table_req_resultStandardScheme.read(ThriftHiveMetastore.java:63293)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_req_result.read(ThriftHiveMetastore.java:63219)
>       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table_req(ThriftHiveMetastore.java:1780)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table_req(ThriftHiveMetastore.java:1767)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:3518)
>       at 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.create_table_with_environment_context(SessionHiveMetaStoreClient.java:145)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:1052)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:1037)
>       at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:498)
>       at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
>       at com.sun.proxy.$Proxy35.createTable(Unknown Source)
>       at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:498)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:3411)
>       at com.sun.proxy.$Proxy35.createTable(Unknown Source)
>       at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1127)
>       ... 28 more
> {code}
> Addtional info:
> - The exception does not occur if I don't specify the location in the create 
> table statement
> - The exception does not occur if I specify the managedlocation as well in 
> the create database statement.
> - If I do not specify any location for the database then I can not point the 
> external table to an arbitrary directory and can not use external files to 
> load data into a managed database via Hive easily this is especially a 
> problem if I have an external data source on something else then HDFS, and 
> want to pull the data into Hive.
> The exception thrown when I try to create the external table specifying an 
> arbitrary location in a database created as {{create database test2}}:
> {code}
> org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:An 
> external table's location should not be located outside the location 
> specified on its database, table:t1,location:/data/test2/t1,Database location 
> for external tables:/warehouse/tablespace/external/hive/test2.db)
>       at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1138)
>       at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1143)
>       at 
> org.apache.hadoop.hive.ql.ddl.table.create.CreateTableOperation.createTableNonReplaceMode(CreateTableOperation.java:148)
>       at 
> org.apache.hadoop.hive.ql.ddl.table.create.CreateTableOperation.execute(CreateTableOperation.java:98)
>       at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:80)
>       at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213)
>       at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) 
>       at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:359)
>       at org.apache.hadoop.hive.ql.Executor.launchTasks(Executor.java:330)
>       at org.apache.hadoop.hive.ql.Executor.runTasks(Executor.java:246)
>       at org.apache.hadoop.hive.ql.Executor.execute(Executor.java:109)
>       at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:721)
>       at org.apache.hadoop.hive.ql.Driver.run(Driver.java:488)
>       at org.apache.hadoop.hive.ql.Driver.run(Driver.java:482)
>       at 
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:225)
>  
>       at 
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
>  
>       at 
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:422)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
>       at 
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:340)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>       at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.hadoop.hive.metastore.api.MetaException: An external 
> table's location should not be located outside the location specified on its 
> database, table:t1,location:/data/test2/t1,Database location for external 
> tables:/warehouse/tablespace/external/hive/test2.db
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_req_result$create_table_req_resultStandardScheme.read(ThriftHiveMetastore.java:63325)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_req_result$create_table_req_resultStandardScheme.read(ThriftHiveMetastore.java:63293)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$create_table_req_result.read(ThriftHiveMetastore.java:63219)
>       at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_create_table_req(ThriftHiveMetastore.java:1780)
>       at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.create_table_req(ThriftHiveMetastore.java:1767)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:3518)
>       at 
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.create_table_with_environment_context(SessionHiveMetaStoreClient.java:145)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:1052)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:1037)
>       at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:498)
>       at 
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:212)
>       at com.sun.proxy.$Proxy35.createTable(Unknown Source)
>       at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:498)
>       at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:3411)
>       at com.sun.proxy.$Proxy35.createTable(Unknown Source)
>       at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:1127)
> {code}
> - The describe database shows the following output for the test database:
> {code}
> +----------+----------+--------------------------------------------------+------------------+-------------+-------------+-------------+
> | db_name  | comment  |                     location                     | 
> managedlocation  | owner_name  | owner_type  | parameters  |
> +----------+----------+--------------------------------------------------+------------------+-------------+-------------+-------------+
> | test     |          | hdfs://<nn_host>:<nn_port>/data  |                  | 
> hive        | USER        |             |
> +----------+----------+--------------------------------------------------+------------------+-------------+-------------+-------------+
> {code}
> Notice that managed location for the database is empty.
> - The code where the exception occurs is [the following 
> line|https://github.com/apache/hive/blob/7cdf362f9f1a53025b371a8d5122336ded71d692/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDefaultTransformer.java#L802]:
> {code}
> dbLocation = Path.getPathWithoutSchemeAndAuthority(new 
> Path(db.getManagedLocationUri()));
> {code}
> it seems that the db.getManagedLocationUri() gives back a null reference for 
> the test table. Metastore debug log does not contain the managed location, so 
> I can not confirm this based on the logs, it would be nice as part of the fix 
> to add the managedlocation used for the path validation of the table as well 
> to the [debug log 
> here|https://github.com/apache/hive/blob/7cdf362f9f1a53025b371a8d5122336ded71d692/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/MetastoreDefaultTransformer.java#L792]
>  also.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to