Niliria commented on issue #4740:
URL: https://github.com/apache/iceberg/issues/4740#issuecomment-1124959842
> Is there anything in the HMS logs for this timeframe?
>
> Also Hive 2.3.9 would be recommended. I think I have seen several issues
with Hive 2.1.0 already (but not specifically this one).
>
> Also if you feel brave enough, you can try Hive 4.0.0-alpha-1, as it has
more comprehensive Iceberg support,
Thank you very much for your reply!
Today I tried again in the current Hive version, but the result still
failed;
I can't find any more error messages. According to the HMS log information,
all I can know is about metaStore connection failure, but if I directly create
a hive table, it is successful, so I don't know what the problem is
The complete log is as follows:
```
2022-05-12T20:45:31,510 INFO [HiveServer2-Handler-Pool: Thread-2423]:
ql.Driver (:()) - Compiling
command(queryId=root_20220512204531_9bce01ff-5a63-44fd-bc37-df442fed368f):
CREATE TABLE test_5(
id INT
,name STRING
)
STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
2022-05-12T20:45:31,510 INFO [HiveServer2-Handler-Pool: Thread-2423]:
parse.CalcitePlanner (:()) - Starting Semantic Analysis
2022-05-12T20:45:31,510 INFO [HiveServer2-Handler-Pool: Thread-2423]:
parse.CalcitePlanner (:()) - Creating table iceberg_test.test_5 position=13
2022-05-12T20:45:31,512 INFO [HiveServer2-Handler-Pool: Thread-2423]:
sqlstd.SQLStdHiveAccessController (:()) - Created SQLStdHiveAccessController
for session context : HiveAuthzSessionContext
[sessionString=d7695262-4414-47af-8c29-58f99da236f3, clientType=HIVESERVER2]
2022-05-12T20:45:31,512 INFO [HiveServer2-Handler-Pool: Thread-2423]:
ql.Driver (:()) - Semantic Analysis Completed
2022-05-12T20:45:31,512 INFO [HiveServer2-Handler-Pool: Thread-2423]:
ql.Driver (:()) - Returning Hive schema: Schema(fieldSchemas:null,
properties:null)
2022-05-12T20:45:31,512 INFO [HiveServer2-Handler-Pool: Thread-2423]:
ql.Driver (:()) - Completed compiling
command(queryId=root_20220512204531_9bce01ff-5a63-44fd-bc37-df442fed368f); Time
taken: 0.002 seconds
2022-05-12T20:45:31,516 INFO [d7695262-4414-47af-8c29-58f99da236f3
HiveServer2-Handler-Pool: Thread-2423]: conf.HiveConf
(HiveConf.java:getLogIdVar(3511)) - Using the default value passed in for log
id: d7695262-4414-47af-8c29-58f99da236f3
2022-05-12T20:45:31,516 INFO [HiveServer2-Handler-Pool: Thread-2423]:
session.SessionState (:()) - Resetting thread name to
HiveServer2-Handler-Pool: Thread-2423
2022-05-12T20:45:31,522 INFO [HiveServer2-Background-Pool: Thread-7328]:
ql.Driver (:()) - Concurrency mode is disabled, not creating a lock manager
2022-05-12T20:45:31,522 INFO [HiveServer2-Background-Pool: Thread-7328]:
ql.Driver (:()) - Executing
command(queryId=root_20220512204531_9bce01ff-5a63-44fd-bc37-df442fed368f):
CREATE TABLE test_5(
id INT
,name STRING
)
STORED BY 'org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
2022-05-12T20:45:31,522 INFO [HiveServer2-Background-Pool: Thread-7328]:
ql.Driver (:()) - Starting task [Stage-0:DDL] in serial mode
2022-05-12T20:45:31,522 INFO [HiveServer2-Background-Pool: Thread-7328]:
plan.CreateTableDesc (:()) - Use StorageHandler-supplied
org.apache.iceberg.mr.hive.HiveIcebergSerDe for table test_5
2022-05-12T20:45:31,523 INFO [HiveServer2-Background-Pool: Thread-7328]:
exec.DDLTask (:()) - creating table iceberg_test.test_5 on null
2022-05-12T20:45:31,637 WARN [HiveServer2-Background-Pool: Thread-7328]:
conf.HiveConf (HiveConf.java:initialize(3683)) - HiveConf of name
hive.metastore.event.db.notification.api.auth does not exist
2022-05-12T20:45:31,637 WARN [HiveServer2-Background-Pool: Thread-7328]:
conf.HiveConf (HiveConf.java:initialize(3683)) - HiveConf of name
hive.internal.ss.authz.settings.applied.marker does not exist
2022-05-12T20:45:31,653 INFO [ForkJoinPool.commonPool-worker-1]:
hive.metastore (:()) - Closed a connection to metastore, current connections: 4
2022-05-12T20:45:31,679 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Trying to connect to metastore with URI
thrift://master:9083
2022-05-12T20:45:31,679 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Opened a connection to metastore, current connections: 5
2022-05-12T20:45:31,686 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Connected to metastore.
2022-05-12T20:45:31,691 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.HiveIcebergSerDe (:()) - Using hive schema
{"type":"struct","schema-id":0,"fields":[{"id":0,"name":"id","required":false,"type":"int"},{"id":1,"name":"name","required":false,"type":"string"}]}
2022-05-12T20:45:31,700 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.HiveIcebergSerDe (:()) - Using hive schema
{"type":"struct","schema-id":0,"fields":[{"id":0,"name":"id","required":false,"type":"int"},{"id":1,"name":"name","required":false,"type":"string"}]}
2022-05-12T20:45:31,795 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Closed a connection to metastore, current connections: 4
2022-05-12T20:45:31,795 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Trying to connect to metastore with URI
thrift://master:9083
2022-05-12T20:45:31,795 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Opened a connection to metastore, current connections: 5
2022-05-12T20:45:31,804 INFO [HiveServer2-Background-Pool: Thread-7328]:
hive.metastore (:()) - Connected to metastore.
2022-05-12T20:45:31,807 ERROR [HiveServer2-Background-Pool: Thread-7328]:
exec.DDLTask (:()) - org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException: Metastore operation failed for iceberg_test.test_5
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:840)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:845)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3979)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:333)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235)
at
org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90)
at
org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.RuntimeException: Metastore operation failed for
iceberg_test.test_5
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:284)
at
org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:126)
at
org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:216)
at
org.apache.iceberg.BaseMetastoreCatalog.createTable(BaseMetastoreCatalog.java:49)
at org.apache.iceberg.mr.Catalogs.createTable(Catalogs.java:165)
at
org.apache.iceberg.mr.hive.HiveIcebergMetaHook.commitCreateTable(HiveIcebergMetaHook.java:147)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:738)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:724)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
at com.sun.proxy.$Proxy22.createTable(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2254)
at com.sun.proxy.$Proxy22.createTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:830)
... 22 more
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:4549)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:4536)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:2104)
at
org.apache.iceberg.hive.HiveTableOperations.lambda$acquireLock$7(HiveTableOperations.java:413)
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:62)
at org.apache.iceberg.hive.CachedClientPool.run(CachedClientPool.java:76)
at
org.apache.iceberg.hive.HiveTableOperations.acquireLock(HiveTableOperations.java:413)
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:217)
... 42 more
2022-05-12T20:45:31,807 ERROR [HiveServer2-Background-Pool: Thread-7328]:
ql.Driver (:()) - FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Metastore
operation failed for iceberg_test.test_5
2022-05-12T20:45:31,807 INFO [HiveServer2-Background-Pool: Thread-7328]:
ql.Driver (:()) - Completed executing
command(queryId=root_20220512204531_9bce01ff-5a63-44fd-bc37-df442fed368f); Time
taken: 0.285 seconds
2022-05-12T20:45:31,808 ERROR [HiveServer2-Background-Pool: Thread-7328]:
operation.Operation (:()) - Error running hive query:
org.apache.hive.service.cli.HiveSQLException: Error while processing
statement: FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. java.lang.RuntimeException: Metastore
operation failed for iceberg_test.test_5
at
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:387)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:237)
at
org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90)
at
org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at
org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.RuntimeException: Metastore operation failed for iceberg_test.test_5
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:840)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:845)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3979)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:333)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077)
at
org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235)
... 11 more
Caused by: java.lang.RuntimeException: Metastore operation failed for
iceberg_test.test_5
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:284)
at
org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:126)
at
org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:216)
at
org.apache.iceberg.BaseMetastoreCatalog.createTable(BaseMetastoreCatalog.java:49)
at org.apache.iceberg.mr.Catalogs.createTable(Catalogs.java:165)
at
org.apache.iceberg.mr.hive.HiveIcebergMetaHook.commitCreateTable(HiveIcebergMetaHook.java:147)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:738)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:724)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:154)
at com.sun.proxy.$Proxy22.createTable(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:2254)
at com.sun.proxy.$Proxy22.createTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:830)
... 22 more
Caused by: org.apache.thrift.transport.TTransportException
at
org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at
org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at
org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at
org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_lock(ThriftHiveMetastore.java:4549)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.lock(ThriftHiveMetastore.java:4536)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.lock(HiveMetaStoreClient.java:2104)
at
org.apache.iceberg.hive.HiveTableOperations.lambda$acquireLock$7(HiveTableOperations.java:413)
at org.apache.iceberg.ClientPoolImpl.run(ClientPoolImpl.java:62)
at org.apache.iceberg.hive.CachedClientPool.run(CachedClientPool.java:76)
at
org.apache.iceberg.hive.HiveTableOperations.acquireLock(HiveTableOperations.java:413)
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:217)
... 42 more
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
