[
https://issues.apache.org/jira/browse/IMPALA-11312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Gaal closed IMPALA-11312.
--------------------------------
Assignee: Gergely Fürnstáhl
Resolution: Cannot Reproduce
This problem is not about Apache Impala, it cannot be reproduced with upstream
code in any environment supported for Apache Impala code.
It is best described and tracked in a downstream environment where the original
problem was encountered.
Also it is good practice to keep internal details off the public Apache Jira
tracker.
> Upgrading metastore db for new hive throws error
> ------------------------------------------------
>
> Key: IMPALA-11312
> URL: https://issues.apache.org/jira/browse/IMPALA-11312
> Project: IMPALA
> Issue Type: Bug
> Reporter: Gergely Fürnstáhl
> Assignee: Gergely Fürnstáhl
> Priority: Minor
>
> I built a new hive package from cdw-master:
> git checkout cdw-master
> ~/bin/cdpd-patcher hive cdw-master
> Create settings file:
> [https://cloudera.atlassian.net/wiki/spaces/ENG/pages/160597896/How+to+build+CDPD-master]
> mvn clean package -Pdist -DskipTests -Dmaven.javadoc.skip=true -s
> hdpsnap.settings.xml
> ~/hive/packaging/target/apache-hive-3.1.3000.2022.0.9.0-18-bin/apache-hive-3.1.3000.2022.0.9.0-18-bin(cdw-master)$
> cp -r ./*
> ~/Impala/toolchain/cdp_components-26065553/apache-hive-3.1.3000.7.2.16.0-33-bin
> Got some errors from hive relating to versioning, tried to upgrade the DB.
> gfurnstahl@gfurnstahl-OptiPlex-7070:~/Impala(IMPALA-11034)$
> bin/create-test-configuration.sh -upgrade_metastore_db
> ...
> Upgrading the schema of metastore db HMS_home_gfurnstahl_Impala_cdp. Check
> /home/gfurnstahl/Impala/logs/cluster/schematool.log for details.
> awk: fatal: cannot open file `bin/create-test-configuration.sh' for reading
> (No such file or directory)
> There were several SQL errors in schematool.log and hive still didn't work
> +-------------------+
> | tab_name |
> +-------------------+
> | hive_warm_up_tbl |
> +-------------------+
> 1 row selected (0.058 seconds)
> 0: jdbc:hive2://localhost:11050/default> select * from hive_warm_up_tbl;
> INFO : Compiling
> command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb):
> select * from hive_warm_up_tbl
> INFO : No Stats for default@hive_warm_up_tbl, Columns: i
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:hive_warm_up_tbl.i, type:int,
> comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb);
> Time taken: 0.066 seconds
> INFO : Executing
> command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb):
> select * from hive_warm_up_tbl
> INFO : Completed executing
> command(queryId=gfurnstahl_20220523150059_32693367-b05a-4c17-895c-d863d4ce07eb);
> Time taken: 0.0 seconds
> INFO : OK
> ERROR : FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.lockmgr.LockException(Error communicating with the
> metastore)
> org.apache.hadoop.hive.ql.lockmgr.LockException: Error communicating with the
> metastore
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:568)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:484)
> at
> org.apache.hadoop.hive.ql.Driver.releaseLocksAndCommitOrRollback(Driver.java:448)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:788)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:526)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:520)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:166)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:232)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:89)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:338)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1899)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:358)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.TApplicationException: Internal error processing
> commit_txn
> at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_commit_txn(ThriftHiveMetastore.java:5995)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.commit_txn(ThriftHiveMetastore.java:5982)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.commitTxn(HiveMetaStoreClient.java:3707)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClientWithLocalCache.commitTxn(HiveMetaStoreClientWithLocalCache.java:944)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:216)
> at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:4022)
> at com.sun.proxy.$Proxy34.commitTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.commitTxn(DbTxnManager.java:559)
> ... 20 more
> *Workaround:* formatting solves the issue
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
