[
https://issues.apache.org/jira/browse/HIVE-28004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17807664#comment-17807664
]
László Végh commented on HIVE-28004:
------------------------------------
[~zhangbutao] This will be addressed as a part of this PR:
[https://github.com/apache/hive/pull/5010]
There are multiple issues found with after merging
https://issues.apache.org/jira/browse/HIVE-27481
Unfortunately the unit and qtets tests are only running on derby HMS backend,
which did not reveal them. The addendum will try to cover all of these, by
running the qtests manually on a PostgreSQL backend and fix the upcoming issues.
> DELETE on ACID table failed with NoClassDefFoundError:
> com/sun/tools/javac/util/List
> ------------------------------------------------------------------------------------
>
> Key: HIVE-28004
> URL: https://issues.apache.org/jira/browse/HIVE-28004
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 4.0.0-beta-1
> Reporter: Butao Zhang
> Assignee: László Végh
> Priority: Blocker
>
> I am not sure if it is a bug or usage question.
> Test on Hive master branch:
>
> {code:java}
> set hive.txn.manager = org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> set hive.support.concurrency = true;
> create table testacid4(id int) stored as orc TBLPROPERTIES
> ('transactional'='true');
> delete from testacid4 where id=110;
> {code}
>
> *beeline console shows error：*
> {code:java}
> 0: jdbc:hive2://127.0.0.1:10000/default> delete from testacid4 where id=110;
> INFO : Compiling
> command(queryId=hive_20240116180628_ec5ac4d8-473b-4b42-b0dd-eecebec71268):
> delete from testacid4 where id=110
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Created Hive schema: Schema(fieldSchemas:[FieldSchema(name:row__id,
> type:struct<writeid:bigint,bucketid:int,rowid:bigint>, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20240116180628_ec5ac4d8-473b-4b42-b0dd-eecebec71268);
> Time taken: 3.554 seconds
> INFO : Operation QUERY obtained 1 locks
> ERROR : FAILED: Hive Internal Error:
> org.apache.hadoop.hive.ql.lockmgr.LockException(org.apache.thrift.TApplicationException:
> Internal error processing get_latest_txnid_in_conflict)
> org.apache.hadoop.hive.ql.lockmgr.LockException:
> org.apache.thrift.TApplicationException: Internal error processing
> get_latest_txnid_in_conflict
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getLatestTxnIdInConflict(DbTxnManager.java:1055)
> at
> org.apache.hadoop.hive.ql.DriverTxnHandler.isValidTxnListState(DriverTxnHandler.java:435)
> at org.apache.hadoop.hive.ql.Driver.validateTxnList(Driver.java:250)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:199)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:154)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:149)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:185)
> at
> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:236)
> at
> org.apache.hive.service.cli.operation.SQLOperation.access$500(SQLOperation.java:90)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:336)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> at
> org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:356)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.thrift.TApplicationException: Internal error processing
> get_latest_txnid_in_conflict
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_latest_txnid_in_conflict(ThriftHiveMetastore.java:6404)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_latest_txnid_in_conflict(ThriftHiveMetastore.java:6391)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getLatestTxnIdInConflict(HiveMetaStoreClient.java:4421)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:213)
> at com.sun.proxy.$Proxy32.getLatestTxnIdInConflict(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient$SynchronizedHandler.invoke(HiveMetaStoreClient.java:4541)
> at com.sun.proxy.$Proxy32.getLatestTxnIdInConflict(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.getLatestTxnIdInConflict(DbTxnManager.java:1053)
> ... 18 more
> {code}
> *Error from metastore:*
> {code:java}
> 2024-01-16T18:06:32,118 ERROR [Metastore-Handler-Pool: Thread-59]
> metastore.RetryingHMSHandler: java.lang.RuntimeException:
> java.lang.NoClassDefFoundError: com/sun/tools/javac/util/List
> at
> org.apache.hadoop.hive.metastore.txn.TransactionalRetryProxy.lambda$invoke$6(TransactionalRetryProxy.java:182)
> at
> org.apache.hadoop.hive.metastore.txn.retry.SqlRetryHandler.executeWithRetryInternal(SqlRetryHandler.java:142)
> at
> org.apache.hadoop.hive.metastore.txn.retry.SqlRetryHandler.executeWithRetry(SqlRetryHandler.java:128)
> at
> org.apache.hadoop.hive.metastore.txn.TransactionalRetryProxy.invoke(TransactionalRetryProxy.java:186)
> at com.sun.proxy.$Proxy30.getLatestTxnIdInConflict(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HMSHandler.get_latest_txnid_in_conflict(HMSHandler.java:8783)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:91)
> at
> org.apache.hadoop.hive.metastore.AbstractHMSHandlerProxy.invoke(AbstractHMSHandlerProxy.java:82)
> at com.sun.proxy.$Proxy28.get_latest_txnid_in_conflict(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_latest_txnid_in_conflict.getResult(ThriftHiveMetastore.java:24190)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_latest_txnid_in_conflict.getResult(ThriftHiveMetastore.java:24169)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:111)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor$1.run(TUGIBasedProcessor.java:107)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:119)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NoClassDefFoundError: com/sun/tools/javac/util/List
> at
> org.apache.hadoop.hive.metastore.txn.jdbc.queries.LatestTxnIdInConflictHandler.getQueryParameters(LatestTxnIdInConflictHandler.java:70)
> at
> org.apache.hadoop.hive.metastore.txn.jdbc.MultiDataSourceJdbcResource.execute(MultiDataSourceJdbcResource.java:328)
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.getLatestTxnIdInConflict(TxnHandler.java:595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.txn.TransactionalRetryProxy.lambda$invoke$0(TransactionalRetryProxy.java:100)
> at
> org.apache.hadoop.hive.metastore.txn.TransactionalRetryProxy.lambda$invoke$5(TransactionalRetryProxy.java:114)
> at
> org.apache.hadoop.hive.metastore.txn.TransactionalRetryProxy.lambda$invoke$6(TransactionalRetryProxy.java:163)
> ... 25 more
> Caused by: java.lang.ClassNotFoundException: com.sun.tools.javac.util.List
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 35 more
> {code}
>
> But if i put the jdk lib *tools.jar* into hive lib, this will be fixed. Is
> there any way to let hms load the tools.jar from jdk path? I am sure i have
> configured the jdk home correctly.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
