[
https://issues.apache.org/jira/browse/IMPALA-13719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-13719.
-------------------------------------
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> test_hive_with_ranger_setup failed due to ClassNotFoundException:
> org.apache.hadoop.thirdparty.com.google.common.collect.Lists
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-13719
> URL: https://issues.apache.org/jira/browse/IMPALA-13719
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 4.5.0
>
>
> test_hive_with_ranger_setup fails when running on Ranger 2.5+. Stacktrace:
> {code:python}
> authorization/test_ranger.py:2532: in test_hive_with_ranger_setup
> check_call([script, '-with_ranger'])
> /data/jenkins/workspace/impala-cdh_main-core/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/subprocess.py:190:
> in check_call
> raise CalledProcessError(retcode, cmd)
> E CalledProcessError: Command
> '['/data/jenkins/workspace/impala-cdh_main-core/repos/Impala/testdata/bin/run-hive-server.sh',
> '-with_ranger']' returned non-zero exit status 1{code}
> Looking into Hive Metastore logs, there is a java.lang.NoClassDefFoundError
> on org/apache/hadoop/thirdparty/com/google/common/collect/Lists and the first
> occurence of it shows that the cause is java.lang.ClassNotFoundException on
> org.apache.hadoop.thirdparty.com.google.common.collect.Lists
> {noformat}
> 2025-01-31T09:27:44,288 ERROR [TThreadPoolServer WorkerProcess-2]
> metastore.RetryingHMSHandler: java.lang.NoClassDefFoundError:
> org/apache/hadoop/thirdparty/com/google/common/collect/Lists
> at
> org.apache.ranger.plugin.model.validation.RangerServiceDefHelper$Delegate.getHierarchies(RangerServiceDefHelper.java:656)
> at
> org.apache.ranger.plugin.model.validation.RangerServiceDefHelper$Delegate.<init>(RangerServiceDefHelper.java:395)
> at
> org.apache.ranger.plugin.model.validation.RangerServiceDefHelper.<init>(RangerServiceDefHelper.java:159)
> at
> org.apache.ranger.plugin.model.validation.RangerServiceDefHelper.<init>(RangerServiceDefHelper.java:129)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:898)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:224)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:175)
> at
> org.apache.ranger.plugin.policyengine.PolicyEngine.<init>(PolicyEngine.java:228)
> at
> org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.<init>(RangerPolicyEngineImpl.java:101)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:392)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:257)
> at
> org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:139)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:260)
> at
> org.apache.ranger.authorization.hive.authorizer.RangerHivePlugin.init(RangerHiveAuthorizer.java:3475)
> at
> org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.<init>(RangerHiveAuthorizer.java:178)
> at
> org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory.createHiveAuthorizer(RangerHiveAuthorizerFactory.java:37)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.HiveMetaStoreAuthorizer.createHiveMetaStoreAuthorizer(HiveMetaStoreAuthorizer.java:530)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.HiveMetaStoreAuthorizer.onEvent(HiveMetaStoreAuthorizer.java:104)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.firePreEvent(HiveMetaStore.java:4339)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database_req(HiveMetaStore.java:1622)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStore.java:1590)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:160)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:121)
> at com.sun.proxy.$Proxy34.get_database(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_database.getResult(ThriftHiveMetastore.java:18834)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_database.getResult(ThriftHiveMetastore.java:18813)
> at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
> at
> org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
> at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.thirdparty.com.google.common.collect.Lists
> at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
> ... 36 more{noformat}
> The cause is that Ranger starts to use Guava included in Hadoop since Ranger
> 2.5 (RANGER-4845).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
