Quanlong Huang created IMPALA-13719:
---------------------------------------
Summary: test_hive_with_ranger_setup failed due to
ClassNotFoundException:
org.apache.hadoop.thirdparty.com.google.common.collect.Lists
Key: IMPALA-13719
URL: https://issues.apache.org/jira/browse/IMPALA-13719
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Reporter: Quanlong Huang
Assignee: Quanlong Huang
test_hive_with_ranger_setup fails when running on Ranger 2.5+. Stacktrace:
{code:python}
authorization/test_ranger.py:2532: in test_hive_with_ranger_setup
check_call([script, '-with_ranger'])
/data/jenkins/workspace/impala-cdh_main-core/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/subprocess.py:190:
in check_call
raise CalledProcessError(retcode, cmd)
E CalledProcessError: Command
'['/data/jenkins/workspace/impala-cdh_main-core/repos/Impala/testdata/bin/run-hive-server.sh',
'-with_ranger']' returned non-zero exit status 1{code}
Looking into Hive Metastore logs, there is a java.lang.NoClassDefFoundError on
org/apache/hadoop/thirdparty/com/google/common/collect/Lists and the first
occurence of it shows that the cause is java.lang.ClassNotFoundException on
org.apache.hadoop.thirdparty.com.google.common.collect.Lists
{noformat}
2025-01-31T09:27:44,288 ERROR [TThreadPoolServer WorkerProcess-2]
metastore.RetryingHMSHandler: java.lang.NoClassDefFoundError:
org/apache/hadoop/thirdparty/com/google/common/collect/Lists
at
org.apache.ranger.plugin.model.validation.RangerServiceDefHelper$Delegate.getHierarchies(RangerServiceDefHelper.java:656)
at
org.apache.ranger.plugin.model.validation.RangerServiceDefHelper$Delegate.<init>(RangerServiceDefHelper.java:395)
at
org.apache.ranger.plugin.model.validation.RangerServiceDefHelper.<init>(RangerServiceDefHelper.java:159)
at
org.apache.ranger.plugin.model.validation.RangerServiceDefHelper.<init>(RangerServiceDefHelper.java:129)
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.init(RangerPolicyRepository.java:898)
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:224)
at
org.apache.ranger.plugin.policyengine.RangerPolicyRepository.<init>(RangerPolicyRepository.java:175)
at
org.apache.ranger.plugin.policyengine.PolicyEngine.<init>(PolicyEngine.java:228)
at
org.apache.ranger.plugin.policyengine.RangerPolicyEngineImpl.<init>(RangerPolicyEngineImpl.java:101)
at
org.apache.ranger.plugin.service.RangerBasePlugin.setPolicies(RangerBasePlugin.java:392)
at
org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:257)
at
org.apache.ranger.plugin.util.PolicyRefresher.startRefresher(PolicyRefresher.java:139)
at
org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:260)
at
org.apache.ranger.authorization.hive.authorizer.RangerHivePlugin.init(RangerHiveAuthorizer.java:3475)
at
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.<init>(RangerHiveAuthorizer.java:178)
at
org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizerFactory.createHiveAuthorizer(RangerHiveAuthorizerFactory.java:37)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.HiveMetaStoreAuthorizer.createHiveMetaStoreAuthorizer(HiveMetaStoreAuthorizer.java:530)
at
org.apache.hadoop.hive.ql.security.authorization.plugin.metastore.HiveMetaStoreAuthorizer.onEvent(HiveMetaStoreAuthorizer.java:104)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.firePreEvent(HiveMetaStore.java:4339)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database_req(HiveMetaStore.java:1622)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_database(HiveMetaStore.java:1590)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:160)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:121)
at com.sun.proxy.$Proxy34.get_database(Unknown Source)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_database.getResult(ThriftHiveMetastore.java:18834)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$get_database.getResult(ThriftHiveMetastore.java:18813)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38)
at
org.apache.hadoop.hive.metastore.TUGIBasedProcessor.process(TUGIBasedProcessor.java:104)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.thirdparty.com.google.common.collect.Lists
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 36 more{noformat}
The cause is that Ranger starts to use Guava included in Hadoop since Ranger
2.5 (RANGER-4845).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
