[
https://issues.apache.org/jira/browse/RANGER-4228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
caijialiang reassigned RANGER-4228:
-----------------------------------
Assignee: caijialiang
> Kafka2.8 can't restart after enable ranger plugin
> -------------------------------------------------
>
> Key: RANGER-4228
> URL: https://issues.apache.org/jira/browse/RANGER-4228
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
> Affects Versions: 2.4.0, 2.3.0
> Reporter: caijialiang
> Assignee: caijialiang
> Priority: Major
>
> 1.Missing hadoop shade package
> A temporary solution is to copy the required JAR files from Hadoop
>
> cp /usr/bigtop/current/hadoop-hdfs-client/lib/hadoop-shaded-guava-1.1.1.jar
> /usr/bigtop/current/kafka-broker/libs/
> {code:java}
> [2023-05-06 14:33:22,254] ERROR [KafkaServer id=1001] Fatal error during
> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
> java.lang.NoClassDefFoundError:
> org/apache/hadoop/thirdparty/com/google/common/base/Preconditions
> at
> org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:430)
> at
> org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:443)
> at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:525)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.<init>(RangerBasePlugin.java:79)
> at
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:100)
> at
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:95)
> at
> kafka.security.authorizer.AuthorizerWrapper.configure(AuthorizerWrapper.scala:86)
> at kafka.server.KafkaServer.$anonfun$startup$11(KafkaServer.scala:352)
> at kafka.server.KafkaServer.$anonfun$startup$11$adapted(KafkaServer.scala:352)
> at scala.Option.foreach(Option.scala:407)
> at kafka.server.KafkaServer.startup(KafkaServer.scala:352)
> at kafka.Kafka$.main(Kafka.scala:109)
> at kafka.Kafka.main(Kafka.scala)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.thirdparty.com.google.common.base.Preconditions
> at java.lang.ClassLoader.findClass(ClassLoader.java:530)
> at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader$MyClassLoader.findClass(RangerPluginClassLoader.java:316)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader.loadClass(RangerPluginClassLoader.java:160)
> ... 13 more {code}
> https://issues.apache.org/jira/browse/HADOOP-14284
> Starting from Hadoop 3, Hadoop uses the shaded Guava library located in the
> hadoop-shade jar instead of the original Guava library. However, unlike HBase
> or Hive, Kafka does not add the Hadoop lib to its class path during startup.
> Therefore, in a big data cluster using Hadoop 3, the HBase and Hive plugins
> can start without errors, but Kafka may report errors due to the lack of the
> hadoop-shade jar in its class path. As a temporary solution, the necessary
> jar files can be copied from Hadoop to Kafka.
>
> 2.Missing common-compress package
> A temporary solution is to copy the required JAR files from Hadoop."
> {code:java}
> [2023-05-08 10:15:18,025] ERROR [KafkaServer id=1001] Fatal error during
> KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
> java.lang.NoClassDefFoundError:
> org/apache/commons/compress/archivers/tar/TarArchiveInputStream
> at
> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfoByNonNativeIO(RawLocalFileSystem.java:864)
> at
> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:855)
> at
> org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:816)
> at
> org.apache.hadoop.security.alias.KeyStoreProvider.stashOriginalFilePermissions(KeyStoreProvider.java:78)
> at
> org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:324)
> at
> org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86)
> at
> org.apache.hadoop.security.alias.KeyStoreProvider.<init>(KeyStoreProvider.java:49)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:42)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:35)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:68)
> at
> org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:91)
> at
> org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialProviders(RangerCredentialProvider.java:69)
> at
> org.apache.ranger.authorization.hadoop.utils.RangerCredentialProvider.getCredentialString(RangerCredentialProvider.java:47)
> at
> org.apache.ranger.audit.provider.MiscUtil.getCredentialString(MiscUtil.java:455)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.getKeyManagers(SolrAuditDestination.java:348)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:131)
> at
> org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:96)
> at
> org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:183)
> at
> org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:234)
> at
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:105)
> at
> org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer.configure(RangerKafkaAuthorizer.java:95)
> at
> kafka.security.authorizer.AuthorizerWrapper.configure(AuthorizerWrapper.scala:86)
> at kafka.server.KafkaServer.$anonfun$startup$11(KafkaServer.scala:352)
> at kafka.server.KafkaServer.$anonfun$startup$11$adapted(KafkaServer.scala:352)
> at scala.Option.foreach(Option.scala:407)
> at kafka.server.KafkaServer.startup(KafkaServer.scala:352)
> at kafka.Kafka$.main(Kafka.scala:109)
> at kafka.Kafka.main(Kafka.scala)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.compress.archivers.tar.TarArchiveInputStream
> at java.lang.ClassLoader.findClass(ClassLoader.java:530)
> at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader$MyClassLoader.findClass(RangerPluginClassLoader.java:316)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.ranger.plugin.classloader.RangerPluginClassLoader.loadClass(RangerPluginClassLoader.java:160)
> ... 28 more
> [2023-05-08 10:15:18,028] INFO [KafkaServer id=1001] shutting down
> (kafka.server.KafkaServer) {code}
> cp /usr/bigtop/current/hadoop-hdfs-client/lib/commons-compress-1.21.jar
> /usr/bigtop/current/kafka-broker/libs
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
