Ramachandran Krishnan created RANGER-5661:
---------------------------------------------
Summary: Fix Kafka plugin packaging for broker-delegate
classloading on Kafka 3.9+
Key: RANGER-5661
URL: https://issues.apache.org/jira/browse/RANGER-5661
Project: Ranger
Issue Type: Task
Components: Ranger
Reporter: Ramachandran Krishnan
Assignee: Ramachandran Krishnan
Fix For: 3.0.0
h3. Summary
{{RangerKafkaAuthorizer}} fails to start on Kafka 3.9.x when the assembled
{{ranger-kafka-plugin-impl}} ships jars already provided by the broker
classpath (Jackson, Jersey/HK2, {{{}kafka-clients{}}}, SLF4J/log4j, JAX-RS
APIs). This causes {{{}LinkageError{}}}, {{{}ClassCastException{}}}, or policy
REST client failures during broker startup.
This ticket fixes the *upstream packaging* so the plugin tarball is complete
and the impl directory uses a *broker-delegate* model: shim + classloader on
the broker classpath, implementation jars without duplicate broker libraries.
h3. Problem
* {{plugin-kafka.xml}} assembly references {{plugin-kafka/template/}} for
{{{}install/conf.templates/default{}}}, but the template directory was never
committed — tarball assembly / {{enable-kafka-plugin.sh}} can be incomplete
(same class of issue as YARN in RANGER-5660).
* {{ranger-kafka-plugin-impl}} currently bundles dependencies that Kafka 3.9
already loads (notably Jackson 2.16, Jersey 2.x, HK2, {{{}kafka-clients{}}},
SLF4J, reload4j/log4j).
* Without excluding *both* {{javax.ws.rs:javax.ws.rs-api}} and
{{{}jakarta.ws.rs:jakarta.ws.rs-api{}}}, the plugin classloader can still
resolve a different JAX-RS API than the broker, causing {{ClassCastException}}
on Jersey types even when Jackson/Jersey jars are excluded.
h3. Proposed fix (upstream)
||File||Change||
|{{plugin-kafka/template/configuration.xml}}|*Add* empty Hadoop-style template
so assembly produces {{install/conf.templates/default/configuration.xml}}|
|{{distro/src/main/assembly/plugin-kafka.xml}}|*Exclude* broker-supplied
artifacts from {{{}ranger-kafka-plugin-impl{}}}: Jackson (*), Jersey (*), HK2
(*), {{{}kafka-clients{}}}, SLF4J (*), log4j/reload4j (*),
{{{}javax.ws.rs-api{}}}, {{jakarta.ws.rs-api}}|
h3. Design notes
* Kafka uses *broker-delegate* classloading (exclude host jars from impl;
child-first plugin classloader falls back to broker classpath). This differs
from HDFS/YARN, which keep Jackson in impl with child-first classloader
h3. Manual testing
Tested on Kafka *3.9.1* with Kerberos ({{{}SASL_PLAINTEXT{}}}) and
{{{}authorizer.class.name=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer{}}}.
# {{mvn package -Pranger-kafka-plugin}} (distro assembly) completed
successfully.
# Assembled {{ranger-*-kafka-plugin.tar.gz}} contains
{{install/conf.templates/enable/}} and
{{{}install/conf.templates/default/configuration.xml{}}}.
# {{ranger-kafka-plugin-impl}} contains *no* jackson, jersey, hk2,
{{{}kafka-clients{}}}, slf4j, reload4j/log4j, or ws.rs API jars
(broker-delegated).
# Kafka broker started cleanly with {{RangerKafkaAuthorizer}} enabled (no
{{LinkageError}} / {{ClassCastException}} on startup).
# Authorization: {{testuser2}} console produce to a restricted topic →
{*}denied{*}; {{testuser1}} with allow policy → {*}allowed{*}.
h3. Acceptance criteria
* Kafka plugin tarball includes complete {{conf.templates}} from assembly
* Clean Kafka 3.9.x broker startup with {{RangerKafkaAuthorizer}}
* No duplicate broker classpath libraries in {{ranger-kafka-plugin-impl}}
* Topic publish/consume authorization enforced per Ranger policies
h2. Related work
* RANGER-5660 — analogous YARN plugin packaging / {{conf.templates}} /
enable-path fix
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
