[
https://issues.apache.org/jira/browse/RANGER-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18093482#comment-18093482
]
Ramachandran Krishnan commented on RANGER-5661:
-----------------------------------------------
Merged into master Commit
details:[https://github.com/apache/ranger/commit/6d9b1b2cf2e9a84eb4eebb2fd71c8672c333e69b]
> Fix Kafka plugin packaging for broker-delegate classloading on Kafka 3.9+
> -------------------------------------------------------------------------
>
> Key: RANGER-5661
> URL: https://issues.apache.org/jira/browse/RANGER-5661
> Project: Ranger
> Issue Type: Task
> Components: Ranger
> Reporter: Ramachandran Krishnan
> Assignee: Ramachandran Krishnan
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> h3. Summary
> {{RangerKafkaAuthorizer}} fails to start on Kafka 3.9.x when the assembled
> {{ranger-kafka-plugin-impl}} ships jars already provided by the broker
> classpath (Jackson, Jersey/HK2, {{{}kafka-clients{}}}, SLF4J/log4j, JAX-RS
> APIs). This causes {{{}LinkageError{}}}, {{{}ClassCastException{}}}, or
> policy REST client failures during broker startup.
> This ticket fixes the *upstream packaging* so the plugin tarball is complete
> and the impl directory uses a *broker-delegate* model: shim + classloader on
> the broker classpath, implementation jars without duplicate broker libraries.
> h3. Problem
> * {{plugin-kafka.xml}} assembly references {{plugin-kafka/template/}} for
> {{{}install/conf.templates/default{}}}, but the template directory was never
> committed — tarball assembly / {{enable-kafka-plugin.sh}} can be incomplete
> (same class of issue as YARN in RANGER-5660).
> * {{ranger-kafka-plugin-impl}} currently bundles dependencies that Kafka 3.9
> already loads (notably Jackson 2.16, Jersey 2.x, HK2, {{{}kafka-clients{}}},
> SLF4J, reload4j/log4j).
> * Without excluding *both* {{javax.ws.rs:javax.ws.rs-api}} and
> {{{}jakarta.ws.rs:jakarta.ws.rs-api{}}}, the plugin classloader can still
> resolve a different JAX-RS API than the broker, causing
> {{ClassCastException}} on Jersey types even when Jackson/Jersey jars are
> excluded.
> h3. Proposed fix (upstream)
> ||File||Change||
> |{{plugin-kafka/template/configuration.xml}}|*Add* empty Hadoop-style
> template so assembly produces
> {{install/conf.templates/default/configuration.xml}}|
> |{{distro/src/main/assembly/plugin-kafka.xml}}|*Exclude* broker-supplied
> artifacts from {{{}ranger-kafka-plugin-impl{}}}: Jackson (*), Jersey (*), HK2
> (*), {{{}kafka-clients{}}}, SLF4J (*), log4j/reload4j (*),
> {{{}javax.ws.rs-api{}}}, {{jakarta.ws.rs-api}}|
> h3. Design notes
> * Kafka uses *broker-delegate* classloading (exclude host jars from impl;
> child-first plugin classloader falls back to broker classpath). This differs
> from HDFS/YARN, which keep Jackson in impl with child-first classloader
> h3. Manual testing
> Tested on Kafka *3.9.1* with Kerberos ({{{}SASL_PLAINTEXT{}}}) and
> {{{}authorizer.class.name=org.apache.ranger.authorization.kafka.authorizer.RangerKafkaAuthorizer{}}}.
> # {{mvn package -Pranger-kafka-plugin}} (distro assembly) completed
> successfully.
> # Assembled {{ranger-*-kafka-plugin.tar.gz}} contains
> {{install/conf.templates/enable/}} and
> {{{}install/conf.templates/default/configuration.xml{}}}.
> # {{ranger-kafka-plugin-impl}} contains *no* jackson, jersey, hk2,
> {{{}kafka-clients{}}}, slf4j, reload4j/log4j, or ws.rs API jars
> (broker-delegated).
> # Kafka broker started cleanly with {{RangerKafkaAuthorizer}} enabled (no
> {{LinkageError}} / {{ClassCastException}} on startup).
> # Authorization: {{testuser2}} console produce to a restricted topic →
> {*}denied{*}; {{testuser1}} with allow policy → {*}allowed{*}.
> h3. Acceptance criteria
> * Kafka plugin tarball includes complete {{conf.templates}} from assembly
> * Clean Kafka 3.9.x broker startup with {{RangerKafkaAuthorizer}}
> * No duplicate broker classpath libraries in {{ranger-kafka-plugin-impl}}
> * Topic publish/consume authorization enforced per Ranger policies
> h2. Related work
> * RANGER-5660 — analogous YARN plugin packaging / {{conf.templates}} /
> enable-path fix
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
