ramackri opened a new pull request, #1020: URL: https://github.com/apache/ranger/pull/1020
## Summary Fixes **Kafka** plugin audit delivery to the **auditserver** ingestor when `XAAUDIT.AUDITSERVER.ENABLE=true` ([RANGER-5642](https://issues.apache.org/jira/browse/RANGER-5642)). **Revert the Kafka portion of [#1015](https://github.com/apache/ranger/pull/1015):** remove duplicate Glassfish Jersey JARs from `plugin-kafka.xml`. The Kafka broker already ships `jersey-client`, `jersey-common`, `jersey-server`, and `jersey-hk2` on the application classpath; packaging the same artifacts into `lib/ranger-kafka-plugin-impl/` causes `RangerPluginClassLoader` SPI skew and audit POST failures. **Keep** in plugin-impl only what the broker lacks: `jersey-entity-filtering`, `jersey-media-json-jackson`, and `ranger-audit-dest-auditserver`. Same principle as Hive [RANGER-5646 / #1019](https://github.com/apache/ranger/commit/9272baf0832007a21982e62be6510c7a1c3e0071) — tighten the assembly whitelist; do not duplicate host-provided libraries. **Scope:** one file — `distro/src/main/assembly/plugin-kafka.xml`. No Java, POM, or Docker changes. > **HBase ([RANGER-5644](https://issues.apache.org/jira/browse/RANGER-5644))** was fixed separately in #1015 (`hbase-agent.xml` adds JSON writer JARs only). Not part of this PR. ## Problem After #1015, `plugin-kafka.xml` whitelisted `jersey-client`, `jersey-common`, `jersey-hk2`, `javax.inject`, and `jakarta.ws.rs-api` into `lib/ranger-kafka-plugin-impl/`. At runtime `RangerPluginClassLoader` merges Jersey SPI from plugin-impl and the broker classpath. Types load from different class loaders: ``` ERROR CRITICAL: Jackson JSON provider is not registered! Classes=[] ERROR Failed to post audit events in privileged action: class org.glassfish.jersey.server.wadl.internal.WadlAutoDiscoverable cannot be cast to class org.glassfish.jersey.internal.spi.AutoDiscoverable ERROR Failed to send audit batch of N events WARN failed to log audit event: {"repo":"dev_kafka",...} ``` | Symptom | Cause | |---------|--------| | `WadlAutoDiscoverable` / `AutoDiscoverable` ClassCastException | Duplicate Jersey JARs in plugin-impl **and** broker `libs/` | | Audits never reach audit ingestor | `RangerAuditServerDestination` REST client fails on first batch | Before #1015, the opposite gap existed (`MessageBodyWriter not found for media type=application/json`) because JSON writer JARs were missing. The broker **does** provide the Jersey client stack; plugin-impl only needs the JSON media modules. ## Changes | File | Change | |------|--------| | `distro/src/main/assembly/plugin-kafka.xml` | **Remove** from `lib/ranger-kafka-plugin-impl` whitelist: `jersey-client`, `jersey-common`, `jersey-hk2`, `javax.inject`, `jakarta.ws.rs-api` | **Unchanged** in plugin-impl whitelist: | Maven coordinate | Role | |------------------|------| | `org.glassfish.jersey.ext:jersey-entity-filtering` | Entity providers | | `org.glassfish.jersey.media:jersey-media-json-jackson` | JSON `MessageBodyWriter` for audit POST | | `ranger-audit-dest-auditserver` | Audit ingestor destination (module JAR) | Broker Jersey is resolved via `RangerPluginClassLoader` parent fallback — single source for client/SPI types. ### Why not copy HBase (#5644) or #1015 Kafka packaging? | Plugin | Host classpath | Plugin-impl Jersey | |--------|----------------|-------------------| | **Kafka** | Full Jersey in `${KAFKA_HOME}/libs/` | JSON writers only — **no** client/common/hk2 | | **HBase** | Partial; no JSON writers | Add `jersey-entity-filtering` + `jersey-media-json-jackson` (#1015) | ## Related - Jira: [RANGER-5642](https://issues.apache.org/jira/browse/RANGER-5642) - Regression: [#1015](https://github.com/apache/ranger/pull/1015) (Kafka Jersey whitelist) - Analogue: [#1019](https://github.com/apache/ranger/commit/9272baf0832007a21982e62be6510c7a1c3e0071) / RANGER-5646 (Hive — remove duplicate JARs) ## Test plan ### Assembly static check - [ ] Confirm `plugin-kafka.xml` lists JSON writers and **does not** list broker Jersey: ```bash grep -E 'jersey-(client|common|hk2)|javax\.inject|jakarta\.ws\.rs-api' distro/src/main/assembly/plugin-kafka.xml \ && echo "UNEXPECTED: broker Jersey must not be whitelisted" || echo "OK" grep -E 'jersey-entity-filtering|jersey-media-json-jackson' distro/src/main/assembly/plugin-kafka.xml ``` ### Rebuild Kafka plugin tarball ```bash mvn package -Pranger-kafka-plugin \ -pl :ranger-kafka-plugin,:ranger-distro -am \ -DskipTests -Dcheckstyle.skip=true -Dpmd.skip=true -Drat.skip=true ``` Output: `target/ranger-*-kafka-plugin.tar.gz` ### Tarball verify - [ ] **Must** be present under `lib/ranger-kafka-plugin-impl/`: - `jersey-entity-filtering-*.jar` - `jersey-media-json-jackson-*.jar` - `ranger-audit-dest-auditserver-*.jar` - [ ] **Must NOT** be present: - `jersey-client-*.jar`, `jersey-common-*.jar`, `jersey-hk2-*.jar`, `javax.inject-*.jar`, `jakarta.ws.rs-api-*.jar` ```bash tar tzf target/ranger-*-kafka-plugin.tar.gz | grep 'plugin-impl/jersey' tar tzf target/ranger-*-kafka-plugin.tar.gz | grep -E 'plugin-impl/(jersey-client|jersey-common|jersey-hk2|javax\.inject)' \ && echo "FAIL: duplicate Jersey in tarball" || echo "OK: no broker Jersey dupes" ``` ### Runtime (optional) - [ ] Redeploy plugin on a Kafka broker with auditserver enabled; confirm **no** `WadlAutoDiscoverable`, `MessageBodyWriter`, or `Failed to send audit batch` in broker logs. - [ ] With `RangerKafkaAuthorizer` enabled, generate access events; confirm `dev_kafka` (or configured repo) audits reach the audit ingestor without HTTP client errors. ### What was tested locally | Step | Result | |------|--------| | Tarball without broker Jersey dupes | **PASS** — audit REST client initializes; no Wadl ClassCastException | | Broker `libs/` provides Jersey client stack | **Confirmed** — `jersey-client`, `jersey-common`, `jersey-server`, `jersey-hk2` present | ## Notes for reviewers - This **reverts** the Kafka half of #1015; it does **not** remove `jersey-entity-filtering` / `jersey-media-json-jackson` (those fix the pre-#1015 `MessageBodyWriter` gap). - **Do not** add `jersey-hk2` back for Kafka — broker already provides it; duplicating it regresses audit delivery. - Customers must **rebuild and redeploy** the Kafka plugin tarball after merge. - HBase RANGER-5644 packaging in #1015 is correct and unchanged by this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
