ramackri opened a new pull request, #1025: URL: https://github.com/apache/ranger/pull/1025
## Summary Fixes **KMS plugin** audit delivery to the **audit-server / audit ingestor** when `xasecure.audit.destination.auditserver=true` ([RANGER-5650](https://issues.apache.org/jira/browse/RANGER-5650)). The KMS Ranger plugin runs in `RangerPluginClassLoader` under the KMS Tomcat WAR (`ranger-kms-plugin-impl`). Without a self-contained Jackson + Jersey **client** stack in plugin-impl, audit batch POSTs fail (`JsonUtilsV2` / `LinkageError`, Jersey SPI `ClassCastException`, HK2 injection errors). This follows the **HBase / RANGER-5644** pattern (**add** client JARs to plugin-impl), **not** the Kafka / RANGER-5642 pattern (remove duplicates from host classpath). Duplicating Jackson/Jersey in both `WEB-INF/lib` and `plugin-impl` is intentional — the isolated plugin classloader must own its audit REST client copies. **Scope:** one file — `distro/src/main/assembly/kms.xml`. No Java, POM, or Docker changes. > **Standalone kms-plugin tarball** (`plugin-kms.xml` / `enable-kms-plugin.sh`) is out of scope for this PR. Built-in and Docker KMS use the **KMS WAR** assembly from `kms.xml`. ## Problem When audit-server destination is enabled on KMS, the plugin cannot serialize and POST audit batches: | Symptom | Cause | |---------|--------| | `LinkageError` / `NoClassDefFoundError: JsonUtilsV2` | `ObjectMapper` loaded by both WAR and plugin classloaders | | `WadlAutoDiscoverable` / `AutoDiscoverable` ClassCastException | Jersey SPI split across classloaders | | `MultiException` / HK2 injection failures | Missing hk2 JARs in plugin-impl | | Audits never reach audit ingestor | `RangerAuditServerDestination` REST client fails on first batch | ## Changes | File | Change | |------|--------| | `distro/src/main/assembly/kms.xml` | Whitelist audit client dependencies in `ranger-kms-plugin-impl` dependency set | **Added to plugin-impl:** | Category | Maven coordinates | |----------|-------------------| | Jackson | `jackson-annotations`, `jackson-core`, `jackson-databind`, `jackson-jaxrs-base`, `jackson-jaxrs-json-provider`, `jackson-module-jaxb-annotations` | | JAX-RS / inject | `jakarta.ws.rs-api`, `javax.inject` | | Jersey | `jersey-client`, `jersey-common`, `jersey-entity-filtering`, `jersey-media-json-jackson`, `jersey-hk2`, **`jersey-server`** | | HK2 | `hk2-api`, `hk2-locator`, `hk2-utils`, `hk2`, `hk2-core` | **Explicitly not added:** * `jersey-container-*` — causes KMS startup/runtime failures when placed in plugin-impl ## Related * Jira: [RANGER-5650](https://issues.apache.org/jira/browse/RANGER-5650) * Closest analogue: [RANGER-5644](https://issues.apache.org/jira/browse/RANGER-5644) / [#1015](https://github.com/apache/ranger/pull/1015) (HBase — add JSON writers + client stack to plugin-impl) * Contrast: [RANGER-5642](https://issues.apache.org/jira/browse/RANGER-5642) / [#1020](https://github.com/apache/ranger/commit/6bf19137341fcbe1cbee1e7cffc2f341ac78f029) (Kafka — remove dupes; inverse of KMS) ## Test plan ### Assembly static check - [ ] Confirm `kms.xml` lists Jackson, Jersey client, and hk2 under `ranger-kms-plugin-impl`: ```bash grep -E 'jackson-|jersey-client|jersey-common|jersey-server|jersey-hk2|hk2-' distro/src/main/assembly/kms.xml ``` - [ ] Confirm **no** `jersey-container` entries in plugin-impl whitelist ### Rebuild KMS tarball ```bash mvn package -Pranger-kms -pl '!plugin-kylin,!ranger-kylin-plugin-shim' -DskipTests -Dcheckstyle.skip=true -Dpmd.skip=true -Drat.skip=true ``` Output: `target/ranger-*-kms.tar.gz` ### Tarball verify - [ ] **Must** be present under `ews/webapp/WEB-INF/lib/ranger-kms-plugin-impl/`: - `jackson-*-*.jar` (6 artifacts) - `jersey-client-*.jar`, `jersey-common-*.jar`, `jersey-entity-filtering-*.jar`, `jersey-media-json-jackson-*.jar`, `jersey-hk2-*.jar`, `jersey-server-*.jar` - `hk2-*.jar`, `hk2-api-*.jar`, `hk2-core-*.jar`, `hk2-locator-*.jar`, `hk2-utils-*.jar` - `ranger-audit-dest-auditserver-*.jar` - [ ] **Must NOT** be present: - `jersey-container-*.jar` ```bash tar tzf target/ranger-*-kms.tar.gz | grep 'plugin-impl' | grep -E 'jackson|jersey|hk2' | sort tar tzf target/ranger-*-kms.tar.gz | grep 'plugin-impl/jersey-container' && echo "FAIL" || echo "OK: no jersey-container in plugin-impl" ``` ### Manual testing (generic) 1. Build and deploy the KMS tarball (or rebuild the `ranger-kms` Docker image) with audit-server destination enabled in `ranger-kms-audit.xml`. 2. Ensure the audit ingestor (and Kafka/Solr dispatch path if used) is running and reachable from KMS over the configured network. 3. Configure Kerberos or other auth as required for plugin → ingestor POSTs. 4. Restart KMS after deploy so the updated `ranger-kms-plugin-impl` classpath is loaded. 5. Exercise KMS key operations as an authorized user (e.g. list keys) and as a denied user to generate allow/deny audit events. 6. Confirm KMS logs show **no** `LinkageError`, `JsonUtilsV2`, `MessageBodyWriter`, or `Failed to send audit batch` errors attributable to missing Jackson/Jersey in plugin-impl. 7. Confirm audit events for the KMS repository appear in the configured audit store (e.g. Solr `repo:<kms_service_name>` count increases). 8. In Ranger Admin **Audit → Access**, verify events for the KMS service (note: KMS audits may require **keyadmin** login — `admin` excludes KMS repoType by design). ### What was tested locally | Step | Result | |------|--------| | KMS tarball rebuild with updated `kms.xml` | **PASS** — plugin-impl contains Jackson (6), Jersey (6), hk2 (5) | | `ensure-kms-plugin-audit-jars.sh --check-only` without brownfield copy | **PASS** | | KMS authorize path (403 guest / 200 keyadmin) | **PASS** | | Solr `repo:dev_kms` count increased after key ops | **PASS** | ## Notes for reviewers * KMS is **not** Kafka-style “remove dupes” — the plugin classloader is isolated from the WAR; plugin-impl must ship its own aligned client stack. * Do **not** flatten plugin-impl into `WEB-INF/lib` as a workaround — known to cause `StackOverflowError` / classloader conflicts. * Customers must **rebuild and redeploy** the KMS tarball (or KMS WAR) after merge; runtime `ensure-jars` scripts are a brownfield workaround only. Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
