ramackri opened a new pull request, #1016: URL: https://github.com/apache/ranger/pull/1016
Fixes [RANGER-5643](https://issues.apache.org/jira/browse/RANGER-5643): Solr audit dispatcher fails to index audits into Kerberos-protected Solr because JAAS principals and Solr HTTP URLs did not match deployed hostnames. ### Problem The Solr dispatcher consumes from Kafka but writes **zero Solr documents** when Kerberos is enabled. Dispatcher logs show SPNEGO/JAAS errors (`LOOKING_UP_SERVER`, `NEGOTIATE`, `No key to store`, `Server not found in Kerberos database`). | Failure | Root cause | |---------|------------| | JAAS keytab login | `InMemoryJAASConfiguration` expanded `rangerauditserver/_HOST@REALM` using the local container hostname, not `ranger.audit.dispatcher.host` | | SPNEGO to Solr | Solr URL used short name (`ranger-solr`) while Solr service principal is `HTTP/ranger-solr.rangernw@REALM` | The HDFS dispatcher already worked: it uses `UserGroupInformation.loginUserFromKeytab` and expands `_HOST` via `ranger.audit.dispatcher.host`. Solr uses SolrJ in-memory JAAS and needed the same host-driven principal model plus Solr URL hostname alignment. **Workaround before this fix:** hardcode full JAAS principals and FQDN Solr URLs in Docker-only site XML — not suitable for shipped defaults. ### Solution 1. **`InMemoryJAASConfiguration`** — expand `_HOST` in JAAS principals using `ranger.audit.dispatcher.host` (fallback `ranger.audit.ingestor.host`), same property as HDFS/Kafka dispatchers. 2. **`SolrAuditDestination`** — when `xasecure.audit.destination.solr.host` is set, rewrite `xasecure.audit.destination.solr.urls` hostname to that FQDN so SPNEGO matches Solr’s `HTTP/<host>@REALM`. 3. **Shipped + Docker site XML** — keep `_HOST` in principals; add `solr.host`; set `useTicketCache=false` for container JAAS. No Docker-only principal overrides required after this change. ### Changes | Area | File | Change | |------|------|--------| | JAAS `_HOST` | `agents-audit/core/.../InMemoryJAASConfiguration.java` | `resolveHostForJaas()`, `resolveJaasPrincipal()` using `SecurityUtil.getServerPrincipal` + `ranger.audit.dispatcher.host` | | Solr URL | `agents-audit/dest-solr/.../SolrAuditDestination.java` | `PROP_SOLR_HOST`, `resolveSolrUrls()` at connect time | | Tests (core) | `agents-audit/core/.../InMemoryJAASConfigurationTest.java` | Principal expansion + full JAAS init | | Tests (solr) | `agents-audit/dest-solr/.../SolrAuditDestinationTest.java` | URL hostname rewrite | | Test deps | `agents-audit/dest-solr/pom.xml` | JUnit Jupiter test scope | | Shipped config | `audit-server/audit-dispatcher/dispatcher-solr/.../ranger-audit-dispatcher-solr-site.xml` | `xasecure.audit.destination.solr.host`; `_HOST` principals; docs | | Docker config | `dev-support/ranger-docker/scripts/audit-dispatcher/ranger-audit-dispatcher-solr-site.xml` | Aligned with shipped defaults (removed hardcoded full principals) | ### Configuration (after merge) ```xml <property> <name>ranger.audit.dispatcher.host</name> <value><dispatcher-fqdn></value> </property> <property> <name>xasecure.audit.jaas.Client.option.principal</name> <value>rangerauditserver/[email protected]</value> </property> <property> <name>xasecure.audit.destination.solr.host</name> <value><solr-spnego-fqdn></value> </property> <property> <name>xasecure.audit.destination.solr.urls</name> <value>http://ranger-solr:8983/solr/ranger_audits</value> </property> ``` ### Related - Jira: [RANGER-5643](https://issues.apache.org/jira/browse/RANGER-5643) - Plugin → ingestor Jersey packaging (separate PR): [RANGER-5642](https://issues.apache.org/jira/browse/RANGER-5642), [RANGER-5644](https://issues.apache.org/jira/browse/RANGER-5644) — [#1015](https://github.com/apache/ranger/pull/1015) - HDFS dispatcher reference: `AuditHDFSDispatcher.initializeRangerUGI()` ### Test plan #### Unit tests - [ ] JAAS `_HOST` expansion: ```bash mvn test -pl agents-audit/core -Dtest=InMemoryJAASConfigurationTest -q ``` - [ ] Solr URL hostname rewrite: ```bash mvn test -pl agents-audit/dest-solr -Dtest=SolrAuditDestinationTest -q ``` #### Rebuild Solr dispatcher tarball - [ ] Package changed modules: ```bash export MAVEN_OPTS="-Xmx8g -Xms2g" mvn package -Pall \ -pl :ranger-audit-core,:ranger-audit-dest-solr,:ranger-audit-dispatcher-app \ -am -DskipTests -Dcheckstyle.skip=true -Dpmd.skip=true -Drat.skip=true ``` - [ ] Assemble tarball (copy to your deployment `dist/` as needed): ```bash cd distro VERSION="$(mvn help:evaluate -Dexpression=project.version -q -DforceStdout)" echo "${VERSION}" > target/version mvn -P-all org.apache.maven.plugins:maven-assembly-plugin:3.6.0:single \ -DskipAssembly=false \ -Ddescriptor=src/main/assembly/audit-dispatcher.xml \ -DfinalName=ranger-${VERSION} \ -DoutputDirectory=../target \ -Dcheckstyle.skip=true -Dpmd.skip=true -Drat.skip=true -q ls -la ../target/ranger-${VERSION}-audit-dispatcher.tar.gz ``` Expected: tarball ~800+ MiB (not a stub). #### Docker redeploy (optional) - [ ] Rebuild and recreate Solr dispatcher using the compose file(s) for your audit stack (site XML from `scripts/audit-dispatcher/ranger-audit-dispatcher-solr-site.xml`): ```bash cd dev-support/ranger-docker COMPOSE="-f docker-compose.ranger-audit-dispatcher-solr.yml" # add other -f overrides as your stack requires docker compose ${COMPOSE} build ranger-audit-dispatcher-solr docker compose ${COMPOSE} up -d --force-recreate --no-deps ranger-audit-dispatcher-solr ``` - [ ] Confirm dispatcher logs: - Resolved JAAS principal uses `ranger.audit.dispatcher.host` (not container short name) - Solr URL hostname rewritten to `xasecure.audit.destination.solr.host` - No `Server not found in Kerberos database` / SPNEGO / `No key to store` during indexing - [ ] Generate audit activity via ingestor; verify Solr doc count increases for your test repo/user ### Notes for reviewers - Solr cannot use `loginUserFromKeytab` for SolrJ SPNEGO; in-memory JAAS remains required — this PR fixes **how** principals are resolved, not the auth stack choice. - `useTicketCache=false` is still required for in-memory JAAS in containers (unchanged). - Customers must rebuild/redeploy the Solr audit dispatcher after merge; editing Java alone does not update running containers. - Shipped `ranger-audit-dispatcher-solr-site.xml` and Docker mount should stay in sync. Made with [Cursor](https://cursor.com) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
