[
https://issues.apache.org/jira/browse/RANGER-5658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramachandran Krishnan updated RANGER-5658:
------------------------------------------
Fix Version/s: 2.9.0
> Tag Sync: remove atlas.kafka.zookeeper.connect and fix Atlas Kafka consumer
> for Kafka 3.9
> -----------------------------------------------------------------------------------------
>
> Key: RANGER-5658
> URL: https://issues.apache.org/jira/browse/RANGER-5658
> Project: Ranger
> Issue Type: Task
> Components: Ranger
> Affects Versions: 3.0.0
> Reporter: Ramachandran Krishnan
> Assignee: Ramachandran Krishnan
> Priority: Major
> Fix For: 3.0.0, 2.9.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> h3. Summary
> Ranger Tag Sync's Atlas Kafka source (ranger.tagsync.source.atlas) consumed
> Atlas entity notifications via kafka-clients but still required the legacy
> property atlas.kafka.zookeeper.connect at startup. That property was never
> used by Atlas KafkaNotification for consumer creation. With Kafka 3.9.x and
> KRaft brokers, clients use bootstrap.servers only.
> This change removes the ZK requirement and fixes Tag Sync Atlas Kafka setup
> so consumption works with both PLAINTEXT and Kerberos (SASL) Atlas Kafka,
> including when Ranger runs with is_secure=true but the Atlas metadata bus is
> PLAINTEXT.
> h3. Problem
> * Tag Sync validated atlas.kafka.zookeeper.connect even though Atlas 2.4+
> KafkaNotification does not use it
> * setup.py wrote Kerberos JAAS for the Atlas Kafka client whenever
> is_secure=true, even when atlas.kafka.security.protocol=PLAINTEXT — breaking
> PLAINTEXT/KRaft metadata buses
> * AtlasTagSource loaded atlas-application.properties from the classpath only;
> installer-generated conf under TAGSYNC_CONF_DIR was not reliably used without
> -Datlas.conf
> h3. Changes
> * AtlasTagSource — remove ZK validation; load config via
> ApplicationProperties; validate bootstrap.servers + consumer group only
> * setup.py — atlas_kafka_uses_kerberos(): emit atlas.jaas.KafkaClient.* only
> when security.protocol starts with SASL; always write
> atlas.kafka.security.protocol
> * ranger-tagsync-services.sh — pass -Datlas.conf=${TAGSYNC_CONF_DIR}
> * install.properties / installprop2xml.properties — remove
> TAG_SOURCE_ATLAS_KAFKA_ZOOKEEPER_CONNECT
> * setup.py — use rangerTagsync_password for initial tagadmin jceks credential
> (fixes 401 on tag upload in basic-auth deployments)
> * dev-support/ranger-docker/scripts/tagsync/ranger-tagsync-install.properties
> — default Atlas REST on, Kafka off; keep Kafka sample settings for
> ranger-kafka.rangernw:9092 + SASL when enabled
> * dev-support/README-TAGSYNC-ATLAS-KAFKA-CONFIG.md — operator guide
> (PLAINTEXT vs SASL, startup, verification)
> h3. Supported Atlas Kafka modes
> || Mode || TAG_SOURCE_ATLAS_KAFKA_SECURITY_PROTOCOL || Kerberos JAAS in
> atlas-application.properties ||
> | PLAINTEXT | PLAINTEXT | No |
> | Kerberos | SASL_PLAINTEXT / SASL_SSL | Yes (when is_secure=true +
> principal/keytab) |
> Atlas Kafka security is independent of Tag Sync → Ranger Admin authentication
> (basic auth or Ranger Kerberos for tag upload).
> h3. Out of scope
> * Tag Sync HA (ranger-tagsync.server.ha.zookeeper.*) — unchanged
> * Atlas REST tag source behavior — unchanged except jceks password fix
> h3. Upgrade note
> Existing installs may still have atlas.kafka.zookeeper.connect in
> conf/atlas-application.properties; harmless, can be removed on next config
> refresh.
> h3. Test plan
> * mvn test -pl tagsync -Dtest=AtlasTagSourceConfigTest
> * Fresh install: atlas-application.properties has no zookeeper.connect
> * PLAINTEXT: is_secure=true + security.protocol=PLAINTEXT → no Kerberos JAAS;
> consumer connects
> * SASL: security.protocol=SASL_PLAINTEXT + principal/keytab → JAAS written;
> consumer connects (e.g. ranger-kafka.rangernw:9092 in ranger-docker)
> * Tag Sync starts with -Datlas.conf and consumes ATLAS_ENTITIES from Kafka
> 3.9.x (KRaft)
> * Upgrade: legacy zookeeper.connect present → Tag Sync still starts
> * Tag Sync HA smoke test (if HA enabled)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
