[
https://issues.apache.org/jira/browse/RANGER-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar updated RANGER-5827:
-----------------------------------
Description:
Ranger's Kafka clients (audit-ingestor, audit-dispatchers, tagsync) each build
their own SASL settings, support only Kerberos keytabs, and cannot authenticate
on platforms that issue short-lived tokens such as Kubernetes projected
service-account tokens. Kafka's bundled OAUTHBEARER handler is not a drop-in
fix: it reads a {{[file://|file:///]}} token once and caches it, so after the
platform rotates the token the client keeps presenting the expired one and
fails to re-authenticate.
Proposed changes:
* New module ranger-kafka-client-auth:
** RangerKafkaClientSecurityConfig builds security.protocol / sasl.* client
properties from .kafka.* for producers, consumers and admin clients. GSSAPI
remains the default mechanism; OAUTHBEARER is selected with
kafka.sasl.mechanism. JAAS is built from
{{{}kafka.sasl.jaas.loginModuleName{}}},
{{kafka.sasl.jaas.loginModuleControlFlag}} and any
{{{}kafka.sasl.jaas.option.<name>{}}}, so Krb5LoginModule options such as
{{useTicketCache}} are configuration, not code. {{<prefix>.kafka.sasl.*}} and
{{<prefix>.kafka.ssl.*}} pass through and override.
** {{RangerKafkaOAuthBearerLoginCallbackHandler}} presents a JWT read from a
file:// URL. It re-reads the file on every login and refresh, so the client
always presents the current token after rotation. http(s):// endpoints use
Kafka’s own client-credentials handler.
** RangerOAuthBearerToken, a reusable OAuthBearerToken built from parsed
claims.
* audit-ingestor, all audit-dispatcher backends and the audit topic bootstrap
call the helper instead of building a Kerberos-only JAAS string each.
audit-ingestor also loads the site XML given by {{-Daudit.config}} after the
packaged default.
* tagsync: new {{TAG_SOURCE_ATLAS_KAFKA_SASL_MECHANISM}} and
{{TAG_SOURCE_ATLAS_KAFKA_OAUTH_TOKEN_URL}} install properties rendered into
{{{}atlas-application.properties{}}}; module added to the tagsync distribution.
* ranger-docker: the Kafka broker accepts GSSAPI and OAUTHBEARER; the
{{ranger-kafka}} container mints and rotates dev tokens into a shared volume;
{{KAFKA_SASL_MECHANISM}} in {{.env}} selects OAUTHBEARER (default) or GSSAPI
for the ingestor, dispatchers and tagsync. The ingestor renders its site XML
from YAML plus environment, as the admin container does.
was:
Ranger's Kafka clients (audit-ingestor, audit-dispatchers, tagsync) each build
their own SASL settings, support only Kerberos keytabs, and cannot authenticate
on platforms that issue short-lived tokens such as Kubernetes projected
service-account tokens. Kafka's bundled OAUTHBEARER handler is not a drop-in
fix: it reads a {{[file://|file:///]}} token once and caches it, so after the
platform rotates the token the client keeps presenting the expired one and
fails to re-authenticate.
Proposed changes:
* New module {{agents-audit/kafka-client-auth}}
({{{}ranger-kafka-client-auth{}}}):
** {{RangerKafkaClientSecurityConfig}} builds {{security.protocol}} and
{{sasl.{*}{*}}} *client properties from {{<prefix>.kafka.}}* configuration for
producers, consumers and admin clients. With a SASL protocol the default
mechanism is OAUTHBEARER; GSSAPI (principal and keytab) must be selected
explicitly. {{<prefix>.kafka.sasl.{*}{*}}} *and {{<prefix>.kafka.ssl.}}* pass
through and override derived values.
** {{RangerKafkaOAuthBearerLoginCallbackHandler}} presents a JWT read from a
{{[file://|file:///]}} URL. It re-reads the file on every login and refresh, so
the client always presents the current token after rotation. {{http(s)://}}
endpoints use Kafka's own client-credentials handler.
** {{{}RangerOAuthBearerToken{}}}, a reusable {{OAuthBearerToken}} built from
parsed claims.
* audit-ingestor, all audit-dispatcher backends and the audit topic bootstrap
call the helper instead of building a Kerberos-only JAAS string each.
audit-ingestor also loads the site XML given by {{-Daudit.config}} after the
packaged default.
* tagsync: new {{TAG_SOURCE_ATLAS_KAFKA_SASL_MECHANISM}} and
{{TAG_SOURCE_ATLAS_KAFKA_OAUTH_TOKEN_URL}} install properties rendered into
{{{}atlas-application.properties{}}}; module added to the tagsync distribution.
* ranger-docker: the Kafka broker accepts GSSAPI and OAUTHBEARER; the
{{ranger-kafka}} container mints and rotates dev tokens into a shared volume;
{{KAFKA_SASL_MECHANISM}} in {{.env}} selects OAUTHBEARER (default) or GSSAPI
for the ingestor, dispatchers and tagsync. The ingestor renders its site XML
from YAML plus environment, as the admin container does.
Behaviour change: with {{SASL_PLAINTEXT}} or {{SASL_SSL}} the default
{{sasl.mechanism}} is now OAUTHBEARER (was PLAIN); no {{sasl.*}} keys are set
for PLAINTEXT or SSL.
> Shared Kafka client authentication for Ranger services with SASL/OAUTHBEARER
> as the default mechanism
> -----------------------------------------------------------------------------------------------------
>
> Key: RANGER-5827
> URL: https://issues.apache.org/jira/browse/RANGER-5827
> Project: Ranger
> Issue Type: Improvement
> Components: audit-server
> Reporter: Abhishek Kumar
> Assignee: Abhishek Kumar
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Ranger's Kafka clients (audit-ingestor, audit-dispatchers, tagsync) each
> build their own SASL settings, support only Kerberos keytabs, and cannot
> authenticate on platforms that issue short-lived tokens such as Kubernetes
> projected service-account tokens. Kafka's bundled OAUTHBEARER handler is not
> a drop-in fix: it reads a {{[file://|file:///]}} token once and caches it, so
> after the platform rotates the token the client keeps presenting the expired
> one and fails to re-authenticate.
> Proposed changes:
> * New module ranger-kafka-client-auth:
> ** RangerKafkaClientSecurityConfig builds security.protocol / sasl.* client
> properties from .kafka.* for producers, consumers and admin clients. GSSAPI
> remains the default mechanism; OAUTHBEARER is selected with
> kafka.sasl.mechanism. JAAS is built from
> {{{}kafka.sasl.jaas.loginModuleName{}}},
> {{kafka.sasl.jaas.loginModuleControlFlag}} and any
> {{{}kafka.sasl.jaas.option.<name>{}}}, so Krb5LoginModule options such as
> {{useTicketCache}} are configuration, not code. {{<prefix>.kafka.sasl.*}} and
> {{<prefix>.kafka.ssl.*}} pass through and override.
> ** {{RangerKafkaOAuthBearerLoginCallbackHandler}} presents a JWT read from a
> file:// URL. It re-reads the file on every login and refresh, so the client
> always presents the current token after rotation. http(s):// endpoints use
> Kafka’s own client-credentials handler.
> ** RangerOAuthBearerToken, a reusable OAuthBearerToken built from parsed
> claims.
> * audit-ingestor, all audit-dispatcher backends and the audit topic
> bootstrap call the helper instead of building a Kerberos-only JAAS string
> each. audit-ingestor also loads the site XML given by {{-Daudit.config}}
> after the packaged default.
> * tagsync: new {{TAG_SOURCE_ATLAS_KAFKA_SASL_MECHANISM}} and
> {{TAG_SOURCE_ATLAS_KAFKA_OAUTH_TOKEN_URL}} install properties rendered into
> {{{}atlas-application.properties{}}}; module added to the tagsync
> distribution.
> * ranger-docker: the Kafka broker accepts GSSAPI and OAUTHBEARER; the
> {{ranger-kafka}} container mints and rotates dev tokens into a shared volume;
> {{KAFKA_SASL_MECHANISM}} in {{.env}} selects OAUTHBEARER (default) or GSSAPI
> for the ingestor, dispatchers and tagsync. The ingestor renders its site XML
> from YAML plus environment, as the admin container does.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
