[
https://issues.apache.org/jira/browse/RANGER-5827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar updated RANGER-5827:
-----------------------------------
Description:
Ranger's Kafka clients (audit-ingestor, audit-dispatchers, tagsync) each build
their own SASL settings, support only Kerberos keytabs, and cannot authenticate
on platforms that issue short-lived tokens such as Kubernetes projected
service-account tokens. Kafka's bundled OAUTHBEARER handler is not a drop-in
fix: it reads a {{[file://|file:///]}} token once and caches it, so after the
platform rotates the token the client keeps presenting the expired one and
fails to re-authenticate.
Proposed changes:
* New module {{agents-audit/kafka-client-auth}}
({{{}ranger-kafka-client-auth{}}}):
** {{RangerKafkaClientSecurityConfig}} builds {{security.protocol}} and
{{sasl.{*}{*}}} *client properties from {{<prefix>.kafka.}}* configuration for
producers, consumers and admin clients. With a SASL protocol the default
mechanism is OAUTHBEARER; GSSAPI (principal and keytab) must be selected
explicitly. {{<prefix>.kafka.sasl.{*}{*}}} *and {{<prefix>.kafka.ssl.}}* pass
through and override derived values.
** {{RangerKafkaOAuthBearerLoginCallbackHandler}} presents a JWT read from a
{{[file://|file:///]}} URL. It re-reads the file on every login and refresh, so
the client always presents the current token after rotation. {{http(s)://}}
endpoints use Kafka's own client-credentials handler.
** {{{}RangerOAuthBearerToken{}}}, a reusable {{OAuthBearerToken}} built from
parsed claims.
* audit-ingestor, all audit-dispatcher backends and the audit topic bootstrap
call the helper instead of building a Kerberos-only JAAS string each.
audit-ingestor also loads the site XML given by {{-Daudit.config}} after the
packaged default.
* tagsync: new {{TAG_SOURCE_ATLAS_KAFKA_SASL_MECHANISM}} and
{{TAG_SOURCE_ATLAS_KAFKA_OAUTH_TOKEN_URL}} install properties rendered into
{{{}atlas-application.properties{}}}; module added to the tagsync distribution.
* ranger-docker: the Kafka broker accepts GSSAPI and OAUTHBEARER; the
{{ranger-kafka}} container mints and rotates dev tokens into a shared volume;
{{KAFKA_SASL_MECHANISM}} in {{.env}} selects OAUTHBEARER (default) or GSSAPI
for the ingestor, dispatchers and tagsync. The ingestor renders its site XML
from YAML plus environment, as the admin container does.
Behaviour change: with {{SASL_PLAINTEXT}} or {{SASL_SSL}} the default
{{sasl.mechanism}} is now OAUTHBEARER (was PLAIN); no {{sasl.*}} keys are set
for PLAINTEXT or SSL.
was:
Ranger's Kafka clients (audit-ingestor, audit-dispatchers, tagsync) each build
their own SASL settings, support only Kerberos keytabs, and cannot authenticate
on platforms that issue short-lived tokens such as Kubernetes projected
service-account tokens. Kafka's bundled OAUTHBEARER handler is not a drop-in
fix: it reads a {{file://}} token once and caches it, so after the platform
rotates the token the client keeps presenting the expired one and fails to
re-authenticate.
Proposed changes:
* New module {{agents-audit/kafka-client-auth}}
({{{}ranger-kafka-client-auth{}}}):
** {{RangerKafkaClientSecurityConfig}} builds {{security.protocol}} and
{{sasl.*}} client properties from {{<prefix>.kafka.*}} configuration for
producers, consumers and admin clients. With a SASL protocol the default
mechanism is OAUTHBEARER; GSSAPI (principal and keytab) must be selected
explicitly. {{<prefix>.kafka.sasl.*}} and {{<prefix>.kafka.ssl.*}} pass through
and override derived values.
** {{RangerKafkaOAuthBearerLoginCallbackHandler}} presents a JWT read from a
{{file://}} URL. It re-reads the file on every login and refresh, so the client
always presents the current token after rotation. {{http(s)://}} endpoints use
Kafka's own client-credentials handler.
** {{{}RangerOAuthBearerToken{}}}, a reusable {{OAuthBearerToken}} built from
parsed claims.
* audit-ingestor, all audit-dispatcher backends and the audit topic bootstrap
call the helper instead of building a Kerberos-only JAAS string each.
audit-ingestor also loads the site XML given by {{-Daudit.config}} after the
packaged default.
* tagsync: new {{TAG_SOURCE_ATLAS_KAFKA_SASL_MECHANISM}} and
{{TAG_SOURCE_ATLAS_KAFKA_OAUTH_TOKEN_URL}} install properties rendered into
{{{}atlas-application.properties{}}}; module added to the tagsync distribution.
* ranger-docker: the Kafka broker accepts GSSAPI and OAUTHBEARER; the
{{ranger-kafka}} container mints and rotates dev tokens into a shared volume;
{{KAFKA_SASL_MECHANISM}} in {{.env}} selects OAUTHBEARER (default) or GSSAPI
for the ingestor, dispatchers and tagsync. The ingestor renders its site XML
from YAML plus environment, as the admin container does.
Behaviour change: with {{SASL_PLAINTEXT}} or {{SASL_SSL}} the default
{{sasl.mechanism}} is now OAUTHBEARER (was PLAIN, which never worked with the
generated Kerberos JAAS); no {{sasl.*}} keys are set for PLAINTEXT or SSL.
> Shared Kafka client authentication for Ranger services with SASL/OAUTHBEARER
> as the default mechanism
> -----------------------------------------------------------------------------------------------------
>
> Key: RANGER-5827
> URL: https://issues.apache.org/jira/browse/RANGER-5827
> Project: Ranger
> Issue Type: Improvement
> Components: audit-server
> Reporter: Abhishek Kumar
> Assignee: Abhishek Kumar
> Priority: Major
>
> Ranger's Kafka clients (audit-ingestor, audit-dispatchers, tagsync) each
> build their own SASL settings, support only Kerberos keytabs, and cannot
> authenticate on platforms that issue short-lived tokens such as Kubernetes
> projected service-account tokens. Kafka's bundled OAUTHBEARER handler is not
> a drop-in fix: it reads a {{[file://|file:///]}} token once and caches it, so
> after the platform rotates the token the client keeps presenting the expired
> one and fails to re-authenticate.
> Proposed changes:
> * New module {{agents-audit/kafka-client-auth}}
> ({{{}ranger-kafka-client-auth{}}}):
> ** {{RangerKafkaClientSecurityConfig}} builds {{security.protocol}} and
> {{sasl.{*}{*}}} *client properties from {{<prefix>.kafka.}}* configuration
> for producers, consumers and admin clients. With a SASL protocol the default
> mechanism is OAUTHBEARER; GSSAPI (principal and keytab) must be selected
> explicitly. {{<prefix>.kafka.sasl.{*}{*}}} *and {{<prefix>.kafka.ssl.}}* pass
> through and override derived values.
> ** {{RangerKafkaOAuthBearerLoginCallbackHandler}} presents a JWT read from a
> {{[file://|file:///]}} URL. It re-reads the file on every login and refresh,
> so the client always presents the current token after rotation.
> {{http(s)://}} endpoints use Kafka's own client-credentials handler.
> ** {{{}RangerOAuthBearerToken{}}}, a reusable {{OAuthBearerToken}} built
> from parsed claims.
> * audit-ingestor, all audit-dispatcher backends and the audit topic
> bootstrap call the helper instead of building a Kerberos-only JAAS string
> each. audit-ingestor also loads the site XML given by {{-Daudit.config}}
> after the packaged default.
> * tagsync: new {{TAG_SOURCE_ATLAS_KAFKA_SASL_MECHANISM}} and
> {{TAG_SOURCE_ATLAS_KAFKA_OAUTH_TOKEN_URL}} install properties rendered into
> {{{}atlas-application.properties{}}}; module added to the tagsync
> distribution.
> * ranger-docker: the Kafka broker accepts GSSAPI and OAUTHBEARER; the
> {{ranger-kafka}} container mints and rotates dev tokens into a shared volume;
> {{KAFKA_SASL_MECHANISM}} in {{.env}} selects OAUTHBEARER (default) or GSSAPI
> for the ingestor, dispatchers and tagsync. The ingestor renders its site XML
> from YAML plus environment, as the admin container does.
> Behaviour change: with {{SASL_PLAINTEXT}} or {{SASL_SSL}} the default
> {{sasl.mechanism}} is now OAUTHBEARER (was PLAIN); no {{sasl.*}} keys are set
> for PLAINTEXT or SSL.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
