[
https://issues.apache.org/jira/browse/KAFKA-21143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
akshatha ramesh updated KAFKA-21143:
------------------------------------
Description:
h2. Problem
A broker configured with an external _SASL_SSL/OAUTHBEARER_ listener can fail
during startup when the broker is intended to act only as an OAuth token
validator and the JWKS endpoint requires a private CA.
h3. Topology
{code:java}
Inter-broker traffic:
PLAINTEXT
External listener:
SASL_SSL + OAUTHBEARER
External clients:
obtain JWTs from an external IdP and present them to Kafka
Kafka broker:
validates incoming JWTs using the IdP JWKS endpoint
does not need to obtain an OAuth token for itself
{code}
The JWKS endpoint uses HTTPS with a private CA, so trust configuration is
supplied for the OAUTHBEARER listener:
{code}
listener.name.oam.oauthbearer.sasl.jaas.config=
org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
ssl.truststore.location="/path/to/idp-ca.crt"
ssl.truststore.type="PEM";
listener.name.oam.oauthbearer.sasl.server.callback.handler.class=
org.apache.kafka.common.security.oauthbearer.OAuthBearerValidatorCallbackHandler
listener.name.oam.oauthbearer.sasl.oauthbearer.jwks.endpoint.url=
https:///.../certs
security.inter.broker.protocol=PLAINTEXT
{code}
h2. Actual behavior
During broker startup, initialization of the external SASL listener reaches
{{OAuthBearerLoginModule.login()}} and fails:
{code:java}
kafka.network.DataPlaneAcceptor.configure
...
org.apache.kafka.common.network.SaslChannelBuilder.configure
...
org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login
javax.security.auth.login.LoginException:
An internal error occurred while retrieving token from callback handler
{code}
The failure occurs during listener initialization; an external client does not
need to connect for the failure to occur.
h2. Existing tokenless server-side behavior
KAFKA-7462 addressed the case where {{OAuthBearerLoginModule}} is initialized
on a broker even though OAUTHBEARER is not being used for inter-broker
authentication.
The default {{OAuthBearerUnsecuredLoginCallbackHandler}} detects the tokenless
server-side case using:
{code:java}
if (moduleOptions.isEmpty()) {
callback.token(null);
return;
}
{code}
{{OAuthBearerLoginModule}} accepts this tokenless login:
{code:java}
identifyToken();
if (tokenRequiringCommit != null)
identifyExtensions();
else
log.debug("Logged in without a token...");
{code}
Therefore, when there are no JAAS module options, the broker can initialize the
OAUTHBEARER listener without obtaining an OAuth token for itself.
Conceptually:
{code:java}
OAUTHBEARER listener
+
no broker OAuth client role
+
empty JAAS module options
|
v
callback.token(null)
|
v
OAuthBearerLoginModule accepts tokenless login
|
v
listener starts
{code}
h2. Problem with a private-CA JWKS endpoint
For a validation-only listener whose JWKS endpoint requires a private CA, the
JAAS configuration contains TLS trust options:
{code}
ssl.truststore.location="/path/to/idp-ca.crt"
ssl.truststore.type="PEM"
{code}
Consequently:
{code:java}
moduleOptions.isEmpty() == false
{code}
However, these options are present to establish TLS trust for the JWKS
endpoint. They do not represent a requirement for the broker to obtain an OAuth
access token.
Because {{moduleOptions}} is no longer empty,
{{OAuthBearerUnsecuredLoginCallbackHandler}} does not take the KAFKA-7462
tokenless path.
It instead continues into its token construction path. Without the required
unsecured-login principal configuration, token construction fails and the error
is eventually surfaced from {{OAuthBearerLoginModule}} as:
{code:java}
javax.security.auth.login.LoginException:
An internal error occurred while retrieving token from callback handler
{code}
h2. Expected behavior
The following topology should be possible without requiring the broker to
obtain an OAuth token for the external listener:
{code:java}
inter-broker = PLAINTEXT
external listener = SASL_SSL/OAUTHBEARER
broker OAuth role = validate incoming JWTs only
JWKS endpoint = HTTPS with private CA
{code}
Providing TLS trust configuration required to access the JWKS endpoint should
not, by itself, require broker-side OAuth token acquisition.
h2. Possible gap
The current {{moduleOptions.isEmpty()}} condition appears unable to distinguish
between:
{code:java}
JAAS options representing broker-side OAuth login/token configuration
{code}
and:
{code:java}
JAAS options required only for TLS trust when accessing the JWKS endpoint
{code}
As a result, adding the TLS trust configuration needed for a private-CA JWKS
endpoint prevents the existing tokenless server-side path from being taken.
h2. Question
Is there an intended supported configuration that allows JWKS TLS trust options
to be supplied while retaining the tokenless server-side behavior introduced by
KAFKA-7462?
If not, should the tokenless server-side detection distinguish TLS/JWKS
configuration from options that actually configure broker-side OAuth token
creation or acquisition?
h2. Related
Confluent Kafka report:
[https://github.com/confluentinc/kafka/issues/2160]
KAFKA-7462:
https://issues.apache.org/jira/browse/KAFKA-7462
was:
h2. Problem
A broker configured with an external _SASL_SSL/OAUTHBEARER_ listener can fail
during startup when the broker is intended to act only as an OAuth token
validator and the JWKS endpoint requires a private CA.
h3. Topology
{code}
Inter-broker traffic:
PLAINTEXT
External listener:
SASL_SSL + OAUTHBEARER
External clients:
obtain JWTs from an external IdP and present them to Kafka
Kafka broker:
validates incoming JWTs using the IdP JWKS endpoint
does not need to obtain an OAuth token for itself
{code}
The JWKS endpoint uses HTTPS with a private CA, so trust configuration is
supplied for the OAUTHBEARER listener:
{code:properties}
listener.name.oam.oauthbearer.sasl.jaas.config=
org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
ssl.truststore.location="/path/to/idp-ca.crt"
ssl.truststore.type="PEM";
listener.name.oam.oauthbearer.sasl.server.callback.handler.class=
org.apache.kafka.common.security.oauthbearer.OAuthBearerValidatorCallbackHandler
listener.name.oam.oauthbearer.sasl.oauthbearer.jwks.endpoint.url=
https:///.../certs
security.inter.broker.protocol=PLAINTEXT
{code}
h2. Actual behavior
During broker startup, initialization of the external SASL listener reaches
\{{OAuthBearerLoginModule.login()}} and fails:
{code}
kafka.network.DataPlaneAcceptor.configure
...
org.apache.kafka.common.network.SaslChannelBuilder.configure
...
org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login
javax.security.auth.login.LoginException:
An internal error occurred while retrieving token from callback handler
{code}
The failure occurs during listener initialization; an external client does not
need to connect for the failure to occur.
h2. Existing tokenless server-side behavior
KAFKA-7462 addressed the case where \{{OAuthBearerLoginModule}} is initialized
on a broker even though OAUTHBEARER is not being used for inter-broker
authentication.
The default \{{OAuthBearerUnsecuredLoginCallbackHandler}} detects the tokenless
server-side case using:
{code:java}
if (moduleOptions.isEmpty()) {
callback.token(null);
return;
}
{code}
{\{OAuthBearerLoginModule}} accepts this tokenless login:
{code:java}
identifyToken();
if (tokenRequiringCommit != null)
identifyExtensions();
else
log.debug("Logged in without a token...");
{code}
Therefore, when there are no JAAS module options, the broker can initialize the
OAUTHBEARER listener without obtaining an OAuth token for itself.
Conceptually:
{code}
OAUTHBEARER listener
+
no broker OAuth client role
+
empty JAAS module options
|
v
callback.token(null)
|
v
OAuthBearerLoginModule accepts tokenless login
|
v
listener starts
{code}
h2. Problem with a private-CA JWKS endpoint
For a validation-only listener whose JWKS endpoint requires a private CA, the
JAAS configuration contains TLS trust options:
{code:properties}
ssl.truststore.location="/path/to/idp-ca.crt"
ssl.truststore.type="PEM"
{code}
Consequently:
{code:java}
moduleOptions.isEmpty() == false
{code}
However, these options are present to establish TLS trust for the JWKS
endpoint. They do not represent a requirement for the broker to obtain an OAuth
access token.
Because \{{moduleOptions}} is no longer empty,
\{{OAuthBearerUnsecuredLoginCallbackHandler}} does not take the KAFKA-7462
tokenless path.
It instead continues into its token construction path. Without the required
unsecured-login principal configuration, token construction fails and the error
is eventually surfaced from \{{OAuthBearerLoginModule}} as:
{code}
javax.security.auth.login.LoginException:
An internal error occurred while retrieving token from callback handler
{code}
h2. Expected behavior
The following topology should be possible without requiring the broker to
obtain an OAuth token for the external listener:
{code}
inter-broker = PLAINTEXT
external listener = SASL_SSL/OAUTHBEARER
broker OAuth role = validate incoming JWTs only
JWKS endpoint = HTTPS with private CA
{code}
Providing TLS trust configuration required to access the JWKS endpoint should
not, by itself, require broker-side OAuth token acquisition.
h2. Possible gap
The current \{{moduleOptions.isEmpty()}} condition appears unable to
distinguish between:
{code}
JAAS options representing broker-side OAuth login/token configuration
{code}
and:
{code}
JAAS options required only for TLS trust when accessing the JWKS endpoint
{code}
As a result, adding the TLS trust configuration needed for a private-CA JWKS
endpoint prevents the existing tokenless server-side path from being taken.
h2. Question
Is there an intended supported configuration that allows JWKS TLS trust options
to be supplied while retaining the tokenless server-side behavior introduced by
KAFKA-7462?
If not, should the tokenless server-side detection distinguish TLS/JWKS
configuration from options that actually configure broker-side OAuth token
creation or acquisition?
h2. Related
Confluent Kafka report:
[https://github.com/confluentinc/kafka/issues/2160]
KAFKA-7462:
https://issues.apache.org/jira/browse/KAFKA-7462
> Server-only OAUTHBEARER listener with private JWKS CA triggers token login
> and fails broker startup
> ---------------------------------------------------------------------------------------------------
>
> Key: KAFKA-21143
> URL: https://issues.apache.org/jira/browse/KAFKA-21143
> Project: Kafka
> Issue Type: Bug
> Components: security
> Affects Versions: 4.2.2
> Environment: Apache Kafka 4.2.0
> Reporter: akshatha ramesh
> Priority: Critical
>
> h2. Problem
> A broker configured with an external _SASL_SSL/OAUTHBEARER_ listener can fail
> during startup when the broker is intended to act only as an OAuth token
> validator and the JWKS endpoint requires a private CA.
> h3. Topology
> {code:java}
> Inter-broker traffic:
> PLAINTEXT
> External listener:
> SASL_SSL + OAUTHBEARER
> External clients:
> obtain JWTs from an external IdP and present them to Kafka
> Kafka broker:
> validates incoming JWTs using the IdP JWKS endpoint
> does not need to obtain an OAuth token for itself
> {code}
> The JWKS endpoint uses HTTPS with a private CA, so trust configuration is
> supplied for the OAUTHBEARER listener:
> {code}
> listener.name.oam.oauthbearer.sasl.jaas.config=
> org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule required
> ssl.truststore.location="/path/to/idp-ca.crt"
> ssl.truststore.type="PEM";
> listener.name.oam.oauthbearer.sasl.server.callback.handler.class=
> org.apache.kafka.common.security.oauthbearer.OAuthBearerValidatorCallbackHandler
> listener.name.oam.oauthbearer.sasl.oauthbearer.jwks.endpoint.url=
> https:///.../certs
> security.inter.broker.protocol=PLAINTEXT
> {code}
> h2. Actual behavior
> During broker startup, initialization of the external SASL listener reaches
> {{OAuthBearerLoginModule.login()}} and fails:
> {code:java}
> kafka.network.DataPlaneAcceptor.configure
> ...
> org.apache.kafka.common.network.SaslChannelBuilder.configure
> ...
> org.apache.kafka.common.security.oauthbearer.OAuthBearerLoginModule.login
> javax.security.auth.login.LoginException:
> An internal error occurred while retrieving token from callback handler
> {code}
> The failure occurs during listener initialization; an external client does
> not need to connect for the failure to occur.
> h2. Existing tokenless server-side behavior
> KAFKA-7462 addressed the case where {{OAuthBearerLoginModule}} is initialized
> on a broker even though OAUTHBEARER is not being used for inter-broker
> authentication.
> The default {{OAuthBearerUnsecuredLoginCallbackHandler}} detects the
> tokenless server-side case using:
> {code:java}
> if (moduleOptions.isEmpty()) {
> callback.token(null);
> return;
> }
> {code}
> {{OAuthBearerLoginModule}} accepts this tokenless login:
> {code:java}
> identifyToken();
> if (tokenRequiringCommit != null)
> identifyExtensions();
> else
> log.debug("Logged in without a token...");
> {code}
> Therefore, when there are no JAAS module options, the broker can initialize
> the OAUTHBEARER listener without obtaining an OAuth token for itself.
> Conceptually:
> {code:java}
> OAUTHBEARER listener
> +
> no broker OAuth client role
> +
> empty JAAS module options
> |
> v
> callback.token(null)
> |
> v
> OAuthBearerLoginModule accepts tokenless login
> |
> v
> listener starts
> {code}
> h2. Problem with a private-CA JWKS endpoint
> For a validation-only listener whose JWKS endpoint requires a private CA, the
> JAAS configuration contains TLS trust options:
> {code}
> ssl.truststore.location="/path/to/idp-ca.crt"
> ssl.truststore.type="PEM"
> {code}
> Consequently:
> {code:java}
> moduleOptions.isEmpty() == false
> {code}
> However, these options are present to establish TLS trust for the JWKS
> endpoint. They do not represent a requirement for the broker to obtain an
> OAuth access token.
> Because {{moduleOptions}} is no longer empty,
> {{OAuthBearerUnsecuredLoginCallbackHandler}} does not take the KAFKA-7462
> tokenless path.
> It instead continues into its token construction path. Without the required
> unsecured-login principal configuration, token construction fails and the
> error is eventually surfaced from {{OAuthBearerLoginModule}} as:
> {code:java}
> javax.security.auth.login.LoginException:
> An internal error occurred while retrieving token from callback handler
> {code}
> h2. Expected behavior
> The following topology should be possible without requiring the broker to
> obtain an OAuth token for the external listener:
> {code:java}
> inter-broker = PLAINTEXT
> external listener = SASL_SSL/OAUTHBEARER
> broker OAuth role = validate incoming JWTs only
> JWKS endpoint = HTTPS with private CA
> {code}
> Providing TLS trust configuration required to access the JWKS endpoint should
> not, by itself, require broker-side OAuth token acquisition.
> h2. Possible gap
> The current {{moduleOptions.isEmpty()}} condition appears unable to
> distinguish between:
> {code:java}
> JAAS options representing broker-side OAuth login/token configuration
> {code}
> and:
> {code:java}
> JAAS options required only for TLS trust when accessing the JWKS endpoint
> {code}
> As a result, adding the TLS trust configuration needed for a private-CA JWKS
> endpoint prevents the existing tokenless server-side path from being taken.
> h2. Question
> Is there an intended supported configuration that allows JWKS TLS trust
> options to be supplied while retaining the tokenless server-side behavior
> introduced by KAFKA-7462?
> If not, should the tokenless server-side detection distinguish TLS/JWKS
> configuration from options that actually configure broker-side OAuth token
> creation or acquisition?
> h2. Related
> Confluent Kafka report:
> [https://github.com/confluentinc/kafka/issues/2160]
> KAFKA-7462:
> https://issues.apache.org/jira/browse/KAFKA-7462
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
