[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1389#comment-1389
]
ASF GitHub Bot commented on KAFKA-4180:
---
GitHub user edoardocomar opened a pull request:
https://github.com/apache/kafka/pull/1989
Kafka 4180 - Shared authentification with multiple actives Kafka
producers/consumers
This PR builds on top of @rajinisivaram
https://github.com/apache/kafka/pull/1979
codeveloped with @mimaison
KAFKA-4180 : Authentication with multiple actives Kafka
producers/consumers
Changed caching in LoginManager to allow one LoginManager per client
JAAS configuration.
Added test to End2EndAuthorization for SASL Plain and Gssapi with two
consumers with different credentials.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/edoardocomar/kafka KAFKA-4180
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/1989.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1989
commit 4bbd859c29a6b425d616756233768f61d2afab05
Author: Rajini Sivaram
Date: 2016-10-06T09:51:43Z
KAFKA-4259: Dynamic JAAS configuration for Kafka clients
commit bbef2add18c768308a68bf6b66e5a5c1106eca0f
Author: Edoardo Comar
Date: 2016-10-06T16:14:21Z
KAFKA-4180 : Authentication with multiple actives Kafka
producers/consumers
Changed caching in LoginManager to allow one LoginManager per client
JAAS configuration.
Added test to End2EndAuthorization for SASL Plain and Gssapi with two
consumers with different credentials.
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
>Assignee: Mickael Maison
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526764#comment-15526764
]
Sriharsha Chintalapani commented on KAFKA-4180:
---
[~ggrossetie] [~ecomar] This is duplicate of KAFKA-3302. We cannot make changes
to only SASL plain. The changes should be applied to LoginManager in general.
Currently, LoginManager is a singleton and we need to break that and pass that
into networking layer.
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
>Assignee: Mickael Maison
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526743#comment-15526743
]
Guillaume Grossetie commented on KAFKA-4180:
Thanks [~mimaison] for looking into this.
I've also tried something like what you described but in my opinion this should
be easier.
The proposed changes in
[KIP-83|https://cwiki.apache.org/confluence/display/KAFKA/KIP-83+-+Allow+multiple+SASL+authenticated+Java+clients+in+a+single+JVM+process]
are good :)
[KAFKA-3302|https://issues.apache.org/jira/browse/KAFKA-3302] is focused on
Kerbos no ?
In my case I want to use SASL plain with a "basic" username/password.
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
>Assignee: Mickael Maison
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526574#comment-15526574
]
Edoardo Comar commented on KAFKA-4180:
--
Hi [~sriharsha] thanks for pointing this out.
I'd say this one didn't look a duplicate of your JIRA, possibly because yours
only contains a title with no description/details.
Also I was going to target KIP-83 to SASL plain, will open a thread on the
mailing list for discussion, as suggested by the template, rather than on the
wiki
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
>Assignee: Mickael Maison
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15526445#comment-15526445
]
Sriharsha Chintalapani commented on KAFKA-4180:
---
[~ggrossetie] [~mimaison] Isn't this duplicate of this JIRA here
https://issues.apache.org/jira/browse/KAFKA-3302. I already started work on
this. Let me know if this is not what your requirements are.
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
>Assignee: Mickael Maison
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15525639#comment-15525639
]
Mickael Maison commented on KAFKA-4180:
---
Actually I had a closer look and we got this to work using a few ugly hacks in
addition of the custom login module, and I'm not sure I want to document them
here !
So I also think we'd need a proper mechanism to support multiple credentials
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-4180) Shared authentification with multiple actives Kafka producers/consumers
[
https://issues.apache.org/jira/browse/KAFKA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15496095#comment-15496095
]
Mickael Maison commented on KAFKA-4180:
---
Hi Guillaume,
You can achieve that by having a custom login module.
Create a class that extends PlainLoginModule and override initialize() with
your credentials logic. For example you can set your client properties as a
ThreadLocal, retrieve the client.id in initialize() and based on that load
different credentials. You can set the username and password by calling
respectively getPublicCredentials().add() and getPrivateCredentials().add() on
the subject.Then update your jaas file to point to your class instead of the
default PlainLoginModule class.
> Shared authentification with multiple actives Kafka producers/consumers
> ---
>
> Key: KAFKA-4180
> URL: https://issues.apache.org/jira/browse/KAFKA-4180
> Project: Kafka
> Issue Type: Bug
> Components: producer , security
>Affects Versions: 0.10.0.1
>Reporter: Guillaume Grossetie
> Labels: authentication, jaas, loginmodule, plain, producer,
> sasl, user
>
> I'm using Kafka 0.10.0.1 with an SASL authentication on the client:
> {code:title=kafka_client_jaas.conf|borderStyle=solid}
> KafkaClient {
> org.apache.kafka.common.security.plain.PlainLoginModule required
> username="guillaume"
> password="secret";
> };
> {code}
> When using multiple Kafka producers the authentification is shared [1]. In
> other words it's not currently possible to have multiple Kafka producers in a
> JVM process.
> Am I missing something ? How can I have multiple active Kafka producers with
> different credentials ?
> My use case is that I have an application that send messages to multiples
> clusters (one cluster for logs, one cluster for metrics, one cluster for
> business data).
> [1]
> https://github.com/apache/kafka/blob/69ebf6f7be2fc0e471ebd5b7a166468017ff2651/clients/src/main/java/org/apache/kafka/common/security/authenticator/LoginManager.java#L35
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
