[jira] [Commented] (ATLAS-3779) Inmemory JAASConfig issue in Atlas
[
https://issues.apache.org/jira/browse/ATLAS-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120356#comment-17120356
]
ASF subversion and git services commented on ATLAS-3779:
Commit 7a8ca51f83bdd15bf5290f718bd6a79e16eca61b in atlas's branch
refs/heads/master from Jayendra Parab
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=7a8ca51 ]
ATLAS-3779: fallback to KafkaClient jaas configiration when
ticket-basedKafkaClient is not specified
Signed-off-by: Madhan Neethiraj
> Inmemory JAASConfig issue in Atlas
> --
>
> Key: ATLAS-3779
> URL: https://issues.apache.org/jira/browse/ATLAS-3779
> Project: Atlas
> Issue Type: Bug
>Reporter: Mayank Jain
>Assignee: Mayank Jain
>Priority: Major
> Fix For: trunk, 3.0.0
>
>
> Spark uses Kafka as source and sink in secure cluster. The test creates a
> JAAS file like this:
> {code:java}
> KafkaClient {
> com.sun.security.auth.module.Krb5LoginModule required
> debug=true
> useKeyTab=true
> storeKey=true
> keyTab="/xxx/keytabs/systest.keytab"
> useTicketCache=false
> serviceName="kafka"
> principal="syst...@gce.example.com";
> };
> {code}
> As one can see serviceName is set properly.
> Then the test pass the JAAS file to Spark's driver + executor as well:
> {code:java}
> "--conf
> \"spark.driver.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> "--conf
> \"spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> {code}
> Later on SAC + atlas makes some magic in the background with the Jvm JAAS
> configuration. As a result Spark is not able to create consumer for
> processing data:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: No serviceName defined in
> either JAAS or Kafka config
> {code}
> When I've turned off SAC then all the problem gone away.
> Atlas replaces the JVM global JAAS configuration with
> InMemoryJAASConfiguration once Atlas configuration is initialized.
> InMemoryJAASConfiguration has an old JAAS config as "parent" but Atlas config
> takes precedence which is unexpected.
> We never want to let Atlas to overwrite existing JAAS configuration if
> there's a conflict. (I believe most endpoints using Atlas client as a library
> would agree with this.) This may be achieved via swapping precedence for
> "parent" vs "Atlas config" in InMemoryJAASConfiguration, but I have no idea
> the change would be safe to Atlas side. In any way, Atlas should at least
> provide a config to let "parent" take precedence for the conflict.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-3779) Inmemory JAASConfig issue in Atlas
[
https://issues.apache.org/jira/browse/ATLAS-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120358#comment-17120358
]
ASF subversion and git services commented on ATLAS-3779:
Commit 09e382b9a0a5f48e3551db69996ad0b1e04cf79e in atlas's branch
refs/heads/branch-2.0 from Jayendra Parab
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=09e382b ]
ATLAS-3779: fallback to KafkaClient jaas configiration when
ticket-basedKafkaClient is not specified
Signed-off-by: Madhan Neethiraj
(cherry picked from commit 7a8ca51f83bdd15bf5290f718bd6a79e16eca61b)
> Inmemory JAASConfig issue in Atlas
> --
>
> Key: ATLAS-3779
> URL: https://issues.apache.org/jira/browse/ATLAS-3779
> Project: Atlas
> Issue Type: Bug
>Reporter: Mayank Jain
>Assignee: Mayank Jain
>Priority: Major
> Fix For: trunk, 3.0.0
>
>
> Spark uses Kafka as source and sink in secure cluster. The test creates a
> JAAS file like this:
> {code:java}
> KafkaClient {
> com.sun.security.auth.module.Krb5LoginModule required
> debug=true
> useKeyTab=true
> storeKey=true
> keyTab="/xxx/keytabs/systest.keytab"
> useTicketCache=false
> serviceName="kafka"
> principal="syst...@gce.example.com";
> };
> {code}
> As one can see serviceName is set properly.
> Then the test pass the JAAS file to Spark's driver + executor as well:
> {code:java}
> "--conf
> \"spark.driver.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> "--conf
> \"spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> {code}
> Later on SAC + atlas makes some magic in the background with the Jvm JAAS
> configuration. As a result Spark is not able to create consumer for
> processing data:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: No serviceName defined in
> either JAAS or Kafka config
> {code}
> When I've turned off SAC then all the problem gone away.
> Atlas replaces the JVM global JAAS configuration with
> InMemoryJAASConfiguration once Atlas configuration is initialized.
> InMemoryJAASConfiguration has an old JAAS config as "parent" but Atlas config
> takes precedence which is unexpected.
> We never want to let Atlas to overwrite existing JAAS configuration if
> there's a conflict. (I believe most endpoints using Atlas client as a library
> would agree with this.) This may be achieved via swapping precedence for
> "parent" vs "Atlas config" in InMemoryJAASConfiguration, but I have no idea
> the change would be safe to Atlas side. In any way, Atlas should at least
> provide a config to let "parent" take precedence for the conflict.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-3779) Inmemory JAASConfig issue in Atlas
[
https://issues.apache.org/jira/browse/ATLAS-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117535#comment-17117535
]
ASF subversion and git services commented on ATLAS-3779:
Commit b873dd96ed32ed4731f50ea991bb51c2b43a8a9b in atlas's branch
refs/heads/branch-2.0 from Jayendra Parab
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=b873dd9 ]
ATLAS-3779 : Refactoring Kafka in-memory JAASConfig in Atlas.
(cherry picked from commit 61abecac22ef3e9341a07be6d5354bf246544a3b)
> Inmemory JAASConfig issue in Atlas
> --
>
> Key: ATLAS-3779
> URL: https://issues.apache.org/jira/browse/ATLAS-3779
> Project: Atlas
> Issue Type: Bug
>Reporter: Mayank Jain
>Assignee: Mayank Jain
>Priority: Major
> Fix For: trunk, 3.0.0
>
>
> Spark uses Kafka as source and sink in secure cluster. The test creates a
> JAAS file like this:
> {code:java}
> KafkaClient {
> com.sun.security.auth.module.Krb5LoginModule required
> debug=true
> useKeyTab=true
> storeKey=true
> keyTab="/xxx/keytabs/systest.keytab"
> useTicketCache=false
> serviceName="kafka"
> principal="syst...@gce.example.com";
> };
> {code}
> As one can see serviceName is set properly.
> Then the test pass the JAAS file to Spark's driver + executor as well:
> {code:java}
> "--conf
> \"spark.driver.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> "--conf
> \"spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> {code}
> Later on SAC + atlas makes some magic in the background with the Jvm JAAS
> configuration. As a result Spark is not able to create consumer for
> processing data:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: No serviceName defined in
> either JAAS or Kafka config
> {code}
> When I've turned off SAC then all the problem gone away.
> Atlas replaces the JVM global JAAS configuration with
> InMemoryJAASConfiguration once Atlas configuration is initialized.
> InMemoryJAASConfiguration has an old JAAS config as "parent" but Atlas config
> takes precedence which is unexpected.
> We never want to let Atlas to overwrite existing JAAS configuration if
> there's a conflict. (I believe most endpoints using Atlas client as a library
> would agree with this.) This may be achieved via swapping precedence for
> "parent" vs "Atlas config" in InMemoryJAASConfiguration, but I have no idea
> the change would be safe to Atlas side. In any way, Atlas should at least
> provide a config to let "parent" take precedence for the conflict.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (ATLAS-3779) Inmemory JAASConfig issue in Atlas
[
https://issues.apache.org/jira/browse/ATLAS-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17117430#comment-17117430
]
ASF subversion and git services commented on ATLAS-3779:
Commit 61abecac22ef3e9341a07be6d5354bf246544a3b in atlas's branch
refs/heads/master from Jayendra Parab
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=61abeca ]
ATLAS-3779 : Refactoring Kafka in-memory JAASConfig in Atlas.
> Inmemory JAASConfig issue in Atlas
> --
>
> Key: ATLAS-3779
> URL: https://issues.apache.org/jira/browse/ATLAS-3779
> Project: Atlas
> Issue Type: Bug
>Reporter: Mayank Jain
>Assignee: Mayank Jain
>Priority: Major
>
> Spark uses Kafka as source and sink in secure cluster. The test creates a
> JAAS file like this:
> {code:java}
> KafkaClient {
> com.sun.security.auth.module.Krb5LoginModule required
> debug=true
> useKeyTab=true
> storeKey=true
> keyTab="/xxx/keytabs/systest.keytab"
> useTicketCache=false
> serviceName="kafka"
> principal="syst...@gce.example.com";
> };
> {code}
> As one can see serviceName is set properly.
> Then the test pass the JAAS file to Spark's driver + executor as well:
> {code:java}
> "--conf
> \"spark.driver.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> "--conf
> \"spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
> {code}
> Later on SAC + atlas makes some magic in the background with the Jvm JAAS
> configuration. As a result Spark is not able to create consumer for
> processing data:
> {code:java}
> Caused by: java.lang.IllegalArgumentException: No serviceName defined in
> either JAAS or Kafka config
> {code}
> When I've turned off SAC then all the problem gone away.
> Atlas replaces the JVM global JAAS configuration with
> InMemoryJAASConfiguration once Atlas configuration is initialized.
> InMemoryJAASConfiguration has an old JAAS config as "parent" but Atlas config
> takes precedence which is unexpected.
> We never want to let Atlas to overwrite existing JAAS configuration if
> there's a conflict. (I believe most endpoints using Atlas client as a library
> would agree with this.) This may be achieved via swapping precedence for
> "parent" vs "Atlas config" in InMemoryJAASConfiguration, but I have no idea
> the change would be safe to Atlas side. In any way, Atlas should at least
> provide a config to let "parent" take precedence for the conflict.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
