Mayank Jain created ATLAS-3779:
----------------------------------
Summary: Inmemory JAASConfig issue in Atlas
Key: ATLAS-3779
URL: https://issues.apache.org/jira/browse/ATLAS-3779
Project: Atlas
Issue Type: Bug
Reporter: Mayank Jain
Spark uses Kafka as source and sink in secure cluster. The test creates a JAAS
file like this:
{code:java}
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
debug=true
useKeyTab=true
storeKey=true
keyTab="/cdep/keytabs/systest.keytab"
useTicketCache=false
serviceName="kafka"
principal="[email protected]";
};
{code}
As one can see serviceName is set properly.
Then the test pass the JAAS file to Spark's driver + executor as well:
{code:java}
"--conf
\"spark.driver.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
"--conf
\"spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..."
{code}
Later on SAC + atlas makes some magic in the background with the Jvm JAAS
configuration. As a result Spark is not able to create consumer for processing
data:
{code:java}
Caused by: java.lang.IllegalArgumentException: No serviceName defined in either
JAAS or Kafka config
{code}
When I've turned off SAC then all the problem gone away.
Atlas replaces the JVM global JAAS configuration with InMemoryJAASConfiguration
once Atlas configuration is initialized. InMemoryJAASConfiguration has an old
JAAS config as "parent" but Atlas config takes precedence which is unexpected.
We never want to let Atlas to overwrite existing JAAS configuration if there's
a conflict. (I believe most endpoints using Atlas client as a library would
agree with this.) This may be achieved via swapping precedence for "parent" vs
"Atlas config" in InMemoryJAASConfiguration, but I have no idea the change
would be safe to Atlas side. In any way, Atlas should at least provide a config
to let "parent" take precedence for the conflict.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
