Mayank Jain created ATLAS-3779: ---------------------------------- Summary: Inmemory JAASConfig issue in Atlas Key: ATLAS-3779 URL: https://issues.apache.org/jira/browse/ATLAS-3779 Project: Atlas Issue Type: Bug Reporter: Mayank Jain
Spark uses Kafka as source and sink in secure cluster. The test creates a JAAS file like this: {code:java} KafkaClient { com.sun.security.auth.module.Krb5LoginModule required debug=true useKeyTab=true storeKey=true keyTab="/cdep/keytabs/systest.keytab" useTicketCache=false serviceName="kafka" principal="syst...@gce.cloudera.com"; }; {code} As one can see serviceName is set properly. Then the test pass the JAAS file to Spark's driver + executor as well: {code:java} "--conf \"spark.driver.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..." "--conf \"spark.executor.extraJavaOptions=-Djava.security.auth.login.config=./kafka_source_jaas.conf..." {code} Later on SAC + atlas makes some magic in the background with the Jvm JAAS configuration. As a result Spark is not able to create consumer for processing data: {code:java} Caused by: java.lang.IllegalArgumentException: No serviceName defined in either JAAS or Kafka config {code} When I've turned off SAC then all the problem gone away. Atlas replaces the JVM global JAAS configuration with InMemoryJAASConfiguration once Atlas configuration is initialized. InMemoryJAASConfiguration has an old JAAS config as "parent" but Atlas config takes precedence which is unexpected. We never want to let Atlas to overwrite existing JAAS configuration if there's a conflict. (I believe most endpoints using Atlas client as a library would agree with this.) This may be achieved via swapping precedence for "parent" vs "Atlas config" in InMemoryJAASConfiguration, but I have no idea the change would be safe to Atlas side. In any way, Atlas should at least provide a config to let "parent" take precedence for the conflict. -- This message was sent by Atlassian Jira (v8.3.4#803005)