[
https://issues.apache.org/jira/browse/IMPALA-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18054165#comment-18054165
]
Fang-Yu Rao commented on IMPALA-14699:
--------------------------------------
We could work around this issue by adding the jceks path via the property
*{{fs.s3a.security.credential.provider.path}}* as suggested at
[https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#Step_2:_Configure_the_hadoop.security.credential.provider.path_property]
according to [~MikaelSmith].
> Impala would encounter the error of "Recursive load of credential provider"
> when a remote jceks path is put in the core-site
> ----------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-14699
> URL: https://issues.apache.org/jira/browse/IMPALA-14699
> Project: IMPALA
> Issue Type: Bug
> Reporter: Fang-Yu Rao
> Priority: Major
>
> Impala would encounter the error of "Recursive load of credential provider"
> when a remote jceks path is put in the core-site.
> The following is part of the stack trace we'd get if we put the following
> key-value pair of ("hadoop.security.credential.provider.path",
> "jceks://[email protected]:8020/secret.jceks")
> in core-site.xml.
> {code}
> 26/01/24 19:22:53 WARN security.LdapGroupsMapping: Exception while trying to
> get password for alias
> hadoop.security.group.mapping.ldap.ssl.keystore.password:
> java.io.IOException: Configuration problem with provider path.
> at
> org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2469)
> at
> org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2388)
> at
> org.apache.hadoop.security.LdapGroupsMapping.getPassword(LdapGroupsMapping.java:879)
> at
> org.apache.hadoop.security.LdapGroupsMapping.loadSslConf(LdapGroupsMapping.java:837)
> at
> org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:733)
> at
> org.apache.hadoop.security.RuleBasedLdapGroupsMapping.setConf(RuleBasedLdapGroupsMapping.java:58)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:105)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:101)
> at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:476)
> at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:352)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:314)
> at
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2008)
> at
> org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:743)
> at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:693)
> at
> org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:604)
> ...
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3509)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:524)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:364)
> at
> org.apache.hadoop.security.alias.KeyStoreProvider.initFileSystem(KeyStoreProvider.java:84)
> at
> org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
> at
> org.apache.hadoop.security.alias.KeyStoreProvider.<init>(KeyStoreProvider.java:49)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:42)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:35)
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:68)
> at
> org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:91)
> at
> org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2450)
> at
> org.apache.hadoop.security.LdapGroupsMapping.getPasswordFromCredentialProviders(LdapGroupsMapping.java:858)
> at
> org.apache.hadoop.security.LdapGroupsMapping.loadSslConf(LdapGroupsMapping.java:846)
> at
> org.apache.hadoop.security.LdapGroupsMapping.setConf(LdapGroupsMapping.java:733)
> at
> org.apache.hadoop.security.RuleBasedLdapGroupsMapping.setConf(RuleBasedLdapGroupsMapping.java:58)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:77)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:105)
> at org.apache.hadoop.security.Groups.<init>(Groups.java:101)
> at
> org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:476)
> at
> org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:352)
> at
> org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:314)
> at
> org.apache.hadoop.security.UserGroupInformation.doSubjectLogin(UserGroupInformation.java:2008)
> at
> org.apache.hadoop.security.UserGroupInformation.createLoginUser(UserGroupInformation.java:743)
> at
> org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:693)
> Caused by: org.apache.hadoop.fs.PathIOException:
> `jceks://[email protected]:8020/secret.jceks':
> Recursive load of credential provider; if loading a JCEKS file, this means
> that the filesystem connector is trying to load the same file
> {code}
> Currently we instantiate Configuration instance at
> https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/service/JniFrontend.java#L719.
> We need to figure out where else to instantiate Configuration so that we
> won't hit this issue described above. I think it's
> "Groups.getUserToGroupsMappingService(CONF)" that triggers the recursion.
> {code}
> // Caching this saves ~50ms per call to getHadoopConfig
> private static final Configuration CONF = new Configuration();
> private static final Groups GROUPS =
> Groups.getUserToGroupsMappingService(CONF);
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
