[
https://issues.apache.org/jira/browse/HADOOP-18066?focusedWorklogId=705957&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-705957
]
ASF GitHub Bot logged work on HADOOP-18066:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 10/Jan/22 10:28
Start Date: 10/Jan/22 10:28
Worklog Time Spent: 10m
Work Description: abstractdog commented on pull request #3865:
URL: https://github.com/apache/hadoop/pull/3865#issuecomment-1008732430
@lmccay , @steveloughran : thanks for your comments
TLDR: finally I made this work on hive side (HIVE-25829) without this
workaround, so I'll close this PR and jira
I understand that password in conf object is against design, because, for
instance, it clearly states that a single password should be used for the
providers
when I started to work on this workaround, I was confused about HIVE-14822,
which looks like messing up the hadoop design by introducing more than 1
credential stores with different passwords (1 for hiveserver2, 1 to distribute
on cluster for mr/spark/tez jobs), however it's not really the case
after fixing my code, I properly use HADOOP_CREDSTORE_PASSWORD in
hiveserver2, and propagate the other password (together with the distributed
provider path) to the jobs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 705957)
Time Spent: 2.5h (was: 2h 20m)
> AbstractJavaKeyStoreProvider: need a way to read credential store password
> from Configuration
> ---------------------------------------------------------------------------------------------
>
> Key: HADOOP-18066
> URL: https://issues.apache.org/jira/browse/HADOOP-18066
> Project: Hadoop Common
> Issue Type: Wish
> Components: security
> Reporter: László Bodor
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.2
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Codepath in focus is
> [this|https://github.com/apache/hadoop/blob/c3006be516ce7d4f970e24e7407b401318ceec3c/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/alias/AbstractJavaKeyStoreProvider.java#L316]
> {code}
> password = ProviderUtils.locatePassword(CREDENTIAL_PASSWORD_ENV_VAR,
> conf.get(CREDENTIAL_PASSWORD_FILE_KEY));
> {code}
> Since HIVE-14822, we can use custom keystore that Hiveserver2 propagates to
> jobs/tasks of different execution engines (mr, tez, spark).
> We're able to pass any "jceks:" url, but not a password, e.g. on this
> codepath:
> {code}
> Caused by: java.security.UnrecoverableKeyException: Password verification
> failed
> at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:879)
> ~[sunjce_provider.jar:1.8.0_232]
> at java.security.KeyStore.load(KeyStore.java:1445) ~[?:1.8.0_232]
> at
> org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.locateKeystore(AbstractJavaKeyStoreProvider.java:326)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:86)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.security.alias.KeyStoreProvider.<init>(KeyStoreProvider.java:49)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:42)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:35)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:68)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2409)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2347)
> ~[hadoop-common-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getPasswordString(AbfsConfiguration.java:295)
> ~[hadoop-azure-3.1.1.7.1.7.0-551.jar:?]
> at
> org.apache.hadoop.fs.azurebfs.AbfsConfiguration.getTokenProvider(AbfsConfiguration.java:525)
> ~[hadoop-azure-3.1.1.7.1.7.0-551.jar:?]
> {code}
> Even there is a chance of reading a text file, it's not secure, we need to
> try reading a Configuration property first and if it's null, we can go to the
> environment variable.
> Hacking the System.getenv() is only possible with reflection, doesn't look so
> good.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
