[jira] [Comment Edited] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs
[
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891402#comment-16891402
]
Erik Krogen edited comment on HADOOP-16245 at 7/23/19 9:43 PM:
---
Thanks [~jojochuang], I agree with your assessment.
Spent a little more time testing this today, and realize that while the tests
we ran previously verified that the LDAP SSL configs no longer broke other
system-wide SSL configs, LDAP SSL also wasn't working properly with the v002
patch; the class name being passed via reflection was wrong, and a few APIs on
{{SocketFactory}} were not implemented as expected by the Java naming services.
I've fixed these issues, and _actually_ verified that this solves the issue
without breaking LDAP SSL, in the v003 patch. [~vagarychen], can you take
another look?
was (Author: xkrogen):
Thanks [~jojochuang], I agree with your assessment.
Spent a little more time testing this today, and realize that while the tests
we ran previously verified that the LDAP SSL configs no longer broke other
system-wide SSL configs, LDAP SSL also wasn't working properly with the v002
patch; the class name being passed via reflection was wrong, and a few APIs on
{{SocketFactory}} were not implemented as expected by the Java naming services.
I've fixed these issues, and _actually_ verified that this solves the issue
without breaking LDAP SSL, in the v003 patch.
> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch,
> HADOOP-16245.002.patch, HADOOP-16245.003.patch
>
>
> When debugging an issue where one of our server components was unable to
> communicate with other components via SSL, we realized that LdapGroupsMapping
> sets its SSL configurations globally, rather than scoping them to the HTTP
> clients it creates.
> {code:title=LdapGroupsMapping}
> DirContext getDirContext() throws NamingException {
> if (ctx == null) {
> // Set up the initial environment for LDAP connectivity
> Hashtable env = new Hashtable();
> env.put(Context.INITIAL_CONTEXT_FACTORY,
> com.sun.jndi.ldap.LdapCtxFactory.class.getName());
> env.put(Context.PROVIDER_URL, ldapUrl);
> env.put(Context.SECURITY_AUTHENTICATION, "simple");
> // Set up SSL security, if necessary
> if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStorePassword",
> truststorePass);
> }
> }
> env.put(Context.SECURITY_PRINCIPAL, bindUser);
> env.put(Context.SECURITY_CREDENTIALS, bindPassword);
> env.put("com.sun.jndi.ldap.connect.timeout",
> conf.get(CONNECTION_TIMEOUT,
> String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
> env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
> String.valueOf(READ_TIMEOUT_DEFAULT)));
> ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings
> JVM-wide. This causes issues for other SSL clients, which may rely on the
> default JVM truststore being used. This behavior was initially introduced by
> HADOOP-8121, and extended to include the truststore configurations in
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP
> requests only. The right approach appears to be to use the
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a
> custom SSL socket factory which correctly sets the key and trust store
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs
[
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891402#comment-16891402
]
Erik Krogen edited comment on HADOOP-16245 at 7/23/19 9:43 PM:
---
Thanks [~jojochuang], I agree with your assessment.
Spent a little more time testing this today, and realized that while the tests
we ran previously verified that the LDAP SSL configs no longer broke other
system-wide SSL configs, LDAP SSL also wasn't working properly with the v002
patch. The class name being passed via reflection was wrong, and a few APIs on
{{SocketFactory}} were not implemented as expected by the Java naming services.
I've fixed these issues, and _actually_ verified that this solves the issue
without breaking LDAP SSL, in the v003 patch. [~vagarychen], can you take
another look?
was (Author: xkrogen):
Thanks [~jojochuang], I agree with your assessment.
Spent a little more time testing this today, and realize that while the tests
we ran previously verified that the LDAP SSL configs no longer broke other
system-wide SSL configs, LDAP SSL also wasn't working properly with the v002
patch; the class name being passed via reflection was wrong, and a few APIs on
{{SocketFactory}} were not implemented as expected by the Java naming services.
I've fixed these issues, and _actually_ verified that this solves the issue
without breaking LDAP SSL, in the v003 patch. [~vagarychen], can you take
another look?
> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch,
> HADOOP-16245.002.patch, HADOOP-16245.003.patch
>
>
> When debugging an issue where one of our server components was unable to
> communicate with other components via SSL, we realized that LdapGroupsMapping
> sets its SSL configurations globally, rather than scoping them to the HTTP
> clients it creates.
> {code:title=LdapGroupsMapping}
> DirContext getDirContext() throws NamingException {
> if (ctx == null) {
> // Set up the initial environment for LDAP connectivity
> Hashtable env = new Hashtable();
> env.put(Context.INITIAL_CONTEXT_FACTORY,
> com.sun.jndi.ldap.LdapCtxFactory.class.getName());
> env.put(Context.PROVIDER_URL, ldapUrl);
> env.put(Context.SECURITY_AUTHENTICATION, "simple");
> // Set up SSL security, if necessary
> if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStorePassword",
> truststorePass);
> }
> }
> env.put(Context.SECURITY_PRINCIPAL, bindUser);
> env.put(Context.SECURITY_CREDENTIALS, bindPassword);
> env.put("com.sun.jndi.ldap.connect.timeout",
> conf.get(CONNECTION_TIMEOUT,
> String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
> env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
> String.valueOf(READ_TIMEOUT_DEFAULT)));
> ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings
> JVM-wide. This causes issues for other SSL clients, which may rely on the
> default JVM truststore being used. This behavior was initially introduced by
> HADOOP-8121, and extended to include the truststore configurations in
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP
> requests only. The right approach appears to be to use the
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a
> custom SSL socket factory which correctly sets the key and trust store
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs
[
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890555#comment-16890555
]
Wei-Chiu Chuang edited comment on HADOOP-16245 at 7/22/19 11:57 PM:
After this patch, a user won't be able to configure LDAP keystore/truststore
using Java command line options javax.net.ssl.keyStore /
javax.net.ssl.trustStore. But this is neither a recommend way (not secure) nor
is ever explicitly supported. I think we are good.
was (Author: jojochuang):
After this patch, a user won't be able to configure LDAP keystore/truststore
using Java command line options javax.net.ssl.keyStore /
javax.net.ssl.trustStore. But this is nor a recommend way (not secure) nor is
ever explicitly supported. I think we are good.
> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch,
> HADOOP-16245.002.patch
>
>
> When debugging an issue where one of our server components was unable to
> communicate with other components via SSL, we realized that LdapGroupsMapping
> sets its SSL configurations globally, rather than scoping them to the HTTP
> clients it creates.
> {code:title=LdapGroupsMapping}
> DirContext getDirContext() throws NamingException {
> if (ctx == null) {
> // Set up the initial environment for LDAP connectivity
> Hashtable env = new Hashtable();
> env.put(Context.INITIAL_CONTEXT_FACTORY,
> com.sun.jndi.ldap.LdapCtxFactory.class.getName());
> env.put(Context.PROVIDER_URL, ldapUrl);
> env.put(Context.SECURITY_AUTHENTICATION, "simple");
> // Set up SSL security, if necessary
> if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStorePassword",
> truststorePass);
> }
> }
> env.put(Context.SECURITY_PRINCIPAL, bindUser);
> env.put(Context.SECURITY_CREDENTIALS, bindPassword);
> env.put("com.sun.jndi.ldap.connect.timeout",
> conf.get(CONNECTION_TIMEOUT,
> String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
> env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
> String.valueOf(READ_TIMEOUT_DEFAULT)));
> ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings
> JVM-wide. This causes issues for other SSL clients, which may rely on the
> default JVM truststore being used. This behavior was initially introduced by
> HADOOP-8121, and extended to include the truststore configurations in
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP
> requests only. The right approach appears to be to use the
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a
> custom SSL socket factory which correctly sets the key and trust store
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs
[
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819183#comment-16819183
]
Erik Krogen edited comment on HADOOP-16245 at 4/16/19 4:12 PM:
---
I thought about how to leverage the code in {{SSLFactory}} but I wasn't sure
how to do so, given that {{SSLFactory}} uses configurations from a separate
file entirely ({{ssl-client.xml}} / {{ssl-server.xml}} by default) whereas the
LDAP SSL configurations are all specified in the standard Configuration
locations. We could create a new {{ssl-ldap.xml}}, but this seems like it might
be overkill, and we would still need to support the old configurations for
backwards compatibility. But I'm open to suggestions on how to better combine
them; I'll also think more about it.
I think it's also worth noting that, AFAIK, I'm just using the default
algorithms, which should match the same behavior prior to this patch.
was (Author: xkrogen):
I thought about how to leverage the code in {{SSLFactory}} but I wasn't sure
how to do so, given that {{SSLFactory}} uses configurations from a separate
file entirely ({{ssl-client.xml}} / {{ssl-server.xml}} by default) whereas the
LDAP SSL configurations are all specified in the standard Configuration
locations. We could create a new {{ssl-ldap.xml}}, but this seems like it might
be overkill, and we would still need to support the old configurations for
backwards compatibility. But I'm open to suggestions on how to better combine
them; I'll also think more about it.
> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
> Issue Type: Bug
> Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to
> communicate with other components via SSL, we realized that LdapGroupsMapping
> sets its SSL configurations globally, rather than scoping them to the HTTP
> clients it creates.
> {code:title=LdapGroupsMapping}
> DirContext getDirContext() throws NamingException {
> if (ctx == null) {
> // Set up the initial environment for LDAP connectivity
> Hashtable env = new Hashtable();
> env.put(Context.INITIAL_CONTEXT_FACTORY,
> com.sun.jndi.ldap.LdapCtxFactory.class.getName());
> env.put(Context.PROVIDER_URL, ldapUrl);
> env.put(Context.SECURITY_AUTHENTICATION, "simple");
> // Set up SSL security, if necessary
> if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
> System.setProperty("javax.net.ssl.trustStorePassword",
> truststorePass);
> }
> }
> env.put(Context.SECURITY_PRINCIPAL, bindUser);
> env.put(Context.SECURITY_CREDENTIALS, bindPassword);
> env.put("com.sun.jndi.ldap.connect.timeout",
> conf.get(CONNECTION_TIMEOUT,
> String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
> env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
> String.valueOf(READ_TIMEOUT_DEFAULT)));
> ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings
> JVM-wide. This causes issues for other SSL clients, which may rely on the
> default JVM truststore being used. This behavior was initially introduced by
> HADOOP-8121, and extended to include the truststore configurations in
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP
> requests only. The right approach appears to be to use the
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a
> custom SSL socket factory which correctly sets the key and trust store
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
