subject:"\[jira\] \[Commented\] \(HADOOP\-16245\) Enabling SSL within LdapGroupsMapping can break system SSL configs"

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-26 Thread Hudson (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894101#comment-16894101
 ] 

Hudson commented on HADOOP-16245:
-

FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #16993 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/16993/])
HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL (xkrogen: rev 
62efb6300619670d3e0554c3ba14c264fa0c705b)
* (edit) 
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/LdapGroupsMapping.java


> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 3.1.3
>
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch, HADOOP-16245.003.patch, HADOOP-16245.004.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-26 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894055#comment-16894055
 ] 

Erik Krogen commented on HADOOP-16245:
--

Thanks [~vagarychen]! just committed this to trunk, branch-3.2, branch-3.1, 
branch-3.0, branch-2.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch, HADOOP-16245.003.patch, HADOOP-16245.004.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-26 Thread Chen Liang (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16894024#comment-16894024
 ] 

Chen Liang commented on HADOOP-16245:
-

Sorry for the delay, v004 patch LGTM, +1.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch, HADOOP-16245.003.patch, HADOOP-16245.004.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-24 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16892093#comment-16892093
 ] 

Hadoop QA commented on HADOOP-16245:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
56s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m  1s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 30s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m  
0s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
40s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}108m 53s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16245 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12975665/HADOOP-16245.004.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux d1d93b08ec29 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / cf9ff08 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16419/testReport/ |
| Max. process+thread count | 1388 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16419/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Enabling SSL within

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-24 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891927#comment-16891927
 ] 

Erik Krogen commented on HADOOP-16245:
--

Attached v004 patch to address checkstyle/whitespace issues.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch, HADOOP-16245.003.patch, HADOOP-16245.004.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-23 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891541#comment-16891541
 ] 

Hadoop QA commented on HADOOP-16245:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m 
12s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m 
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m 59s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 18m  
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 49s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch 
generated 4 new + 21 unchanged - 0 fixed = 25 total (was 21) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m  1s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  9m  
8s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
47s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}119m 31s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16245 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12975563/HADOOP-16245.003.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 435696035ae6 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3 
08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ecb9f81 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16412/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
 |
| whitespace | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16412/artifact/out/whitespace-eol.txt
 |
|  Test Results |

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-23 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891402#comment-16891402
 ] 

Erik Krogen commented on HADOOP-16245:
--

Thanks [~jojochuang], I agree with your assessment.

Spent a little more time testing this today, and realize that while the tests 
we ran previously verified that the LDAP SSL configs no longer broke other 
system-wide SSL configs, LDAP SSL also wasn't working properly with the v002 
patch; the class name being passed via reflection was wrong, and a few APIs on 
{{SocketFactory}} were not implemented as expected by the Java naming services. 
I've fixed these issues, and _actually_ verified that this solves the issue 
without breaking LDAP SSL, in the v003 patch.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch, HADOOP-16245.003.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Wei-Chiu Chuang (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890555#comment-16890555
 ] 

Wei-Chiu Chuang commented on HADOOP-16245:
--

After this patch, a user won't be able to configure LDAP keystore/truststore 
using Java command line options javax.net.ssl.keyStore / 
javax.net.ssl.trustStore. But this is nor a recommend way (not secure) nor is 
ever explicitly supported. I think we are good.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890531#comment-16890531
 ] 

Hadoop QA commented on HADOOP-16245:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  1m 
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 21m 
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
16m 37s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 20m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 20m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m  6s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m 
10s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
59s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}125m  1s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.7 Server=18.09.7 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16245 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12975438/HADOOP-16245.002.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux d53716be4e95 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4 
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / cdc36fe |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16405/testReport/ |
| Max. process+thread count | 1747 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16405/console |
| Powered by | Apache Yetus 0.8.0   http://yetus.apache.org |


This message was automatically generated.



> Enabling SSL within

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Chen Liang (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890476#comment-16890476
 ] 

Chen Liang commented on HADOOP-16245:
-

Thanks Erik, +1 to v002 patch.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890471#comment-16890471
 ] 

Erik Krogen commented on HADOOP-16245:
--

Thanks [~vagarychen]! I expanded the description a bit more. Let me know what 
you think.

I was unable to reproduce the TestIPC failure locally; I believe it is 
unrelated.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch, 
> HADOOP-16245.002.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890426#comment-16890426
 ] 

Hadoop QA commented on HADOOP-16245:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 59s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m  9s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}  8m 44s{color} 
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
38s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 90m 40s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ipc.TestIPC |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16245 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12975423/HADOOP-16245.001.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux c96e7a4763c4 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 340bbaf |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16404/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16404/testReport/ |
| Max. process+thread count | 1402 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U: 
hadoop-common-project/hadoop-common |
| Console output |

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Chen Liang (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890353#comment-16890353
 ] 

Chen Liang commented on HADOOP-16245:
-

Thanks for fixing this. v001 patch LGTM.

Just one nit: For future references, maybe we should document with more detail 
on why we are doing it this way. e.g. why a static class is enough in this case 
(like you mentioned); the static class has the same behaviour as the 
SSLSocketFactory class (based on my understanding); new code should be aware of 
the static fields so what Daryn mentioned could be avoided for future code 
changes. With this addressed, +1 from me.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-07-22 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890292#comment-16890292
 ] 

Erik Krogen commented on HADOOP-16245:
--

I've now tested this on one of our live clusters and confirmed that I was able 
to configure {{LdapGroupsMapping}} without negatively impacting other SSL 
connections, fixing the issue discussed here.

I rebased the patch and cleaned up the documentation for v001. I think it 
should be ready for commit now. I'm can't think of a good way to test this in a 
unit test so I haven't added one for now.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch, HADOOP-16245.001.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-16 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819245#comment-16819245
 ] 

Erik Krogen commented on HADOOP-16245:
--

I agree that storing the values in a static class is not ideal, but I was 
unable to find any other way. Looking at the definition of 
[{{SocketFactory}}|https://docs.oracle.com/javase/8/docs/api/index.html?javax/net/SocketFactory.html],
 I don't see any possible way for JNDI to pass in a context. I also couldn't 
find any examples that did anything better than what I'm doing here. But if 
anyone has ideas, please chime in.

I think in this case it's safe to use the static configurations, given that the 
only place in which {{LdapGroupsMappingService}} is used is from the context of 
{{Groups}}, which is a singleton (stored in the static field 
{{Groups#GROUPS}}). Even if you call {{getUserToGroupsMappingService()}} with a 
new {{Configuration}}, in which case you might expect it to construct a new 
{{Groups}}, it simply returns the old one.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-16 Thread Daryn Sharp (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819236#comment-16819236
 ] 

Daryn Sharp commented on HADOOP-16245:
--

Storing the SSL parameters as statics in a static class is better but not much 
safer than in properties.  If you create multiple instances with different 
parameters, the last instance's parameters win and the other instances use 
those.  I've not messed with the JNDI stuff before.  Are you sure it won't pass 
the context to the factory?

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-16 Thread Wei-Chiu Chuang (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819212#comment-16819212
 ] 

Wei-Chiu Chuang commented on HADOOP-16245:
--

Yeah you're right about that. Also currently I don't think LdapGroupsMapping 
allow you to specify enabled protocol/algorithms (unless you explicitly set 
java properties -Djdk.tls.client.protocols), so I'm okay if that's not part of 
this jra.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-16 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819183#comment-16819183
 ] 

Erik Krogen commented on HADOOP-16245:
--

I thought about how to leverage the code in {{SSLFactory}} but I wasn't sure 
how to do so, given that {{SSLFactory}} uses configurations from a separate 
file entirely ({{ssl-client.xml}} / {{ssl-server.xml}} by default) whereas the 
LDAP SSL configurations are all specified in the standard Configuration 
locations. We could create a new {{ssl-ldap.xml}}, but this seems like it might 
be overkill, and we would still need to support the old configurations for 
backwards compatibility. But I'm open to suggestions on how to better combine 
them; I'll also think more about it.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-16 Thread Wei-Chiu Chuang (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819032#comment-16819032
 ] 

Wei-Chiu Chuang commented on HADOOP-16245:
--

Hi [~xkrogen] thanks for reporting the issue. 

I suspect you'll also want to specify enabled SSL protocols and algorithms. In 
fact, you may want to reuse some code in 
org.apache.hadoop.security.ssl.SSLFactory so you don't create additional 
configuration parameters for LDAPS connections.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-12 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816654#comment-16816654
 ] 

Hadoop QA commented on HADOOP-16245:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m 
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m  3s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m  
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m  
8s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 53s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch 
generated 1 new + 21 unchanged - 0 fixed = 22 total (was 21) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m 32s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  9m 
11s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
45s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}101m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:bdbca0e |
| JIRA Issue | HADOOP-16245 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12965749/HADOOP-16245.000.patch
 |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux ef29bc40103c 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0c1fec3 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16148/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HADOOP-Build/16148/testReport/ |
| Max. process+thread count | 1347 (vs. ulimit of 1) |
| modules | C: hadoop-common-project/hadoop-common U:

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

2019-04-12 Thread Erik Krogen (JIRA)



[ 
https://issues.apache.org/jira/browse/HADOOP-16245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816588#comment-16816588
 ] 

Erik Krogen commented on HADOOP-16245:
--

Attaching a v000 patch for initial review while we start doing some testing 
internally to make sure we didn't break anything.

> Enabling SSL within LdapGroupsMapping can break system SSL configs
> --
>
> Key: HADOOP-16245
> URL: https://issues.apache.org/jira/browse/HADOOP-16245
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, security
>Affects Versions: 2.9.1, 2.8.4, 2.7.6, 3.1.1, 3.0.3
>Reporter: Erik Krogen
>Assignee: Erik Krogen
>Priority: Major
> Attachments: HADOOP-16245.000.patch
>
>
> When debugging an issue where one of our server components was unable to 
> communicate with other components via SSL, we realized that LdapGroupsMapping 
> sets its SSL configurations globally, rather than scoping them to the HTTP 
> clients it creates.
> {code:title=LdapGroupsMapping}
>   DirContext getDirContext() throws NamingException {
> if (ctx == null) {
>   // Set up the initial environment for LDAP connectivity
>   Hashtable env = new Hashtable();
>   env.put(Context.INITIAL_CONTEXT_FACTORY,
>   com.sun.jndi.ldap.LdapCtxFactory.class.getName());
>   env.put(Context.PROVIDER_URL, ldapUrl);
>   env.put(Context.SECURITY_AUTHENTICATION, "simple");
>   // Set up SSL security, if necessary
>   if (useSsl) {
> env.put(Context.SECURITY_PROTOCOL, "ssl");
> if (!keystore.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStore", keystore);
> }
> if (!keystorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.keyStorePassword", keystorePass);
> }
> if (!truststore.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStore", truststore);
> }
> if (!truststorePass.isEmpty()) {
>   System.setProperty("javax.net.ssl.trustStorePassword",
>   truststorePass);
> }
>   }
>   env.put(Context.SECURITY_PRINCIPAL, bindUser);
>   env.put(Context.SECURITY_CREDENTIALS, bindPassword);
>   env.put("com.sun.jndi.ldap.connect.timeout", 
> conf.get(CONNECTION_TIMEOUT,
>   String.valueOf(CONNECTION_TIMEOUT_DEFAULT)));
>   env.put("com.sun.jndi.ldap.read.timeout", conf.get(READ_TIMEOUT,
>   String.valueOf(READ_TIMEOUT_DEFAULT)));
>   ctx = new InitialDirContext(env);
> }
> {code}
> Notice the {{System.setProperty()}} calls, which will change settings 
> JVM-wide. This causes issues for other SSL clients, which may rely on the 
> default JVM truststore being used. This behavior was initially introduced by 
> HADOOP-8121, and extended to include the truststore configurations in 
> HADOOP-12862.
> The correct approach is to use a mechanism which is scoped to the LDAP 
> requests only. The right approach appears to be to use the 
> {{java.naming.ldap.factory.socket}} parameter to set the socket factory to a 
> custom SSL socket factory which correctly sets the key and trust store 
> parameters. See an example [here|https://stackoverflow.com/a/4615497/4979203].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

[jira] [Commented] (HADOOP-16245) Enabling SSL within LdapGroupsMapping can break system SSL configs

21 matches

Site Navigation

Mail list logo

Footer information