[jira] [Comment Edited] (HADOOP-12862) LDAP Group Mapping over SSL can not specify trust store
[
https://issues.apache.org/jira/browse/HADOOP-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16419252#comment-16419252
]
Zhe Zhang edited comment on HADOOP-12862 at 3/29/18 4:03 PM:
-
Thanks [~shv]. v9 patch LGTM. +1
was (Author: zhz):
Thanks [~shv]. +1 on v9 patch.
> LDAP Group Mapping over SSL can not specify trust store
> ---
>
> Key: HADOOP-12862
> URL: https://issues.apache.org/jira/browse/HADOOP-12862
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: release-blocker
> Attachments: HADOOP-12862.001.patch, HADOOP-12862.002.patch,
> HADOOP-12862.003.patch, HADOOP-12862.004.patch, HADOOP-12862.005.patch,
> HADOOP-12862.006.patch, HADOOP-12862.007.patch, HADOOP-12862.008.patch,
> HADOOP-12862.009.patch
>
>
> In a secure environment, SSL is used to encrypt LDAP request for group
> mapping resolution.
> We (+[~yoderme], +[~tgrayson]) have found that its implementation is strange.
> For information, Hadoop name node, as an LDAP client, talks to a LDAP server
> to resolve the group mapping of a user. In the case of LDAP over SSL, a
> typical scenario is to establish one-way authentication (the client verifies
> the server's certificate is real) by storing the server's certificate in the
> client's truststore.
> A rarer scenario is to establish two-way authentication: in addition to store
> truststore for the client to verify the server, the server also verifies the
> client's certificate is real, and the client stores its own certificate in
> its keystore.
> However, the current implementation for LDAP over SSL does not seem to be
> correct in that it only configures keystore but no truststore (so LDAP server
> can verify Hadoop's certificate, but Hadoop may not be able to verify LDAP
> server's certificate)
> I think there should an extra pair of properties to specify the
> truststore/password for LDAP server, and use that to configure system
> properties {{javax.net.ssl.trustStore}}/{{javax.net.ssl.trustStorePassword}}
> I am a security layman so my words can be imprecise. But I hope this makes
> sense.
> Oracle's SSL LDAP documentation:
> http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
> JSSE reference guide:
> http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12862) LDAP Group Mapping over SSL can not specify trust store
[
https://issues.apache.org/jira/browse/HADOOP-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414581#comment-16414581
]
Wei-Chiu Chuang edited comment on HADOOP-12862 at 3/26/18 9:23 PM:
---
Hi Konstantin, There's probably a misunderstanding here.
I fully agree that storing passwords in config files is a bad thing,
so I proposed in HADOOP-15325 to add an option to NOT getting passwords in the
config. Once HADOOP-15325 is in, I can update this patch and that it won't read
passwords from config files, but from credential files.
Or, make Configuration#getPasswordFromCredentialProviders() public instead of
protected. That way I can update this patch to call
Configuration#getPasswordFromCredentialProviders() to read from credential
files.
was (Author: jojochuang):
Hi Konstantin, There's probably a misunderstanding here.
I fully agree that storing passwords in config files is a bad thing,
so I proposed in HADOOP-15325 to add an option to NOT getting passwords in the
config. Once HADOOP-15325 is in, I can update this patch and that it won't read
passwords from config files, but from credential files.
> LDAP Group Mapping over SSL can not specify trust store
> ---
>
> Key: HADOOP-12862
> URL: https://issues.apache.org/jira/browse/HADOOP-12862
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-12862.001.patch, HADOOP-12862.002.patch,
> HADOOP-12862.003.patch, HADOOP-12862.004.patch, HADOOP-12862.005.patch,
> HADOOP-12862.006.patch, HADOOP-12862.007.patch, HADOOP-12862.008.patch
>
>
> In a secure environment, SSL is used to encrypt LDAP request for group
> mapping resolution.
> We (+[~yoderme], +[~tgrayson]) have found that its implementation is strange.
> For information, Hadoop name node, as an LDAP client, talks to a LDAP server
> to resolve the group mapping of a user. In the case of LDAP over SSL, a
> typical scenario is to establish one-way authentication (the client verifies
> the server's certificate is real) by storing the server's certificate in the
> client's truststore.
> A rarer scenario is to establish two-way authentication: in addition to store
> truststore for the client to verify the server, the server also verifies the
> client's certificate is real, and the client stores its own certificate in
> its keystore.
> However, the current implementation for LDAP over SSL does not seem to be
> correct in that it only configures keystore but no truststore (so LDAP server
> can verify Hadoop's certificate, but Hadoop may not be able to verify LDAP
> server's certificate)
> I think there should an extra pair of properties to specify the
> truststore/password for LDAP server, and use that to configure system
> properties {{javax.net.ssl.trustStore}}/{{javax.net.ssl.trustStorePassword}}
> I am a security layman so my words can be imprecise. But I hope this makes
> sense.
> Oracle's SSL LDAP documentation:
> http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
> JSSE reference guide:
> http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12862) LDAP Group Mapping over SSL can not specify trust store
[
https://issues.apache.org/jira/browse/HADOOP-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403075#comment-16403075
]
Konstantin Shvachko edited comment on HADOOP-12862 at 3/22/18 9:40 PM:
---
Sounds like testing is a longer-term issue. BTW if I look into Hadoop
dependencies I see apacheds and ldapsdk. May they be useful for testing. I
wouldn't know.
The testing went well in our environment.
What do you think about removing {{.ssl.truststore.password}}? I really think
people should not use configs for passwords. Typically configs are checked in
in git repositories, so having passwords there is even worse than printing them
on a command line, which [as you
suggested|https://issues.apache.org/jira/browse/HADOOP-15315?focusedCommentId=16399166=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16399166]
is a bad practice.
was (Author: shv):
Sounds like testing is a longer-term issue. BTW if I look into Hadoop
dependencies I see apacheds and ldapsdk. May they be useful for testing. I
wouldn't know.
The testing went well in our environment.
What do you think about removing {{.ssl.truststore.password}}? I really think
people should not use configs for passwords. Typically configs are checked in
in git repositories, so having passwords there is even worth than printing them
on a command line, which [as you
suggested|https://issues.apache.org/jira/browse/HADOOP-15315?focusedCommentId=16399166=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16399166]
is a bad practice.
> LDAP Group Mapping over SSL can not specify trust store
> ---
>
> Key: HADOOP-12862
> URL: https://issues.apache.org/jira/browse/HADOOP-12862
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-12862.001.patch, HADOOP-12862.002.patch,
> HADOOP-12862.003.patch, HADOOP-12862.004.patch, HADOOP-12862.005.patch,
> HADOOP-12862.006.patch, HADOOP-12862.007.patch, HADOOP-12862.008.patch
>
>
> In a secure environment, SSL is used to encrypt LDAP request for group
> mapping resolution.
> We (+[~yoderme], +[~tgrayson]) have found that its implementation is strange.
> For information, Hadoop name node, as an LDAP client, talks to a LDAP server
> to resolve the group mapping of a user. In the case of LDAP over SSL, a
> typical scenario is to establish one-way authentication (the client verifies
> the server's certificate is real) by storing the server's certificate in the
> client's truststore.
> A rarer scenario is to establish two-way authentication: in addition to store
> truststore for the client to verify the server, the server also verifies the
> client's certificate is real, and the client stores its own certificate in
> its keystore.
> However, the current implementation for LDAP over SSL does not seem to be
> correct in that it only configures keystore but no truststore (so LDAP server
> can verify Hadoop's certificate, but Hadoop may not be able to verify LDAP
> server's certificate)
> I think there should an extra pair of properties to specify the
> truststore/password for LDAP server, and use that to configure system
> properties {{javax.net.ssl.trustStore}}/{{javax.net.ssl.trustStorePassword}}
> I am a security layman so my words can be imprecise. But I hope this makes
> sense.
> Oracle's SSL LDAP documentation:
> http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
> JSSE reference guide:
> http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12862) LDAP Group Mapping over SSL can not specify trust store
[
https://issues.apache.org/jira/browse/HADOOP-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16401815#comment-16401815
]
Wei-Chiu Chuang edited comment on HADOOP-12862 at 3/16/18 12:31 PM:
Thank you for your testing effort [~shv]. I've previously manually tested the
code against a CDH5.13 cluster. Won't hurt to have more people test it in more
environments.
I proposed adding apache directory service libs so we could test against a mini
LDAP server a while back. But didn't seem to get much traction. The other way
(lighter weight) is to include Kerby. But I've not studied it further.
[~drankye] or [~jiajia] any idea if we could include Kerby for unit testing
LDAP related code?
was (Author: jojochuang):
Thank you for your testing effort [~shv]. I've previously manually tested the
code against a CDH5.13 cluster. Won't hurt to have more people test it in more
environments.
I proposed adding apache directory service libs so we could test against a mini
LDAP server a while back. But didn't seem to get much traction. The other way
(lighter weight) is to include Kerby. But I've not studied it further.
[~drankye] or [~jiajia] any idea if we could include Kerby unit test purposes
for LDAP related code?
> LDAP Group Mapping over SSL can not specify trust store
> ---
>
> Key: HADOOP-12862
> URL: https://issues.apache.org/jira/browse/HADOOP-12862
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Attachments: HADOOP-12862.001.patch, HADOOP-12862.002.patch,
> HADOOP-12862.003.patch, HADOOP-12862.004.patch, HADOOP-12862.005.patch,
> HADOOP-12862.006.patch, HADOOP-12862.007.patch, HADOOP-12862.008.patch
>
>
> In a secure environment, SSL is used to encrypt LDAP request for group
> mapping resolution.
> We (+[~yoderme], +[~tgrayson]) have found that its implementation is strange.
> For information, Hadoop name node, as an LDAP client, talks to a LDAP server
> to resolve the group mapping of a user. In the case of LDAP over SSL, a
> typical scenario is to establish one-way authentication (the client verifies
> the server's certificate is real) by storing the server's certificate in the
> client's truststore.
> A rarer scenario is to establish two-way authentication: in addition to store
> truststore for the client to verify the server, the server also verifies the
> client's certificate is real, and the client stores its own certificate in
> its keystore.
> However, the current implementation for LDAP over SSL does not seem to be
> correct in that it only configures keystore but no truststore (so LDAP server
> can verify Hadoop's certificate, but Hadoop may not be able to verify LDAP
> server's certificate)
> I think there should an extra pair of properties to specify the
> truststore/password for LDAP server, and use that to configure system
> properties {{javax.net.ssl.trustStore}}/{{javax.net.ssl.trustStorePassword}}
> I am a security layman so my words can be imprecise. But I hope this makes
> sense.
> Oracle's SSL LDAP documentation:
> http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
> JSSE reference guide:
> http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-12862) LDAP Group Mapping over SSL can not specify trust store
[
https://issues.apache.org/jira/browse/HADOOP-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16136258#comment-16136258
]
Wei-Chiu Chuang edited comment on HADOOP-12862 at 8/22/17 4:48 AM:
---
Hello, trying to bump up this patch.
I have tested the last patch against Cloudera's latest CDH version. Our
internal LDAP server is self signed so Hadoop needs to specify a trust store.
Without this patch, LdapGroupsMapping fails with SSLHandshakeException error.
{quote}
2017-08-21 21:07:51,517 WARN org.apache.hadoop.security.LdapGroupsMapping:
Failed to get groups for user mapred (retry=2) by
javax.naming.CommunicationException
: simple bind failed:.cloudera.com:636 [Root exception is
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException:
PKIX
path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target]
{quote}
After applying the patch, groups mapping works successfully.
was (Author: jojochuang):
Hello, trying to bump up this patch.
I have tested the last patch against Cloudera's latest CDH version. Our
internal LDAP server is self signed so Hadoop needs to specify a trust store.
Without this patch, LdapGroupsMapping fails with SSLHandshakeException error.
{quote}
2017-08-21 21:07:51,517 WARN org.apache.hadoop.security.LdapGroupsMapping:
Failed to get groups for user mapred (retry=2) by
javax.naming.CommunicationException
: simple bind failed: scale-ad.ad.halxg.cloudera.com:636 [Root exception is
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException:
PKIX
path building failed:
sun.security.provider.certpath.SunCertPathBuilderException: unable to find
valid certification path to requested target]
{quote}
After applying the patch, groups mapping works successfully.
> LDAP Group Mapping over SSL can not specify trust store
> ---
>
> Key: HADOOP-12862
> URL: https://issues.apache.org/jira/browse/HADOOP-12862
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
> Attachments: HADOOP-12862.001.patch, HADOOP-12862.002.patch,
> HADOOP-12862.003.patch, HADOOP-12862.004.patch, HADOOP-12862.005.patch,
> HADOOP-12862.006.patch, HADOOP-12862.007.patch
>
>
> In a secure environment, SSL is used to encrypt LDAP request for group
> mapping resolution.
> We (+[~yoderme], +[~tgrayson]) have found that its implementation is strange.
> For information, Hadoop name node, as an LDAP client, talks to a LDAP server
> to resolve the group mapping of a user. In the case of LDAP over SSL, a
> typical scenario is to establish one-way authentication (the client verifies
> the server's certificate is real) by storing the server's certificate in the
> client's truststore.
> A rarer scenario is to establish two-way authentication: in addition to store
> truststore for the client to verify the server, the server also verifies the
> client's certificate is real, and the client stores its own certificate in
> its keystore.
> However, the current implementation for LDAP over SSL does not seem to be
> correct in that it only configures keystore but no truststore (so LDAP server
> can verify Hadoop's certificate, but Hadoop may not be able to verify LDAP
> server's certificate)
> I think there should an extra pair of properties to specify the
> truststore/password for LDAP server, and use that to configure system
> properties {{javax.net.ssl.trustStore}}/{{javax.net.ssl.trustStorePassword}}
> I am a security layman so my words can be imprecise. But I hope this makes
> sense.
> Oracle's SSL LDAP documentation:
> http://docs.oracle.com/javase/jndi/tutorial/ldap/security/ssl.html
> JSSE reference guide:
> http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
