[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13810561#comment-13810561
]
Robert Kanter commented on HADOOP-8883:
---
I was talking to [~tucu00] about this, and he could go into more detail, but
from what I understand, the JDK doesn't quite implement the spec correctly, and
in some cases it will do SPNEGO when we weren't expecting it to yet. So, while
we can do it again (as the code always does now), that's wasteful and we can
just extract the token in that case (what the code is supposed to be doing).
In any case, you are correct that the first {{if}} block is never executed
because of the change introduced by this JIRA. I'll work on a fix for that and
create a new JIRA soon.
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[ https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13810713#comment-13810713 ] Robert Kanter commented on HADOOP-8883: --- I created HADOOP-10078 to fix the if statement. Anonymous fallback in KerberosAuthenticator is broken - Key: HADOOP-8883 URL: https://issues.apache.org/jira/browse/HADOOP-8883 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Robert Kanter Assignee: Robert Kanter Labels: security Fix For: 2.0.3-alpha Attachments: HADOOP-8883.patch HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the SPNEGO already; but this change broke using the fallback authenticator (PseudoAuthenticator) with an anonymous user (see OOZIE-1010). -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808231#comment-13808231
]
Robert Kanter commented on HADOOP-8883:
---
Alejandro told me offline that he checked and Oracle JDK 1.6 does actually
exclude the {{Authorization}} header as well. I know that the patch fixed the
problem we were having, so now I'm confused as to why it was even working...
I'll try to look into this.
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808241#comment-13808241
]
Andrey Klochkov commented on HADOOP-8883:
-
Robert,
{{TestKerberosAuthenticator#testFallbacktoPseudoAuthenticatorAnonymous}}
passes, but it doesn't verify the presence of the {{Authorization}} header, it
verifies _absence_ of it. Effectively the check
conn.getRequestProperty(AUTHORIZATION) != null would never succeed, so the
first block in the code below would never be used.
{code}
if (conn.getRequestProperty(AUTHORIZATION) != null conn.getResponseCode() ==
HttpURLConnection.HTTP_OK) {
LOG.debug(JDK performed authentication on our behalf.);
// If the JDK already did the SPNEGO back-and-forth for
// us, just pull out the token.
AuthenticatedURL.extractToken(conn, token);
return;
} else if (isNegotiate()) {
LOG.debug(Performing our own SPNEGO sequence.);
doSpnegoSequence(token);
} else {
LOG.debug(Using fallback authenticator sequence.);
getFallBackAuthenticator().authenticate(url, token);
}
{code}
Can you please give me more context on why it is needed to do manual
authorization, i.e. why JDK does not do the authentication every time, i.e. in
what cases {{KerberosAuthenticator.doSpnegoSequence}} should be used? I'm
interested in this as we're fighting with a related issue - in some
circumstances JDK just stops doing the authentication we rely on (it stops
caching tokens), and so all requests associated with {{HttpOpParam.Op}}
instances with {requreAuth==false}} start failing -- effectively, it's all
operations except {{GETDELEGATIONTOKEN}}. If
{{o.a.h.hdfs.web.URLConnectionFactory.openConnection}} used
{{KerberosAuthenticator}} for all operations (not just getting delegation
tokens), then we wouldn't have the problem. So I'm wondering who can shed some
light on how JDK is supposed to handle that, and in what cases
{{KerberosAuthenticator.doSpnegoSequence}} is really needed. Sorry, it's not
related directly to this issue, I'm just trying to find who can have answers.
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807596#comment-13807596
]
Andrey Klochkov commented on HADOOP-8883:
-
Actually JDK does not allow to read the content of Authorization request
property so this fix doesn't change the behavior.
Here's an extract from OpenJDK 1.7 sources. OpenJDK 1.6 is similar. My
experiments with Oracle JDK7 shows the same behavior (the property is not
available to the user).
{code}
249 // the following http request headers should NOT have their values
250 // returned for security reasons.
251 private static final String[] EXCLUDE_HEADERS = {
252 Proxy-Authorization,
253 Authorization
254 };
2709 @Override
2710 public synchronized String getRequestProperty (String key) {
2711 if (key == null) {
2712 return null;
2713 }
2714
2715 // don't return headers containing security sensitive
information
2716 for (int i=0; i EXCLUDE_HEADERS.length; i++) {
2717 if (key.equalsIgnoreCase(EXCLUDE_HEADERS[i])) {
2718 return null;
2719 }
2720 }
{code}
Should this Jira be re-opened or another one created?
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807696#comment-13807696
]
Robert Kanter commented on HADOOP-8883:
---
Looking at that code snippet, it does indeed seem like that the
{{Authorization}} header would be excluded and return {{null}}. But then it
seems weird that this had fixed the problem. There's a unit test and we saw it
fix the issue in OOZIE-1010.
Is it possible that OpenJDK 1.7, OpenJDK 1.6, and Oracle JDK 7 exclude the
header but Oracle JDK 6 does not? If so, then this could be a JDK
compatibility issue, and we should create a new JIRA to figure out a new way of
fixing this.
Can you check if the unit test in the patch
{{TestKerberosAuthenticator#testFallbacktoPseudoAuthenticatorAnonymous}} fails
on OpenJDK 1.7, OpenJDK 1.6, or Oracle JDK 7? I'm sure it passes on Oracle JDK
6.
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[ https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1341#comment-1341 ] Hudson commented on HADOOP-8883: Integrated in Hadoop-Yarn-trunk #6 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/6/]) HADOOP-8883. Anonymous fallback in KerberosAuthenticator is broken. (rkanter via tucu) (Revision 1398895) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1398895 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Anonymous fallback in KerberosAuthenticator is broken - Key: HADOOP-8883 URL: https://issues.apache.org/jira/browse/HADOOP-8883 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Robert Kanter Assignee: Robert Kanter Labels: security Fix For: 2.0.3-alpha Attachments: HADOOP-8883.patch HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the SPNEGO already; but this change broke using the fallback authenticator (PseudoAuthenticator) with an anonymous user (see OOZIE-1010). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[ https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477840#comment-13477840 ] Hudson commented on HADOOP-8883: Integrated in Hadoop-Hdfs-trunk #1198 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1198/]) HADOOP-8883. Anonymous fallback in KerberosAuthenticator is broken. (rkanter via tucu) (Revision 1398895) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1398895 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Anonymous fallback in KerberosAuthenticator is broken - Key: HADOOP-8883 URL: https://issues.apache.org/jira/browse/HADOOP-8883 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Robert Kanter Assignee: Robert Kanter Labels: security Fix For: 2.0.3-alpha Attachments: HADOOP-8883.patch HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the SPNEGO already; but this change broke using the fallback authenticator (PseudoAuthenticator) with an anonymous user (see OOZIE-1010). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[ https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477869#comment-13477869 ] Hudson commented on HADOOP-8883: Integrated in Hadoop-Mapreduce-trunk #1228 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1228/]) HADOOP-8883. Anonymous fallback in KerberosAuthenticator is broken. (rkanter via tucu) (Revision 1398895) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1398895 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Anonymous fallback in KerberosAuthenticator is broken - Key: HADOOP-8883 URL: https://issues.apache.org/jira/browse/HADOOP-8883 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Robert Kanter Assignee: Robert Kanter Labels: security Fix For: 2.0.3-alpha Attachments: HADOOP-8883.patch HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the SPNEGO already; but this change broke using the fallback authenticator (PseudoAuthenticator) with an anonymous user (see OOZIE-1010). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[ https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477074#comment-13477074 ] Alejandro Abdelnur commented on HADOOP-8883: +1 Anonymous fallback in KerberosAuthenticator is broken - Key: HADOOP-8883 URL: https://issues.apache.org/jira/browse/HADOOP-8883 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Robert Kanter Assignee: Robert Kanter Labels: security Fix For: 2.0.3-alpha Attachments: HADOOP-8883.patch HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the SPNEGO already; but this change broke using the fallback authenticator (PseudoAuthenticator) with an anonymous user (see OOZIE-1010). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[ https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477204#comment-13477204 ] Hudson commented on HADOOP-8883: Integrated in Hadoop-trunk-Commit #2871 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2871/]) HADOOP-8883. Anonymous fallback in KerberosAuthenticator is broken. (rkanter via tucu) (Revision 1398895) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1398895 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/client/KerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/AuthenticatorTestCase.java * /hadoop/common/trunk/hadoop-common-project/hadoop-auth/src/test/java/org/apache/hadoop/security/authentication/client/TestKerberosAuthenticator.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt Anonymous fallback in KerberosAuthenticator is broken - Key: HADOOP-8883 URL: https://issues.apache.org/jira/browse/HADOOP-8883 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.3-alpha Reporter: Robert Kanter Assignee: Robert Kanter Labels: security Fix For: 2.0.3-alpha Attachments: HADOOP-8883.patch HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the SPNEGO already; but this change broke using the fallback authenticator (PseudoAuthenticator) with an anonymous user (see OOZIE-1010). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13471763#comment-13471763
]
Hadoop QA commented on HADOOP-8883:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12548267/HADOOP-8883.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-auth.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1574//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1574//console
This message is automatically generated.
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Priority: Minor
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8883) Anonymous fallback in KerberosAuthenticator is broken
[
https://issues.apache.org/jira/browse/HADOOP-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13471824#comment-13471824
]
Robert Kanter commented on HADOOP-8883:
---
By the way, Jenkins didn't test it (because its not configured with Kerberos?),
but I checked that all tests in
{{org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator}}
pass.
Anonymous fallback in KerberosAuthenticator is broken
-
Key: HADOOP-8883
URL: https://issues.apache.org/jira/browse/HADOOP-8883
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.3-alpha
Reporter: Robert Kanter
Assignee: Robert Kanter
Priority: Minor
Labels: security
Fix For: 2.0.3-alpha
Attachments: HADOOP-8883.patch
HADOOP-8855 changed KerberosAuthenticator to handle when the JDK did the
SPNEGO already; but this change broke using the fallback authenticator
(PseudoAuthenticator) with an anonymous user (see OOZIE-1010).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
