[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2247: -- Attachment: YARN-2247.6.patch Did some minor touch on the patch to change to description/log from Yarn authentication filter to RM Will commit it once jenkins +1. Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Priority: Blocker Attachments: YARN-2247.6.patch, apache-yarn-2247.0.patch, apache-yarn-2247.1.patch, apache-yarn-2247.2.patch, apache-yarn-2247.3.patch, apache-yarn-2247.4.patch, apache-yarn-2247.5.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2247: -- Fix Version/s: 2.5.0 Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Priority: Blocker Fix For: 2.5.0 Attachments: YARN-2247.6.patch, apache-yarn-2247.0.patch, apache-yarn-2247.1.patch, apache-yarn-2247.2.patch, apache-yarn-2247.3.patch, apache-yarn-2247.4.patch, apache-yarn-2247.5.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2247:
Attachment: apache-yarn-2247.5.patch
{quote}
1. I meant RM has the same problem, and we need to do null check
{noformat}
+if (testMiniKDC != null) {
+ testMiniKDC.stop();
+}
+rm.stop();
{noformat}
{quote}
Got it. Fixed.
{quote}
2. YarnAuthenticationFilter(Initializer) - RMAuthenticationFilter(Initializer)
{quote}
Fixed.
Allow RM web services users to authenticate using delegation tokens
---
Key: YARN-2247
URL: https://issues.apache.org/jira/browse/YARN-2247
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Priority: Blocker
Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch,
apache-yarn-2247.2.patch, apache-yarn-2247.3.patch, apache-yarn-2247.4.patch,
apache-yarn-2247.5.patch
The RM webapp should allow users to authenticate using delegation tokens to
maintain parity with RPC.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2247:
Attachment: apache-yarn-2247.4.patch
{quote}
Varun Vasudev, thanks for your patience on my comments. The new patch looks
almost good to me. Just some nits:
1. Should not be necessary. Always load TimelineAuthenticationFilter. With
simple type, still the pseudo handler is to used.
{noformat}
+if (authType.equals(simple)
!UserGroupInformation.isSecurityEnabled()) {
+ container.addFilter(authentication,
+AuthenticationFilter.class.getName(), filterConfig);
+ return;
+}
{noformat}
{quote}
Good point. Fixed.
{quote}
2. Check not null first for testMiniKDC and rm? Same for
TestRMWebappAuthentication
{noformat}
+testMiniKDC.stop();
+rm.stop();
{noformat}
{quote}
Fixed.
{quote}
3. I didn't find the logic to forbid it. Anyway, is it good to mention it in
the document as well?
{noformat}
+ // Test to make sure that we can't do delegation token
+ // functions using just delegation token auth
{noformat}
{quote}
The test is in RMWebServices.
{noformat}
callerUGI = createKerberosUserGroupInformation(hsr);
{noformat}
which in turn has this check
{noformat}
String authType = hsr.getAuthType();
if (!KerberosAuthenticationHandler.TYPE.equals(authType)) {
String msg =
Delegation token operations can only be carried out on a
+ Kerberos authenticated channel;
throw new YarnException(msg);
}
{noformat}
I've documented it under the delegation token rest API section:
{noformat}
All delegation token requests must be carried out on a Kerberos authenticated
connection(using SPNEGO).
{noformat}
Allow RM web services users to authenticate using delegation tokens
---
Key: YARN-2247
URL: https://issues.apache.org/jira/browse/YARN-2247
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Priority: Blocker
Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch,
apache-yarn-2247.2.patch, apache-yarn-2247.3.patch, apache-yarn-2247.4.patch
The RM webapp should allow users to authenticate using delegation tokens to
maintain parity with RPC.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[
https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-2247:
Attachment: apache-yarn-2247.3.patch
{quote}
bq.The current implementation uses the standard http authentication for
hadoop. Users can set it to simple if they choose.
I was trying to make the point that when kerberos authentication is not used,
simple authentication is not implicitly set, isn't it? In this case, without
the authentication filter, we cannot identify the user via HTTP interface, such
that we cannot behave correctly for those operations that require the knowledge
of user information, such as submit/kill an application.
One step back, and let's look at the analog RPC interfaces. By default, the
authentication is SIMPLE, and at the server side, we can still identify who the
user is, such that the feature such as ACLs are is still working in the SIMPLE
case.
{quote}
Got it. I've added support for simple auth in the default case. I also spoke
with [~vinodkv] offline and we felt that in secure mode the default static user
should not be allowed to submit jobs. I made that change as well.
{quote}
bq.For now I'd like to use the same configs as the standard hadoop http
auth. I'm open to changing them if we feel strongly about it in the future.
It's okay to keep the configuration same. Just think it out loudly. If so, you
may not want to add RM_WEBAPP_USE_YARN_AUTH_FILTER at all, and not load
YarnAuthenticationFilterInitializer programatically. The rationales behind them
are similar. Previously, I tried to add TimelineAuthenticationFilterInitializer
programmatically because I find the same http auth config applies to different
daemons, and I think it's annoying that at a single node cluster, I want to
config something only for timeline server, it will affect others. Afterwards, I
tried to make timeline server to use a set of configs with timeline-service
prefix. This is what we did for the RPC interface configurations.
{quote}
I see your point but I don't think forcing users to replicate existing configs
makes sense at this point. The RM web interfaces are already controlled by the
common http auth configs and I'd like to preserve that behaviour.
{quote}
bq.I didn't understand - can you explain further?
Let's take RMWebServices#getApp as an example. Previously we don't have (at
least don't know) the auth filter, such that we cannot detect the user.
Therefore, we don't check the ACLs, and simply get the application from
RMContext and return the user. Now, we have the auth filter, and we can
identify the user. Hence, it's possible for use to fix this API to only return
the application information to the user that has the access. This is also
another reason why I suggest to always have authentication filter on, whether
it is simple or kerberos.
{quote}
Agree with your tickets.
{quote}
bq.Am I looking at the wrong file?
This is the right file, but I'm afraid it is not the correct logic.
AuthenticationFilter accept null secret file. However, if we use
AuthenticationFilterInitializer to construct AuthenticationFilter, the null
case is denied. I previously open a ticket for this issue (HADOOP-10600).
{quote}
Thanks for pointing that out. Fixed.
Allow RM web services users to authenticate using delegation tokens
---
Key: YARN-2247
URL: https://issues.apache.org/jira/browse/YARN-2247
Project: Hadoop YARN
Issue Type: Sub-task
Reporter: Varun Vasudev
Assignee: Varun Vasudev
Priority: Blocker
Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch,
apache-yarn-2247.2.patch, apache-yarn-2247.3.patch
The RM webapp should allow users to authenticate using delegation tokens to
maintain parity with RPC.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2247: Attachment: apache-yarn-2247.1.patch Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2247: Attachment: (was: apache-yarn-2247.2.patch) Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2247: Attachment: apache-yarn-2247.2.patch Upload correct patch file. Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch, apache-yarn-2247.2.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhijie Shen updated YARN-2247: -- Priority: Blocker (was: Major) Target Version/s: 2.5.0 Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Priority: Blocker Attachments: apache-yarn-2247.0.patch, apache-yarn-2247.1.patch, apache-yarn-2247.2.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2247) Allow RM web services users to authenticate using delegation tokens
[ https://issues.apache.org/jira/browse/YARN-2247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Vasudev updated YARN-2247: Attachment: apache-yarn-2247.0.patch Allow RM web services users to authenticate using delegation tokens --- Key: YARN-2247 URL: https://issues.apache.org/jira/browse/YARN-2247 Project: Hadoop YARN Issue Type: Sub-task Reporter: Varun Vasudev Assignee: Varun Vasudev Attachments: apache-yarn-2247.0.patch The RM webapp should allow users to authenticate using delegation tokens to maintain parity with RPC. -- This message was sent by Atlassian JIRA (v6.2#6252)
