[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13808656#comment-13808656 ] Kai Zheng commented on HADOOP-9797: --- Hi Larry, Thanks for taking care of this. You're right you found a coding error in getJaasLoginConfiguration(). The code was obsolete when introduced HadoopLoginFactory, and I should have removed the dead code. Thanks anyway. Sure to simply add the TokenAuthn method, we don't have to need this, as can be seen in HADOOP-9804. I'm probably running this for the long term trying various possible enhancements with incremental patches. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch, HADOOP-9797-v2.patch, HADOOP-9797-v3.patch, Pluggable and Compatible UGI Change.pdf As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message was sent by Atlassian JIRA (v6.1#6144)
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[
https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13807188#comment-13807188
]
Larry McCay commented on HADOOP-9797:
-
Hello Kai - I've spent some more time reviewing this patch and have found what
I believe to be a bug in the HadoopLoginManager.
In addition, I would like to discuss how we might be able to make progress with
a pluggable authentication mechanism without such a large change to UGI.
I fear that this change introduces a lot of risk to existing usage by both the
Hadoop infrastructure itself and to unknown clients from end users and third
party ecosystem usage.
If we can come up with a less intrusive change, I think that would be easier to
move forward.
What would you consider the least amount of change that would make plugging in
(or maybe just adding) token auth possible?
Anyway, the following code snippet seems like it may be wrong to me:
+ /**
+ * Get and return Jaas login configuration, as convenient method for Jaas
login
+ */
+ public static JaasLoginConfiguration getJaasLoginConfiguration(String
confName) {
+HadoopLoginConfiguration hlc = createLoginConfiguration(confName);
+
+if (hlc != null) {
+ return null;
+}
+
+if (! (hlc instanceof JaasLoginConfiguration) ) {
+ throw new IllegalArgumentException(Not JAAS configuration: +
confName);
+}
+JaasLoginConfiguration jlc = (JaasLoginConfiguration) hlc;
+return (JaasLoginConfiguration) jlc;
+ }
+}
Pluggable and compatible UGI change
---
Key: HADOOP-9797
URL: https://issues.apache.org/jira/browse/HADOOP-9797
Project: Hadoop Common
Issue Type: Sub-task
Components: security
Reporter: Kai Zheng
Assignee: Kai Zheng
Labels: Rhino
Fix For: 3.0.0
Attachments: HADOOP-9797-v1.patch, HADOOP-9797-v2.patch,
HADOOP-9797-v3.patch, Pluggable and Compatible UGI Change.pdf
As already widely discussed current UGI related classes needs to be improved
in many aspects. This is to improve and make UGI so that it can be:
* Pluggable, new authentication method with its login module can be
dynamically registered and plugged without having to change the UGI class;
* Extensible, login modules with their options can be dynamically extended
and customized so that can be reusable elsewhere, like in TokenAuth;
* No Kerberos relevant, remove any Kerberos relevant functionalities out of
it to make it simple and suitable for other login mechanisms;
* Of appropriate abstraction and API, with improved abstraction and API it’s
possible to allow authentication implementations not using JAAS modules;
* Compatible, should be compatible with previous deployment and
authentication methods, so the existing APIs won’t be removed and some of
them are just to be deprecated.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13763671#comment-13763671 ] Kai Zheng commented on HADOOP-9797: --- HADOOP-9942 and HADOOP-9943 were opened to allow unit tests for this UGI change using MiniKdc. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch, Pluggable and Compatible UGI Change.pdf As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13755963#comment-13755963 ] Kai Zheng commented on HADOOP-9797: --- Larry Dilli, This UGI change desires to be both pluggable and compatible. To be pluggable, it sure needs to be modular first and this requires removing the relevant global static variables in the UGI class. On the other hand to be compatible, as stated in the JIRA description and Dilli suggested, the static public methods in the UGI class will remain and are to be just deprecated since they’re part of the API. Sure this focuses on pluggable, removing statics is more like a side effect. I would keep the support of multiple clusters in mind in the design and implementation, though. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748145#comment-13748145 ] Larry McCay commented on HADOOP-9797: - In fact, we have to be really careful about changing the client facing APIs. For instance, clients - including third parties - leverage static loginUserFromKeyTab methods. It is difficult to impossible to know all the consumers of those APIs. So anyway, that is just a heads-up. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13748151#comment-13748151 ] Dilli Arumugam commented on HADOOP-9797: May be the static methods of UserGroupInformation should be marked deprecated in this patch. May be they should be marked deprecated independent of this patch. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13746709#comment-13746709 ] Kai Zheng commented on HADOOP-9797: --- Hi Sanjay, bq. Having read the patch, I agree with Daryn, can you split this jira into smaller ones and submit some updated patches please. Yes it's going. HADOOP-9840 and HADOOP-9841 were opened and two incremental patches were attached. I'm working on the left part and subsequent ones will be coming. Would you help review them and provide your inputs then. Thanks. bq. can you please add a comment on what you will be testing beyond the unit tests. Sure. I am working with our QA engineers on testing both Simple and Kerberos cases, covering HDFS for the initial patch. As you suggested, we will also add more tests covering YARN component and trusted proxy case when submitting related patches. Any other testing scenario that you would like to see? Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13746721#comment-13746721 ] Kai Zheng commented on HADOOP-9797: --- Hi Lars, bq. is there any chance to do away with all of the static members and methods on UGI Yes it’s possible. We’re working on incremental patches and in them getting rid of global and static stuffs is considered. bq. the same JVM we need to connect to some kerberos secured and some unsecured clusters. Good idea! This provides another strong case to validate the change to support multiple clusters for client. The change will ensure to use fresh UGI and its internals after cluster switching, beside this, any security concerns do you have? If any what kind of convenient support the UGI library can provide? Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13746740#comment-13746740 ] Larry McCay commented on HADOOP-9797: - I would suggest keeping the refactoring of statics and the like as a separate effort. Don't mix it with the pluggability work which is complex enough. Not that it shouldn't be done - just make it a separate issue. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13740654#comment-13740654 ] Lars Hofhansl commented on HADOOP-9797: --- While we are at it, is there any chance to do away with all of the static members and methods on UGI (loginUser is still static)? We ran into a problem were from the same JVM we need to connect to some kerberos secured and some unsecured clusters. That is currently not possible. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735045#comment-13735045 ] Sanjay Radia commented on HADOOP-9797: -- [~daryn] bq. but a change this large might need to be decomposed into incremental steps. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735050#comment-13735050 ] Sanjay Radia commented on HADOOP-9797: -- OOps hit add by mistake. [~daryn] bq. but a change this large might need to be decomposed into incremental steps. Having read the patch, I agree with Daryn, can you split this jira into smaller ones and submit some updated patches please. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13735218#comment-13735218 ] Sanjay Radia commented on HADOOP-9797: -- Given that this jira is going change a key part of the code, can you please add a comment on what you will be testing beyond the unit tests. For security, we have relied on a fair amount of manual testing. You should test the classic kerberos case for both HDFS and MR, aloog with a trusted proxy (say OOzie). Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13731814#comment-13731814 ] Kai Zheng commented on HADOOP-9797: --- bq.Along the same lines as HADOOP-9840, this is further locking in a client having one and only one identity. Please see my response on HADOOP-9840, it should still allow for support of multiple identities. bq.I've often considered having subclasses of UGI that were login-type specific. Good to hear. In the initial patch it comes up LiteUGI and various subclasses of it are login-type specific. I will resolve some concerns like naming issues and provide another patch based on existing ones. bq.a client UGI should do JAAS login on-demand for a given AuthMethod. Quite agree. It's particularly useful for token authn client which is instructed by server to use specific authn mechanisms and JAAS login modules dynamically. That's also the primary goal for this issue although we should approach it incrementally as you suggested. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13730826#comment-13730826 ] Daryn Sharp commented on HADOOP-9797: - Along the same lines as HADOOP-9840, this is further locking in a client having one and only one identity. I've often considered having subclasses of UGI that were login-type specific. Owen had concerns that this was once tried and failed but I thought I could make it work. Now that there's these alternate login methods coming, there's a problem if the user has a TGT - it's authMethod KERBEROS but then accesses a service requiring HSSO/TokenAuth. The UGI must simultaneously support both. My general thinking from before the summit has been a client UGI should do JAAS login on-demand for a given AuthMethod. A few examples are only trigger kerberos auth if a web service wants spnego or SASL service wants GSSAPI. Being on the 2.1 critical path has prevented me from having the time to flesh out how that may be accomplished... Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: Rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13726231#comment-13726231 ] Kai Zheng commented on HADOOP-9797: --- Larry, Thanks for your comments. 1. If you have any suggestions for another term to distinguish the difference, perhaps we could discuss it on the TokenAuth related JIRA, HADOOP-9804 Hadoop RPC TokenAuthn method. 2. 3. Regarding LiteUGI, let me think of a better name for it with the next drop. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13726462#comment-13726462 ] Larry McCay commented on HADOOP-9797: - Sounds good, Kai. Regarding #23 - I think we should consider the interface, abstract base class and interface being returned through the methods instead of the abstract class. I don't really think it is just a naming issue. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725242#comment-13725242 ] Daryn Sharp commented on HADOOP-9797: - I'll review today once I get some other RPC changes done, and see if/how it conflicts with another effort to reduce sync'ing in the UGI. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725287#comment-13725287 ] Larry McCay commented on HADOOP-9797: - Thanks for the patch, Kai. I will also be reviewing this work today and in particular considering how it may be leveraged for a REST AuthenticationHandler that I have for validating incoming JWT access tokens. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725504#comment-13725504 ] Daryn Sharp commented on HADOOP-9797: - At a high level this looks like a good change, but wow it's big. The UGI is a critical class. Subtle bugs exposed by race conditions can knock out a server - those issues are extremely hard to debug. I'm actively reviewing but a change this large might need to be decomposed into incremental steps. It's taking me a bit to unwind what it's doing and verify/document problems. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9797) Pluggable and compatible UGI change
[ https://issues.apache.org/jira/browse/HADOOP-9797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13725592#comment-13725592 ] Larry McCay commented on HADOOP-9797: - As for the design changes, I think that you introduce some nice abstraction that is missing in UGI. In general the pattern of using an additional check everywhere isSecurityEnabled is called is unfortunate - but I understand why you've take that approach for backward compatibility. I think that this pattern can be leveraged for REST endpoints as well - within AuthFilter. A couple things that bother me a bit: 1. It is even more difficult to distinguish the difference between TokenAuth and the existing TOKEN(AuthMethod.TOKEN) in the code. I think that in our previous discussions we had the context of those discussions to keep it all straight. For the uninitiated developer trying to discern what this code does - I think it is a problem. Unfortunately, I don't have an alternative term to propose yet. 2. LiteUgi: I think that this is probably mostly a classname issue. Lite brings certain connotations to mind that I don't think make sense for this class. It appears to be a base/abstract class for deriving concrete UGI implementations from but at the same time it is returned from methods. It is also used in method names. I think that what we really need here is an abstract implementation of a new UGI interface. The interface should be returned by related methods instead of an abstract class and method names should not include the abstract/base classname. I also don't think Lite is an appropriate name - I kept thinking that it was somehow related to Simple or some other lighter context. This would facilitate the ability to have UGI impls that don't need to extend that particular abstract class. 3. Using the LiteUgi name in methods - just calling this out as a separate issue from the classname in #2 above. I also share Daryn's opinion that it is a large change. Refactorings such as these are difficult to decompose into smaller steps but doing so would allow the review to be done more easily. Additionally, smaller changes would need to be reverted in the case of problems being introduced through one of the patches. I will continue to dig through the patch to provide a more detailed review - but I thought that I would share my high level thinking at this time. Pluggable and compatible UGI change --- Key: HADOOP-9797 URL: https://issues.apache.org/jira/browse/HADOOP-9797 Project: Hadoop Common Issue Type: Sub-task Components: security Reporter: Kai Zheng Assignee: Kai Zheng Labels: rhino Fix For: 3.0.0 Attachments: HADOOP-9797-v1.patch As already widely discussed current UGI related classes needs to be improved in many aspects. This is to improve and make UGI so that it can be: * Pluggable, new authentication method with its login module can be dynamically registered and plugged without having to change the UGI class; * Extensible, login modules with their options can be dynamically extended and customized so that can be reusable elsewhere, like in TokenAuth; * No Kerberos relevant, remove any Kerberos relevant functionalities out of it to make it simple and suitable for other login mechanisms; * Of appropriate abstraction and API, with improved abstraction and API it’s possible to allow authentication implementations not using JAAS modules; * Compatible, should be compatible with previous deployment and authentication methods, so the existing APIs won’t be removed and some of them are just to be deprecated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
