[jira] [Commented] (HADOOP-9392) Token based authentication and Single Sign On
[ https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678961#comment-13678961 ] Kai Zheng commented on HADOOP-9392: --- Daryn – bq. Delegation tokens are embedded at the RPC layer, so it's a capability that any service using the common RPC may use. Thanks for the clarification. Yes that part was misspoken. The term ‘delegation’ is being overloaded here. The relevant fact is delegation can be done only where Hadoop RPC is used. We will update the document to be more clear about issues of delegation. bq. Given all the discussions involving more radical changes to the security framework, I'm very keen to providing the modularity required to implement these systems, but in a manner that will not destabilize the existing security implementation, else Yahoo's 2.x deployments may be delayed. Agreed. The proposal here implements Hadoop side changes using SASL and Hadoop RPC of today as a starting point, with a requirement that the end result remains backwards compatible and interoperable with existing deployments. Kevin – bq. Aligning this area of work across all interested parties is critical. We need to be able to clearly articulate the goals of the effort and then understand how we can all work together to accomplish them without duplicate, conflicting work and destabilizing Hadoop. […] We all have different ideas and are approaching this from different angles. We need to figure out how all the puzzle pieces fit together. This is exactly what we hoped opening this JIRA would spark and would like very much for the whole community of interested parties to work in a cooperative way. In addition to putting up an agenda for the summit meetup to bring some structure, bringing all related discussion under the umbrella of this JIRA would perhaps be helpful in having everyone working together. Token based authentication and Single Sign On - Key: HADOOP-9392 URL: https://issues.apache.org/jira/browse/HADOOP-9392 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Kai Zheng Assignee: Kai Zheng Fix For: 3.0.0 Attachments: token-based-authn-plus-sso.pdf This is an umbrella entry for one of project Rhino’s topic, for details of project Rhino, please refer to https://github.com/intel-hadoop/project-rhino/. The major goal for this entry as described in project Rhino was “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at the RPC layer, via SASL. However this does not provide valuable attributes such as group membership, classification level, organizational identity, or support for user defined attributes. Hadoop components must interrogate external resources for discovering these attributes and at scale this is problematic. There is also no consistent delegation model. HDFS has a simple delegation capability, and only Oozie can take limited advantage of it. We will implement a common token based authentication framework to decouple internal user and service authentication from external mechanisms used to support it (like Kerberos)” We’d like to start our work from Hadoop-Common and try to provide common facilities by extending existing authentication framework which support: 1.Pluggable token provider interface 2.Pluggable token verification protocol and interface 3.Security mechanism to distribute secrets in cluster nodes 4.Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679000#comment-13679000 ] Hudson commented on HADOOP-9599: Integrated in Hadoop-Yarn-trunk #235 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/235/]) HADOOP-9599. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly. Contributed by Mostafa Elhemali. (Revision 1491030) Result = SUCCESS ivanmi : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1491030 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Fix For: 2.1.0-beta Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9636) UNIX like sort options for ls shell command
[ https://issues.apache.org/jira/browse/HADOOP-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679017#comment-13679017 ] Gopal V commented on HADOOP-9636: - Test failing is help: help for ls. Didn't run for test-patch, but it shouldn't be a hard test-case to fix. UNIX like sort options for ls shell command --- Key: HADOOP-9636 URL: https://issues.apache.org/jira/browse/HADOOP-9636 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 3.0.0 Reporter: Varun Dhussa Priority: Minor Attachments: HADOOP-9636-001.patch Add support for unix ls like sort options in fs -ls: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : sort by acess time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679040#comment-13679040 ] Hudson commented on HADOOP-9599: Integrated in Hadoop-Hdfs-trunk #1425 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1425/]) HADOOP-9599. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly. Contributed by Mostafa Elhemali. (Revision 1491030) Result = FAILURE ivanmi : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1491030 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Fix For: 2.1.0-beta Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679066#comment-13679066 ] Hudson commented on HADOOP-9599: Integrated in Hadoop-Mapreduce-trunk #1452 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1452/]) HADOOP-9599. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly. Contributed by Mostafa Elhemali. (Revision 1491030) Result = SUCCESS ivanmi : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1491030 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Fix For: 2.1.0-beta Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9421) Convert SASL to use ProtoBuf and add lengths for non-blocking processing
[ https://issues.apache.org/jira/browse/HADOOP-9421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679107#comment-13679107 ] Luke Lu commented on HADOOP-9421: - bq. As-is, it would be possible for the client to blindly send the connection header INITIATE and just ignore the NEGOTIATE response. I'm concerned about the client code needed to maintain backward-compatibility. If client is enhanced to INITIATE first, client will need to always ignore the first NEGOTIATE and have to handle the second NEGOTIATE, which makes the client logic confusing and you cannot make the first dumb NEGOTIATE (which is size of O(number of mechs/protocols)) go away, it'll always be part of the protocol. The code is also replicated per client per language, which is a higher cost than just maintaining it at the server side. If client always sends a INITIATE, which could be empty, it is IMO easier to understand and evolve: server can handle the INITIATE and there will not be any wasted NEGOTIATE. In the era of YARN, where NN is resource strapped to handle the flood of requests from containers/tasks, I think it'd be worthwhile to leave room for optimization to reduce the amount of extra processing at the server side. The current approach will always have an extra NEGOTIATE that cannot be optimized away. Also, I noticed that you deprecated DIGEST and added TOKEN as an auth method (essentially renamed DIGEST to TOKEN) in SaslRpcServer. Though DIGEST is not exactly a precise word here, TOKEN is, IMO, even more nebulous. How about CHALLENGE_RESPONSE or simply CR? I'd prefer the rename to be in separate JIRA as well, as it doesn't really affect the wire protocol. Convert SASL to use ProtoBuf and add lengths for non-blocking processing Key: HADOOP-9421 URL: https://issues.apache.org/jira/browse/HADOOP-9421 Project: Hadoop Common Issue Type: Sub-task Affects Versions: 2.0.3-alpha Reporter: Sanjay Radia Assignee: Daryn Sharp Attachments: HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421.patch, HADOOP-9421-v2-demo.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9636) UNIX like sort options for ls shell command
[ https://issues.apache.org/jira/browse/HADOOP-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13679168#comment-13679168 ] Jonathan Allen commented on HADOOP-9636: This is a duplicate of HADOOP-8934 UNIX like sort options for ls shell command --- Key: HADOOP-9636 URL: https://issues.apache.org/jira/browse/HADOOP-9636 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 3.0.0 Reporter: Varun Dhussa Priority: Minor Attachments: HADOOP-9636-001.patch Add support for unix ls like sort options in fs -ls: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : sort by acess time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
