[jira] [Commented] (HADOOP-9392) Token based authentication and Single Sign On
[ https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678805#comment-13678805 ] Andrew Purtell commented on HADOOP-9392: bq. Currently, a room has been allocated on the 26th from 1:45 to 3:30 PT. Specific location will be available at the Summit and any changes in date or time will be announced publicly to the best of our abilities. In order to create a manageable agenda for this session, I'd like to schedule some prep meetings via meetup.com. [~kevin.minder] Is there a link to that meetup group? Token based authentication and Single Sign On - Key: HADOOP-9392 URL: https://issues.apache.org/jira/browse/HADOOP-9392 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Kai Zheng Assignee: Kai Zheng Fix For: 3.0.0 Attachments: token-based-authn-plus-sso.pdf This is an umbrella entry for one of project Rhino’s topic, for details of project Rhino, please refer to https://github.com/intel-hadoop/project-rhino/. The major goal for this entry as described in project Rhino was “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at the RPC layer, via SASL. However this does not provide valuable attributes such as group membership, classification level, organizational identity, or support for user defined attributes. Hadoop components must interrogate external resources for discovering these attributes and at scale this is problematic. There is also no consistent delegation model. HDFS has a simple delegation capability, and only Oozie can take limited advantage of it. We will implement a common token based authentication framework to decouple internal user and service authentication from external mechanisms used to support it (like Kerberos)” We’d like to start our work from Hadoop-Common and try to provide common facilities by extending existing authentication framework which support: 1.Pluggable token provider interface 2.Pluggable token verification protocol and interface 3.Security mechanism to distribute secrets in cluster nodes 4.Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678815#comment-13678815 ] Ivan Mitic commented on HADOOP-9599: Thanks Mostafa, +1 again, will commit the patch shortly hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME. The net effect is that running e.g. hdfs namenode would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678819#comment-13678819 ] Hudson commented on HADOOP-9599: Integrated in Hadoop-trunk-Commit #3886 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3886/]) HADOOP-9599. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly. Contributed by Mostafa Elhemali. (Revision 1491030) Result = SUCCESS ivanmi : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1491030 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-config.cmd hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME. The net effect is that running e.g. hdfs namenode would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated HADOOP-9599: --- Description: In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. was: In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME. The net effect is that running e.g. hdfs namenode would error out with UnsatisfiedLinkError because it can't access hadoop.dll. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated HADOOP-9599: --- Resolution: Fixed Fix Version/s: 2.1.0-beta Status: Resolved (was: Patch Available) Patch committed to trunk, branch-2 and branch-2.1-beta. Thanks Mostafa for the patch and Chuan for the review. hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Fix For: 2.1.0-beta Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9599) hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly
[ https://issues.apache.org/jira/browse/HADOOP-9599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Mitic updated HADOOP-9599: --- Hadoop Flags: Reviewed hadoop-config.cmd doesn't set JAVA_LIBRARY_PATH correctly - Key: HADOOP-9599 URL: https://issues.apache.org/jira/browse/HADOOP-9599 Project: Hadoop Common Issue Type: Bug Affects Versions: 3.0.0 Environment: Windows Reporter: Mostafa Elhemali Assignee: Mostafa Elhemali Fix For: 2.1.0-beta Attachments: HADOOP-9599.2.patch, HADOOP-9599.3.patch, HADOOP-9599.patch In Windows, hadoop-config.cmd uses the non-existent-variable HADOOP_CORE_HOME when setting the JAVA_LIBRAR_PATH variable. It should use HADOOP_HOME or HADOOP_COMMON_HOME. The net effect is that running e.g. hdfs namenode directly (outside of hadoop command prompt) would error out with UnsatisfiedLinkError because it can't access hadoop.dll. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9635) Potential Stack Overflow in DomainSocket.c
V. Karthik Kumar created HADOOP-9635:
Summary: Potential Stack Overflow in DomainSocket.c
Key: HADOOP-9635
URL: https://issues.apache.org/jira/browse/HADOOP-9635
Project: Hadoop Common
Issue Type: Bug
Components: native
Affects Versions: 2.3.0
Environment: OSX 10.8
Reporter: V. Karthik Kumar
When I was running on OSX, the DataNode was segfaulting. On investigation, it
was tracked down to this code. A potential stack overflow was also identified.
{code}
utfLength = (*env)-GetStringUTFLength(env, jstr);
if (utfLength sizeof(path)) {
jthr = newIOException(env, path is too long! We expected a path
no longer than %zd UTF-8 bytes., sizeof(path));
goto done;
}
// GetStringUTFRegion does not pad with NUL
(*env)-GetStringUTFRegion(env, jstr, 0, utfLength, path);
...
//strtok_r can set rest pointer to NULL when no tokens found.
//Causes JVM to crash in rest[0]
for (check[0] = '/', check[1] = '\0', rest = path, token = ;
token rest[0];
token = strtok_r(rest, /, rest)) {
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9635) Potential Stack Overflow in DomainSocket.c
[
https://issues.apache.org/jira/browse/HADOOP-9635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
V. Karthik Kumar updated HADOOP-9635:
-
Attachment: DomainSocket.diff
Attached Patch.
Potential Stack Overflow in DomainSocket.c
--
Key: HADOOP-9635
URL: https://issues.apache.org/jira/browse/HADOOP-9635
Project: Hadoop Common
Issue Type: Bug
Components: native
Affects Versions: 2.3.0
Environment: OSX 10.8
Reporter: V. Karthik Kumar
Attachments: DomainSocket.diff
When I was running on OSX, the DataNode was segfaulting. On investigation, it
was tracked down to this code. A potential stack overflow was also
identified.
{code}
utfLength = (*env)-GetStringUTFLength(env, jstr);
if (utfLength sizeof(path)) {
jthr = newIOException(env, path is too long! We expected a path
no longer than %zd UTF-8 bytes., sizeof(path));
goto done;
}
// GetStringUTFRegion does not pad with NUL
(*env)-GetStringUTFRegion(env, jstr, 0, utfLength, path);
...
//strtok_r can set rest pointer to NULL when no tokens found.
//Causes JVM to crash in rest[0]
for (check[0] = '/', check[1] = '\0', rest = path, token = ;
token rest[0];
token = strtok_r(rest, /, rest)) {
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9635) Potential Stack Overflow in DomainSocket.c
[
https://issues.apache.org/jira/browse/HADOOP-9635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678897#comment-13678897
]
V. Karthik Kumar commented on HADOOP-9635:
--
{code}
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x
[Switching to process 406 thread 0x1103]
0x0001094a67fc in
Java_org_apache_hadoop_net_unix_DomainSocket_validateSocketPathSecurity0
(env=0x1005004c0, clazz=0x1094a90e9, jstr=0x0, skipComponents=0) at
DomainSocket.c:308
308token rest[0];
(gdb) bt
#0 0x0001094a67fc in
Java_org_apache_hadoop_net_unix_DomainSocket_validateSocketPathSecurity0
(env=0x1005004c0, clazz=0x1094a90e9, jstr=0x0, skipComponents=0) at
DomainSocket.c:308
#1 0x000105011eee in ?? ()
#2 0x00010500685a in ?? ()
#3 0x0001050069b3 in ?? ()
#4 0x00010500685a in ?? ()
#5 0x0001050069b3 in ?? ()
#6 0x00010500685a in ?? ()
#7 0x00010500685a in ?? ()
#8 0x00010500685a in ?? ()
#9 0x0001050069b3 in ?? ()
#10 0x0001050069b3 in ?? ()
#11 0x0001050069b3 in ?? ()
#12 0x00010500685a in ?? ()
#13 0x000105001438 in ?? ()
#14 0x000101096a12 in JVM_Lseek ()
#15 0x0001010967d6 in JVM_Lseek ()
#16 0x0001010b95d3 in JVM_FindLoadedClass ()
#17 0x0001010b947b in JVM_FindLoadedClass ()
#18 0x00012cf3 in ?? ()
#19 0x00013240 in ?? ()
#20 0x7fff9753b7a2 in _pthread_start ()
#21 0x7fff975281e1 in thread_start ()
(gdb)
{code}
Potential Stack Overflow in DomainSocket.c
--
Key: HADOOP-9635
URL: https://issues.apache.org/jira/browse/HADOOP-9635
Project: Hadoop Common
Issue Type: Bug
Components: native
Affects Versions: 2.3.0
Environment: OSX 10.8
Reporter: V. Karthik Kumar
Attachments: DomainSocket.diff
When I was running on OSX, the DataNode was segfaulting. On investigation, it
was tracked down to this code. A potential stack overflow was also
identified.
{code}
utfLength = (*env)-GetStringUTFLength(env, jstr);
if (utfLength sizeof(path)) {
jthr = newIOException(env, path is too long! We expected a path
no longer than %zd UTF-8 bytes., sizeof(path));
goto done;
}
// GetStringUTFRegion does not pad with NUL
(*env)-GetStringUTFRegion(env, jstr, 0, utfLength, path);
...
//strtok_r can set rest pointer to NULL when no tokens found.
//Causes JVM to crash in rest[0]
for (check[0] = '/', check[1] = '\0', rest = path, token = ;
token rest[0];
token = strtok_r(rest, /, rest)) {
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9621) Document/analyze current Hadoop security model
[ https://issues.apache.org/jira/browse/HADOOP-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678902#comment-13678902 ] Kevin Minder commented on HADOOP-9621: -- Kyle, I added some general details about Tokens to the top of the doc. Mostly just notes I captured while researching client tokens. I think you were looking for this generic detail but I'm don't really have a stron sense for where it belongs in the doc. Kevin. Document/analyze current Hadoop security model -- Key: HADOOP-9621 URL: https://issues.apache.org/jira/browse/HADOOP-9621 Project: Hadoop Common Issue Type: Task Components: security Reporter: Brian Swan Priority: Minor Labels: documentation Original Estimate: 336h Remaining Estimate: 336h In light of the proposed changes to Hadoop security in Hadoop-9533 and Hadoop-9392, having a common, detailed understanding (in the form of a document) of the benefits/drawbacks of the current security model and how it works would be useful. The document should address all security principals, their authentication mechanisms, and handling of shared secrets through the lens of the following principles: Minimize attack surface area, Establish secure defaults, Principle of Least privilege, Principle of Defense in depth, Fail securely, Don’t trust services, Separation of duties, Avoid security by obscurity, Keep security simple, Fix security issues correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9636) UNIX like sort options for ls shell command
Varun Dhussa created HADOOP-9636: Summary: UNIX like sort options for ls shell command Key: HADOOP-9636 URL: https://issues.apache.org/jira/browse/HADOOP-9636 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 3.0.0 Reporter: Varun Dhussa Add support for unix ls like sort options in fs -ls: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : sort by acess time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9636) UNIX like sort options for ls shell command
[ https://issues.apache.org/jira/browse/HADOOP-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Dhussa updated HADOOP-9636: - Priority: Minor (was: Major) UNIX like sort options for ls shell command --- Key: HADOOP-9636 URL: https://issues.apache.org/jira/browse/HADOOP-9636 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 3.0.0 Reporter: Varun Dhussa Priority: Minor Attachments: HADOOP-9636-001.patch Add support for unix ls like sort options in fs -ls: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : sort by acess time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9636) UNIX like sort options for ls shell command
[ https://issues.apache.org/jira/browse/HADOOP-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Dhussa updated HADOOP-9636: - Status: Patch Available (was: Open) UNIX like sort options for ls shell command --- Key: HADOOP-9636 URL: https://issues.apache.org/jira/browse/HADOOP-9636 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 3.0.0 Reporter: Varun Dhussa Priority: Minor Attachments: HADOOP-9636-001.patch Add support for unix ls like sort options in fs -ls: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : sort by acess time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9636) UNIX like sort options for ls shell command
[ https://issues.apache.org/jira/browse/HADOOP-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Varun Dhussa updated HADOOP-9636: - Attachment: HADOOP-9636-001.patch UNIX like sort options for ls shell command --- Key: HADOOP-9636 URL: https://issues.apache.org/jira/browse/HADOOP-9636 Project: Hadoop Common Issue Type: Improvement Components: fs Affects Versions: 3.0.0 Reporter: Varun Dhussa Attachments: HADOOP-9636-001.patch Add support for unix ls like sort options in fs -ls: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : sort by acess time -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9636) UNIX like sort options for ls shell command
[
https://issues.apache.org/jira/browse/HADOOP-9636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13678943#comment-13678943
]
Hadoop QA commented on HADOOP-9636:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12586914/HADOOP-9636-001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.cli.TestCLI
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2625//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2625//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-common.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2625//console
This message is automatically generated.
UNIX like sort options for ls shell command
---
Key: HADOOP-9636
URL: https://issues.apache.org/jira/browse/HADOOP-9636
Project: Hadoop Common
Issue Type: Improvement
Components: fs
Affects Versions: 3.0.0
Reporter: Varun Dhussa
Priority: Minor
Attachments: HADOOP-9636-001.patch
Add support for unix ls like sort options in fs -ls:
-t : sort by modification time
-S : sort by file size
-r : reverse the sort order
-u : sort by acess time
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
