[jira] [Created] (HADOOP-9066) Sorting for FileStatus[]
david king created HADOOP-9066: -- Summary: Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 Reporter: david king I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] david king updated HADOOP-9066: --- Attachment: TestFileStatusTool.java FileStatusTool.java FileStatusComparable.java ConcreteFileStatusDescComparable.java ConcreteFileStatusAscComparable.java Hi ,I come from cina, I would be happy to take on this issue as I am interested in contributing. I use the FileStatusTool class to sort FileStatus[],you can run TestFileStatusTool to test it, But this TestCase need to make appropriate modification in Test Enviroment. Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500934#comment-13500934 ] david king commented on HADOOP-9066: Hi ,I come from china, I am glad to contribute the codes for hadoop, It is first time to submit code, if has any problem ,please contact me , my Email : xpws2...@163.com . thank you . Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500936#comment-13500936 ] Harsh J commented on HADOOP-9066: - Also you may want to read http://wiki.apache.org/hadoop/HowToContribute. Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500935#comment-13500935 ] Harsh J commented on HADOOP-9066: - Hey David, Thanks! Could you add a comment in for more use-cases of what you are proposing to enhance/add so that we have a better idea? Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] david king updated HADOOP-9066: --- Description: I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator, the Comparator not only customer to realizate , but alse use the example code. (was: I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator) Environment: java7 , RedHat9 , Hadoop 0.20.2 ,eclipse-jee-juno-linux-gtk.tar.gz (was: java7 , RedHat9 , Hadoop 0.20.2) Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 ,eclipse-jee-juno-linux-gtk.tar.gz Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator, the Comparator not only customer to realizate , but alse use the example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[ https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9044: --- Status: Open (was: Patch Available) add FindClass main class to provide classpath checking of installations --- Key: HADOOP-9044 URL: https://issues.apache.org/jira/browse/HADOOP-9044 Project: Hadoop Common Issue Type: New Feature Components: util Affects Versions: 1.1.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Attachments: HADOOP-9044.patch It's useful in postflight checking of a hadoop installation to verify that classes load, especially codes with external JARs and native codecs. An entry point designed to load a named class and create an instance of that class can do this -and be invoked from any shell script or tool that does the installation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (HADOOP-9044) add FindClass main class to provide classpath checking of installations
[ https://issues.apache.org/jira/browse/HADOOP-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-9044 started by Steve Loughran. add FindClass main class to provide classpath checking of installations --- Key: HADOOP-9044 URL: https://issues.apache.org/jira/browse/HADOOP-9044 Project: Hadoop Common Issue Type: New Feature Components: util Affects Versions: 1.1.0 Reporter: Steve Loughran Assignee: Steve Loughran Priority: Minor Attachments: HADOOP-9044.patch It's useful in postflight checking of a hadoop installation to verify that classes load, especially codes with external JARs and native codecs. An entry point designed to load a named class and create an instance of that class can do this -and be invoked from any shell script or tool that does the installation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] david king updated HADOOP-9066: --- Attachment: (was: TestFileStatusTool.java) Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 ,eclipse-jee-juno-linux-gtk.tar.gz Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator, the Comparator not only customer to realizate , but alse use the example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] david king updated HADOOP-9066: --- Attachment: TestFileStatusTool.java Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 ,eclipse-jee-juno-linux-gtk.tar.gz Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator, the Comparator not only customer to realizate , but alse use the example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500947#comment-13500947 ] david king commented on HADOOP-9066: I have upload the new TestCase-TestFileStatusTool , Use the FileStatusTool to sort the array of FileStstus[], for examle: //Ascending the array of FileStatus[] FileStatusTool.sortFileStatus(inputFiles, new ConcreteFileStatusAscComparable()); //Descending the array of FileStatus[] FileStatusTool.sortFileStatus(inputFiles, new ConcreteFileStatusDescComparable()); Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 ,eclipse-jee-juno-linux-gtk.tar.gz Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator, the Comparator not only customer to realizate , but alse use the example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6616) Improve documentation for rack awareness
[ https://issues.apache.org/jira/browse/HADOOP-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500989#comment-13500989 ] Hudson commented on HADOOP-6616: Integrated in Hadoop-Yarn-trunk #42 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/42/]) HADOOP-6616. Improve documentation for rack awareness. Contributed by Adam Faris. (Revision 1411359) Result = SUCCESS jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411359 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml Improve documentation for rack awareness Key: HADOOP-6616 URL: https://issues.apache.org/jira/browse/HADOOP-6616 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Jeff Hammerbacher Assignee: Adam Faris Labels: newbie Fix For: 3.0.0 Attachments: hadoop-6616.patch, hadoop-6616.patch.2, hadoop-6616.patch.3, hadoop-6616.patch.4 The current documentation for rack awareness (http://hadoop.apache.org/common/docs/r0.20.0/cluster_setup.html#Hadoop+Rack+Awareness) should be augmented to include a sample script. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9045) In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
[ https://issues.apache.org/jira/browse/HADOOP-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated HADOOP-9045: --- Attachment: HADOOP-9045-v2.patch Fix find bug warning in v2 patch(although it is not a issue). In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup -- Key: HADOOP-9045 URL: https://issues.apache.org/jira/browse/HADOOP-9045 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Junping Du Assignee: Junping Du Attachments: HADOOP-9045.patch, HADOOP-9045-v2.patch In previous implementation for HADOOP-8468, 3rd replica is avoid to place on the same nodegroup of 2nd replica. But it didn't provide check on nodegroup of 1st replica, so if 2nd replica's rack is not efficient to place replica, then it is possible to place 3rd and 1st replica within the same node group. We need a change to remove all nodes from available nodes for placing replica if there already replica on the same nodegroup. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9045) In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
[
https://issues.apache.org/jira/browse/HADOOP-9045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501152#comment-13501152
]
Hadoop QA commented on HADOOP-9045:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554332/HADOOP-9045-v2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1777//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1777//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1777//console
This message is automatically generated.
In nodegroup-aware case, make sure nodes are avoided to place replica if some
replica are already under the same nodegroup
--
Key: HADOOP-9045
URL: https://issues.apache.org/jira/browse/HADOOP-9045
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.2-alpha
Reporter: Junping Du
Assignee: Junping Du
Attachments: HADOOP-9045.patch, HADOOP-9045-v2.patch
In previous implementation for HADOOP-8468, 3rd replica is avoid to place on
the same nodegroup of 2nd replica. But it didn't provide check on nodegroup
of 1st replica, so if 2nd replica's rack is not efficient to place replica,
then it is possible to place 3rd and 1st replica within the same node group.
We need a change to remove all nodes from available nodes for placing replica
if there already replica on the same nodegroup.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6616) Improve documentation for rack awareness
[ https://issues.apache.org/jira/browse/HADOOP-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501158#comment-13501158 ] Hudson commented on HADOOP-6616: Integrated in Hadoop-Hdfs-trunk #1232 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1232/]) HADOOP-6616. Improve documentation for rack awareness. Contributed by Adam Faris. (Revision 1411359) Result = SUCCESS jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411359 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml Improve documentation for rack awareness Key: HADOOP-6616 URL: https://issues.apache.org/jira/browse/HADOOP-6616 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Jeff Hammerbacher Assignee: Adam Faris Labels: newbie Fix For: 3.0.0 Attachments: hadoop-6616.patch, hadoop-6616.patch.2, hadoop-6616.patch.3, hadoop-6616.patch.4 The current documentation for rack awareness (http://hadoop.apache.org/common/docs/r0.20.0/cluster_setup.html#Hadoop+Rack+Awareness) should be augmented to include a sample script. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9067) provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long)
Ivan A. Veselovsky created HADOOP-9067: -- Summary: provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long) Key: HADOOP-9067 URL: https://issues.apache.org/jira/browse/HADOOP-9067 Project: Hadoop Common Issue Type: Test Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Priority: Minor this method is not covered by the existing unit tests. Provide a test to cover it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9067) provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long)
[ https://issues.apache.org/jira/browse/HADOOP-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-9067: --- Status: Patch Available (was: Open) provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long) -- Key: HADOOP-9067 URL: https://issues.apache.org/jira/browse/HADOOP-9067 Project: Hadoop Common Issue Type: Test Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-9067.patch this method is not covered by the existing unit tests. Provide a test to cover it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6616) Improve documentation for rack awareness
[ https://issues.apache.org/jira/browse/HADOOP-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501177#comment-13501177 ] Hudson commented on HADOOP-6616: Integrated in Hadoop-Mapreduce-trunk #1263 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1263/]) HADOOP-6616. Improve documentation for rack awareness. Contributed by Adam Faris. (Revision 1411359) Result = FAILURE jghoman : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411359 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/docs/src/documentation/content/xdocs/cluster_setup.xml Improve documentation for rack awareness Key: HADOOP-6616 URL: https://issues.apache.org/jira/browse/HADOOP-6616 Project: Hadoop Common Issue Type: Improvement Components: documentation Reporter: Jeff Hammerbacher Assignee: Adam Faris Labels: newbie Fix For: 3.0.0 Attachments: hadoop-6616.patch, hadoop-6616.patch.2, hadoop-6616.patch.3, hadoop-6616.patch.4 The current documentation for rack awareness (http://hadoop.apache.org/common/docs/r0.20.0/cluster_setup.html#Hadoop+Rack+Awareness) should be augmented to include a sample script. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9067) provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long)
[
https://issues.apache.org/jira/browse/HADOOP-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501189#comment-13501189
]
Hadoop QA commented on HADOOP-9067:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554342/HADOOP-9067.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1778//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1778//console
This message is automatically generated.
provide test for method
org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path,
FSDataInputStream, long, FSDataInputStream, long)
--
Key: HADOOP-9067
URL: https://issues.apache.org/jira/browse/HADOOP-9067
Project: Hadoop Common
Issue Type: Test
Reporter: Ivan A. Veselovsky
Assignee: Ivan A. Veselovsky
Priority: Minor
Attachments: HADOOP-9067.patch
this method is not covered by the existing unit tests. Provide a test to
cover it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9067) provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long)
[ https://issues.apache.org/jira/browse/HADOOP-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-9067: --- Attachment: HADOOP-9067--b.patch the new test sligtly improved to cover more methods from LocalFileSystem. provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long) -- Key: HADOOP-9067 URL: https://issues.apache.org/jira/browse/HADOOP-9067 Project: Hadoop Common Issue Type: Test Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-9067--b.patch, HADOOP-9067.patch this method is not covered by the existing unit tests. Provide a test to cover it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9067) provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long)
[
https://issues.apache.org/jira/browse/HADOOP-9067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501205#comment-13501205
]
Hadoop QA commented on HADOOP-9067:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554344/HADOOP-9067--b.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1779//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1779//console
This message is automatically generated.
provide test for method
org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path,
FSDataInputStream, long, FSDataInputStream, long)
--
Key: HADOOP-9067
URL: https://issues.apache.org/jira/browse/HADOOP-9067
Project: Hadoop Common
Issue Type: Test
Reporter: Ivan A. Veselovsky
Assignee: Ivan A. Veselovsky
Priority: Minor
Attachments: HADOOP-9067--b.patch, HADOOP-9067.patch
this method is not covered by the existing unit tests. Provide a test to
cover it.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9068) Reuse (and not duplicate) globbing logic between FileSystem and FileContext
Harsh J created HADOOP-9068: --- Summary: Reuse (and not duplicate) globbing logic between FileSystem and FileContext Key: HADOOP-9068 URL: https://issues.apache.org/jira/browse/HADOOP-9068 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Harsh J FileSystem's globbing code is currently duplicated in FileContext.Util class. We should reuse the implementation rather than maintain two pieces of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9068) Reuse (and not duplicate) globbing logic between FileSystem and FileContext
[ https://issues.apache.org/jira/browse/HADOOP-9068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eli Collins updated HADOOP-9068: Labels: newbie (was: ) Reuse (and not duplicate) globbing logic between FileSystem and FileContext --- Key: HADOOP-9068 URL: https://issues.apache.org/jira/browse/HADOOP-9068 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.0-alpha Reporter: Harsh J Labels: newbie FileSystem's globbing code is currently duplicated in FileContext.Util class. We should reuse the implementation rather than maintain two pieces of it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8845) When looking for parent paths info, globStatus must filter out non-directory elements to avoid an AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501261#comment-13501261
]
Harsh J commented on HADOOP-8845:
-
I'll revise the patch here to include fixes for FileContext's globbing
implementation as well, but for the reuse goal I have filed HADOOP-9068 for
later.
When looking for parent paths info, globStatus must filter out non-directory
elements to avoid an AccessControlException
Key: HADOOP-8845
URL: https://issues.apache.org/jira/browse/HADOOP-8845
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: Harsh J
Labels: glob
Attachments: HADOOP-8845.patch, HADOOP-8845.patch, HADOOP-8845.patch
A brief description from my colleague Stephen Fritz who helped discover it:
{code}
[root@node1 ~]# su - hdfs
-bash-4.1$ echo My Test Stringtestfile -- just a text file, for testing
below
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir -- create a directory
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir/1 -- create a subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/1/testfile -- put the test
file in the subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/testfile -- put the test
file in the directory
-bash-4.1$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x - hdfs hadoop 0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
All files are where we expect them...OK, let's try reading
-bash-4.1$ hadoop dfs -cat /tmp/testdir/testfile
My Test String -- success!
-bash-4.1$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String -- success!
-bash-4.1$ hadoop dfs -cat /tmp/testdir/*/testfile
My Test String -- success!
Note that we used an '*' in the cat command, and it correctly found the
subdirectory '/tmp/testdir/1', and ignore the regular file
'/tmp/testdir/testfile'
-bash-4.1$ exit
logout
[root@node1 ~]# su - testuser -- lets try it as a different user:
[testuser@node1 ~]$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x - hdfs hadoop 0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/testfile
My Test String -- good
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String -- so far so good
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/*/testfile
cat: org.apache.hadoop.security.AccessControlException: Permission denied:
user=testuser, access=EXECUTE,
inode=/tmp/testdir/testfile:hdfs:hadoop:-rw-r--r--
{code}
Essentially, we hit a ACE with access=EXECUTE on file /tmp/testdir/testfile
cause we tried to access the /tmp/testdir/testfile/testfile as a path. This
shouldn't happen, as the testfile is a file and not a path parent to be
looked up upon.
{code}
2012-09-25 07:24:27,406 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 8020, call getFileInfo(/tmp/testdir/testfile/testfile)
{code}
Surprisingly the superuser avoids hitting into the error, as a result of
bypassing permissions, but that can be looked up on another JIRA - if it is
fine to let it be like that or not.
This JIRA targets a client-sided fix to not cause such /path/file/dir or
/path/file/file kinda lookups.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501290#comment-13501290 ] Yanbo Liang commented on HADOOP-9041: - @Alejandro， There may be many reasons this has not been observed before: 1, There are no test case cover this situation until now. In other words, no test case registering FsUrlStreamHandlerFactory as the current URLStreamHandlerFactory before calling FileSystem.getFileSystem()-FileSystem.loadFileSystems(). 2, The case is environment dependent. @Radim had attach the junit testcase, but the trunk still run success over this test case before we fix this bug. I also encounter this situation and run this test case base on trunk code on different environment, but with different result. # simple java application, the test case failure as expected. # maven+junit, the test case success(this is not expected). # eclipse+junit, the test case failure as expected. I still work on tracing the different result in different environment. Thanks to @Radim for attaching the junit based test case. FileSystem initialization can go into infinite loop --- Key: HADOOP-9041 URL: https://issues.apache.org/jira/browse/HADOOP-9041 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Radim Kolar Assignee: Yanbo Liang Priority: Critical Attachments: fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, HADOOP-9041.patch More information is there: https://jira.springsource.org/browse/SHDP-111 Referenced source code from example is: https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java from isolating that cause it looks like if you register: org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9069) FileSystem.get leads to stack overflow if default FS is not configured with a scheme
Jason Lowe created HADOOP-9069:
--
Summary: FileSystem.get leads to stack overflow if default FS is
not configured with a scheme
Key: HADOOP-9069
URL: https://issues.apache.org/jira/browse/HADOOP-9069
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.1-alpha, 0.23.3
Reporter: Jason Lowe
Priority: Minor
If fs.defaultFS is configured without a scheme, e.g.: /, then FileSystem.get
will infinitely recurse and lead to a stack overflow. An example stacktrace
from 0.23:
{noformat}
java.lang.StackOverflowError
at java.util.AbstractCollection.init(AbstractCollection.java:66)
at java.util.AbstractList.init(AbstractList.java:76)
at java.util.ArrayList.init(ArrayList.java:128)
at java.util.ArrayList.init(ArrayList.java:139)
at
org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:430)
at org.apache.hadoop.conf.Configuration.get(Configuration.java:852)
at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:171)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:163)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:163)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:290)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:163)
...
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
Daryn Sharp created HADOOP-9070:
---
Summary: Kerberos SASL server cannot find kerberos key
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of the
sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-9070:
Attachment: HADOOP-9070.patch
Restore the {{doAs}}. I can't add tests, but this has been verified to work on
a secure cluster that experienced the failure.
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-9070:
Status: Patch Available (was: Open)
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501319#comment-13501319
]
Robert Joseph Evans commented on HADOOP-9070:
-
The change looks good to me, assuming Jenkins is OK with it. +1
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501336#comment-13501336
]
Hadoop QA commented on HADOOP-9070:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554369/HADOOP-9070.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1780//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1780//console
This message is automatically generated.
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8860) Split MapReduce and YARN sections in documentation navigation
[ https://issues.apache.org/jira/browse/HADOOP-8860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved HADOOP-8860. --- Resolution: Fixed Fix Version/s: 0.23.5 I just pulled this into 0.23 too after discussing with Bobby and Tom. Split MapReduce and YARN sections in documentation navigation - Key: HADOOP-8860 URL: https://issues.apache.org/jira/browse/HADOOP-8860 Project: Hadoop Common Issue Type: Task Components: documentation Affects Versions: 2.0.1-alpha Reporter: Tom White Assignee: Tom White Fix For: 2.0.3-alpha, 0.23.5 Attachments: HADOOP-8860-23.patch, HADOOP-8860-23.sh, HADOOP-8860.patch, HADOOP-8860.patch, HADOOP-8860.sh, HADOOP-8860.sh This JIRA is to change the navigation on http://hadoop.apache.org/docs/r2.0.1-alpha/ to reflect the fact that MapReduce and YARN are separate modules/sub-projects. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9055) POM files for hadoop-core 1.x should depend on Jackson 1.8.8
[ https://issues.apache.org/jira/browse/HADOOP-9055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan resolved HADOOP-9055. Resolution: Duplicate duplicate of HADOOP-8745 POM files for hadoop-core 1.x should depend on Jackson 1.8.8 Key: HADOOP-9055 URL: https://issues.apache.org/jira/browse/HADOOP-9055 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.0.0, 1.1.0 Reporter: Gunther Hagleitner Assignee: Giridharan Kesavan According to this: https://issues.apache.org/jira/browse/HADOOP-7470 Jackson has been upgraded to 1.8.8, but the POMs on the apache maven repo for the hadoop 1.x line still specify 1.0.1 for the library. That's causing build problems for hive (which uses 1.0.0 to build its 20S shim). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8845) When looking for parent paths info, globStatus must filter out non-directory elements to avoid an AccessControlException
[
https://issues.apache.org/jira/browse/HADOOP-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501362#comment-13501362
]
Daryn Sharp commented on HADOOP-8845:
-
Are you sure this patch is still needed? This should have already been fixed
by HADOOP-8906.
When looking for parent paths info, globStatus must filter out non-directory
elements to avoid an AccessControlException
Key: HADOOP-8845
URL: https://issues.apache.org/jira/browse/HADOOP-8845
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 2.0.0-alpha
Reporter: Harsh J
Assignee: Harsh J
Labels: glob
Attachments: HADOOP-8845.patch, HADOOP-8845.patch, HADOOP-8845.patch
A brief description from my colleague Stephen Fritz who helped discover it:
{code}
[root@node1 ~]# su - hdfs
-bash-4.1$ echo My Test Stringtestfile -- just a text file, for testing
below
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir -- create a directory
-bash-4.1$ hadoop dfs -mkdir /tmp/testdir/1 -- create a subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/1/testfile -- put the test
file in the subdirectory
-bash-4.1$ hadoop dfs -put testfile /tmp/testdir/testfile -- put the test
file in the directory
-bash-4.1$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x - hdfs hadoop 0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
All files are where we expect them...OK, let's try reading
-bash-4.1$ hadoop dfs -cat /tmp/testdir/testfile
My Test String -- success!
-bash-4.1$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String -- success!
-bash-4.1$ hadoop dfs -cat /tmp/testdir/*/testfile
My Test String -- success!
Note that we used an '*' in the cat command, and it correctly found the
subdirectory '/tmp/testdir/1', and ignore the regular file
'/tmp/testdir/testfile'
-bash-4.1$ exit
logout
[root@node1 ~]# su - testuser -- lets try it as a different user:
[testuser@node1 ~]$ hadoop dfs -lsr /tmp/testdir
drwxr-xr-x - hdfs hadoop 0 2012-09-25 06:52 /tmp/testdir/1
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/1/testfile
-rw-r--r-- 3 hdfs hadoop 15 2012-09-25 06:52 /tmp/testdir/testfile
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/testfile
My Test String -- good
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/1/testfile
My Test String -- so far so good
[testuser@node1 ~]$ hadoop dfs -cat /tmp/testdir/*/testfile
cat: org.apache.hadoop.security.AccessControlException: Permission denied:
user=testuser, access=EXECUTE,
inode=/tmp/testdir/testfile:hdfs:hadoop:-rw-r--r--
{code}
Essentially, we hit a ACE with access=EXECUTE on file /tmp/testdir/testfile
cause we tried to access the /tmp/testdir/testfile/testfile as a path. This
shouldn't happen, as the testfile is a file and not a path parent to be
looked up upon.
{code}
2012-09-25 07:24:27,406 INFO org.apache.hadoop.ipc.Server: IPC Server
handler 2 on 8020, call getFileInfo(/tmp/testdir/testfile/testfile)
{code}
Surprisingly the superuser avoids hitting into the error, as a result of
bypassing permissions, but that can be looked up on another JIRA - if it is
fine to let it be like that or not.
This JIRA targets a client-sided fix to not cause such /path/file/dir or
/path/file/file kinda lookups.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9071) configure ivy log levels for resolve/retrieve
Giridharan Kesavan created HADOOP-9071: -- Summary: configure ivy log levels for resolve/retrieve Key: HADOOP-9071 URL: https://issues.apache.org/jira/browse/HADOOP-9071 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 1.1.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9071) configure ivy log levels for resolve/retrieve
[ https://issues.apache.org/jira/browse/HADOOP-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501381#comment-13501381 ] Giridharan Kesavan commented on HADOOP-9071: build user should be able to configure ivy resolve/retrieve log levels - configure ivy log levels for resolve/retrieve - Key: HADOOP-9071 URL: https://issues.apache.org/jira/browse/HADOOP-9071 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 1.1.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9041) FileSystem initialization can go into infinite loop
[ https://issues.apache.org/jira/browse/HADOOP-9041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501382#comment-13501382 ] Radim Kolar commented on HADOOP-9041: - in maven + junit it probably didnt fails because JVM is reused and its already initialized. FileSystem initialization can go into infinite loop --- Key: HADOOP-9041 URL: https://issues.apache.org/jira/browse/HADOOP-9041 Project: Hadoop Common Issue Type: Bug Components: fs Affects Versions: 2.0.2-alpha Reporter: Radim Kolar Assignee: Yanbo Liang Priority: Critical Attachments: fsinit-unit.txt, fstest.groovy, HADOOP-9041.patch, HADOOP-9041.patch More information is there: https://jira.springsource.org/browse/SHDP-111 Referenced source code from example is: https://github.com/SpringSource/spring-hadoop/blob/master/src/main/java/org/springframework/data/hadoop/configuration/ConfigurationFactoryBean.java from isolating that cause it looks like if you register: org.apache.hadoop.fs.FsUrlStreamHandlerFactory before calling FileSystem.loadFileSystems() then it goes into infinite loop. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9071) configure ivy log levels for resolve/retrieve
[
https://issues.apache.org/jira/browse/HADOOP-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Giridharan Kesavan updated HADOOP-9071:
---
Attachment: HADOOP-9071.PATCH
with this patch loglevels could be set from the command line.
{code}
ant -Dloglevel = quiet | default | download-only
{code}
configure ivy log levels for resolve/retrieve
-
Key: HADOOP-9071
URL: https://issues.apache.org/jira/browse/HADOOP-9071
Project: Hadoop Common
Issue Type: Improvement
Components: build
Affects Versions: 1.1.0
Reporter: Giridharan Kesavan
Assignee: Giridharan Kesavan
Attachments: HADOOP-9071.PATCH
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated HADOOP-9072: -- Status: Patch Available (was: Open) Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Parker updated HADOOP-9072: -- Attachment: HADOOP-9072.patch Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9071) configure ivy log levels for resolve/retrieve
[ https://issues.apache.org/jira/browse/HADOOP-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giridharan Kesavan updated HADOOP-9071: --- Attachment: HADOOP-9071-V1.patch add the loglevel property to build-contrib.xml configure ivy log levels for resolve/retrieve - Key: HADOOP-9071 URL: https://issues.apache.org/jira/browse/HADOOP-9071 Project: Hadoop Common Issue Type: Improvement Components: build Affects Versions: 1.1.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Attachments: HADOOP-9071.PATCH, HADOOP-9071-V1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9056) Build native library on Windows
[ https://issues.apache.org/jira/browse/HADOOP-9056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501440#comment-13501440 ] Arpit Agarwal commented on HADOOP-9056: --- Attached a patch to port Chuan Liu's changes to branch-trunk-win. Some additional changes to account for branch divergence. Build native library on Windows --- Key: HADOOP-9056 URL: https://issues.apache.org/jira/browse/HADOOP-9056 Project: Hadoop Common Issue Type: Improvement Components: native Affects Versions: trunk-win Reporter: Arpit Agarwal Assignee: Arpit Agarwal Fix For: trunk-win Attachments: HADOOP-9056.patch Original Estimate: 168h Remaining Estimate: 168h The native library (hadoop.dll) must be compiled on Windows. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9042) Add a test for umask in FileSystemContractBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501469#comment-13501469
]
Eli Collins commented on HADOOP-9042:
-
+1 latest patch looks good
Add a test for umask in FileSystemContractBaseTest
--
Key: HADOOP-9042
URL: https://issues.apache.org/jira/browse/HADOOP-9042
Project: Hadoop Common
Issue Type: Test
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Fix For: 2.0.3-alpha
Attachments: HADOOP-9042.008.patch, HADOOP-9042.009.patch,
HADOOP-9042.010.patch, HDFS-3975.001.patch, HDFS-3975.002.patch,
HDFS-3975.003.patch, HDFS-3975.004.patch, HDFS-3975.005.patch,
HDFS-3975.006.patch
Add a unit test to make sure {{umask}} is working correctly in
FileSystemContractBaseTest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501482#comment-13501482 ] Thomas Graves commented on HADOOP-9072: --- +1 lgtm. Thanks Rob. Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8934) Shell command ls should include sort options
[ https://issues.apache.org/jira/browse/HADOOP-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501486#comment-13501486 ] Jonathan Allen commented on HADOOP-8934: Daryn - did you have a change to review this yet? Shell command ls should include sort options Key: HADOOP-8934 URL: https://issues.apache.org/jira/browse/HADOOP-8934 Project: Hadoop Common Issue Type: Improvement Components: fs Reporter: Jonathan Allen Assignee: Jonathan Allen Priority: Minor Attachments: HADOOP-8934.patch, HADOOP-8934.patch The shell command ls should include options to sort the output similar to the unix ls command. The following options seem appropriate: -t : sort by modification time -S : sort by file size -r : reverse the sort order -u : use access time rather than modification time for sort and display -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8878) uppercase namenode hostname causes hadoop dfs calls with webhdfs filesystem and fsck to fail when security is on
[ https://issues.apache.org/jira/browse/HADOOP-8878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-8878: --- Fix Version/s: (was: 1.2.0) 1.1.1 uppercase namenode hostname causes hadoop dfs calls with webhdfs filesystem and fsck to fail when security is on Key: HADOOP-8878 URL: https://issues.apache.org/jira/browse/HADOOP-8878 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.0.3, 1.1.0, 1.2.0, 3.0.0 Reporter: Arpit Gupta Assignee: Arpit Gupta Fix For: 1.1.1, 2.0.3-alpha Attachments: HADOOP-8878.branch-1.patch, HADOOP-8878.branch-1.patch, HADOOP-8878.branch-1.patch, HADOOP-8878.patch, HADOOP-8878.patch, HADOOP-8878.patch This was noticed on a secure cluster where the namenode had an upper case hostname and the following command was issued hadoop dfs -ls webhdfs://NN:PORT/PATH the above command failed because delegation token retrieval failed. Upon looking at the kerberos logs it was determined that we tried to get the ticket for kerberos principal with upper case hostnames and that host did not exit in kerberos. We should convert the hostnames to lower case. Take a look at HADOOP-7988 where the same fix was applied on a different class. I have noticed this issue exists on branch-1. Will investigate trunk and branch-2 and update accordingly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9042) Add a test for umask in FileSystemContractBaseTest
[
https://issues.apache.org/jira/browse/HADOOP-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501491#comment-13501491
]
Hudson commented on HADOOP-9042:
Integrated in Hadoop-trunk-Commit #3049 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/3049/])
HADOOP-9042. Add a test for umask in FileSystemContractBaseTest.
Contributed by Colin McCabe (Revision 1411879)
Result = SUCCESS
eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411879
Files :
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
*
/hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
*
/hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestHDFSFileSystemContract.java
Add a test for umask in FileSystemContractBaseTest
--
Key: HADOOP-9042
URL: https://issues.apache.org/jira/browse/HADOOP-9042
Project: Hadoop Common
Issue Type: Test
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Fix For: 2.0.3-alpha
Attachments: HADOOP-9042.008.patch, HADOOP-9042.009.patch,
HADOOP-9042.010.patch, HDFS-3975.001.patch, HDFS-3975.002.patch,
HDFS-3975.003.patch, HDFS-3975.004.patch, HDFS-3975.005.patch,
HDFS-3975.006.patch
Add a unit test to make sure {{umask}} is working correctly in
FileSystemContractBaseTest.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8882) uppercase namenode host name causes fsck to fail when useKsslAuth is on
[
https://issues.apache.org/jira/browse/HADOOP-8882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501492#comment-13501492
]
Matt Foley commented on HADOOP-8882:
included in branch-1.1
uppercase namenode host name causes fsck to fail when useKsslAuth is on
---
Key: HADOOP-8882
URL: https://issues.apache.org/jira/browse/HADOOP-8882
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 1.0.3, 1.1.0, 1.2.0
Reporter: Arpit Gupta
Assignee: Arpit Gupta
Fix For: 1.1.1
Attachments: HADOOP-8882.branch-1.patch, HADOOP-8882.branch-1.patch
{code}
public static void fetchServiceTicket(URL remoteHost) throws IOException {
if(!UserGroupInformation.isSecurityEnabled())
return;
String serviceName = host/ + remoteHost.getHost();
{code}
the hostname should be converted to lower case. Saw this in branch 1, will
look at trunk and update the bug accordingly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8995) Remove unnecessary bogus exception log from Configuration
[ https://issues.apache.org/jira/browse/HADOOP-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-8995: --- Fix Version/s: (was: 1.2.0) 1.1.1 Remove unnecessary bogus exception log from Configuration - Key: HADOOP-8995 URL: https://issues.apache.org/jira/browse/HADOOP-8995 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jing Zhao Assignee: Jing Zhao Priority: Minor Fix For: 1.1.1 Attachments: Hadoop.8995.b1.001.patch In Configuration#Configuration(boolean) and Configuration#Configuration(Configuration), bogus exceptions are thrown when Log level is DEBUG. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8995) Remove unnecessary bogus exception log from Configuration
[ https://issues.apache.org/jira/browse/HADOOP-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501495#comment-13501495 ] Matt Foley commented on HADOOP-8995: included in branch-1.1 Remove unnecessary bogus exception log from Configuration - Key: HADOOP-8995 URL: https://issues.apache.org/jira/browse/HADOOP-8995 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jing Zhao Assignee: Jing Zhao Priority: Minor Fix For: 1.1.1 Attachments: Hadoop.8995.b1.001.patch In Configuration#Configuration(boolean) and Configuration#Configuration(Configuration), bogus exceptions are thrown when Log level is DEBUG. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated HADOOP-9072: -- Resolution: Fixed Fix Version/s: 0.23.6 2.0.3-alpha 3.0.0 Status: Resolved (was: Patch Available) Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9017) fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for version
[ https://issues.apache.org/jira/browse/HADOOP-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-9017: --- Fix Version/s: (was: 1.2.0) fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for version -- Key: HADOOP-9017 URL: https://issues.apache.org/jira/browse/HADOOP-9017 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 1.0.4, 1.1.0 Reporter: Giridharan Kesavan Assignee: Giridharan Kesavan Fix For: 1.1.1 Attachments: HADOOP-9017.patch hadoop-client-pom-template.xml and hadoop-client-pom-template.xml references to project.version variable, instead they should refer to @version token. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8916) make it possible to build hadoop tarballs without java5+ forrest
[
https://issues.apache.org/jira/browse/HADOOP-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Foley updated HADOOP-8916:
---
Fix Version/s: (was: 1.1.1)
1.2.0
make it possible to build hadoop tarballs without java5+ forrest
Key: HADOOP-8916
URL: https://issues.apache.org/jira/browse/HADOOP-8916
Project: Hadoop Common
Issue Type: Improvement
Components: build
Affects Versions: 1.0.3
Reporter: Steve Loughran
Priority: Trivial
Fix For: 1.2.0
Attachments: HADOOP-8916.patch
Original Estimate: 1m
Remaining Estimate: 1m
Although you can build hadoop binaries without java5 Forrest, you can't do
the tarballs as {{tar}} depends on {{packaged}}, which depends on the
{{docs}} and {{cn-docs}}, which both depend on the forrest/java5 checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8916) make it possible to build hadoop tarballs without java5+ forrest
[
https://issues.apache.org/jira/browse/HADOOP-8916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501507#comment-13501507
]
Matt Foley commented on HADOOP-8916:
Is this a superior solution vs. HADOOP-8166 and HADOOP-8399? If so, please
submit to 1.2 (via branch-1), otherwise, please close as duplicate. Thanks.
make it possible to build hadoop tarballs without java5+ forrest
Key: HADOOP-8916
URL: https://issues.apache.org/jira/browse/HADOOP-8916
Project: Hadoop Common
Issue Type: Improvement
Components: build
Affects Versions: 1.0.3
Reporter: Steve Loughran
Priority: Trivial
Fix For: 1.2.0
Attachments: HADOOP-8916.patch
Original Estimate: 1m
Remaining Estimate: 1m
Although you can build hadoop binaries without java5 Forrest, you can't do
the tarballs as {{tar}} depends on {{packaged}}, which depends on the
{{docs}} and {{cn-docs}}, which both depend on the forrest/java5 checks.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9072) Hadoop-Common-0.23-Build Fails to build in Jenkins
[ https://issues.apache.org/jira/browse/HADOOP-9072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501506#comment-13501506 ] Hudson commented on HADOOP-9072: Integrated in Hadoop-trunk-Commit #3050 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3050/]) HADOOP-9072. Hadoop-Common-0.23-Build Fails to build in Jenkins (Robert Parker via tgraves) (Revision 1411884) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411884 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/pom.xml Hadoop-Common-0.23-Build Fails to build in Jenkins -- Key: HADOOP-9072 URL: https://issues.apache.org/jira/browse/HADOOP-9072 Project: Hadoop Common Issue Type: Bug Affects Versions: 0.23.4 Reporter: Robert Parker Assignee: Robert Parker Fix For: 3.0.0, 2.0.3-alpha, 0.23.6 Attachments: HADOOP-9072.patch [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:testCompile (default-testCompile) on project hadoop-common: Compilation failure: Compilation failure: [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecInt.java:[4,7] duplicate class: org.apache.hadoop.record.RecInt [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecRecordOld.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordOld [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecString.java:[4,7] duplicate class: org.apache.hadoop.record.RecString [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/generated-test-sources/java/org/apache/hadoop/record/RecBuffer.java:[4,7] duplicate class: org.apache.hadoop.record.RecBuffer [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord1.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord1 [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecordNew.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecordNew [ERROR] /home/jenkins/jenkins-slave/workspace/Hadoop-Common-0.23-Build/trunk/hadoop-common-project/hadoop-common/target/clover/src-test-instrumented/org/apache/hadoop/record/RecRecord0.java:[4,7] duplicate class: org.apache.hadoop.record.RecRecord0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-6453) Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH
[ https://issues.apache.org/jira/browse/HADOOP-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501512#comment-13501512 ] Matt Foley commented on HADOOP-6453: closing for apparent lack of interest. Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH Key: HADOOP-6453 URL: https://issues.apache.org/jira/browse/HADOOP-6453 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 0.20.2, 0.21.0, 0.22.0 Reporter: Chad Metcalf Priority: Minor Fix For: 0.22.1 Attachments: HADOOP-6453-0.20.patch, HADOOP-6453-0.20v2.patch, HADOOP-6453-0.20v3.patch, HADOOP-6453.trunk.patch, HADOOP-6453-trunkv2.patch, HADOOP-6453-trunkv3.patch Currently the hadoop wrapper script assumes its the only place that uses JAVA_LIBRARY_PATH and initializes it to a blank line. JAVA_LIBRARY_PATH='' This prevents anyone from setting this outside of the hadoop wrapper (say hadoop-config.sh) for their own native libraries. The fix is pretty simple. Don't initialize it to '' and append the native libs like normal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-6453) Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH
[ https://issues.apache.org/jira/browse/HADOOP-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-6453: --- Target Version/s: 0.22.0 (was: 1.1.1, 0.22.0) Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH Key: HADOOP-6453 URL: https://issues.apache.org/jira/browse/HADOOP-6453 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 0.20.2, 0.21.0, 0.22.0 Reporter: Chad Metcalf Priority: Minor Fix For: 0.22.1 Attachments: HADOOP-6453-0.20.patch, HADOOP-6453-0.20v2.patch, HADOOP-6453-0.20v3.patch, HADOOP-6453.trunk.patch, HADOOP-6453-trunkv2.patch, HADOOP-6453-trunkv3.patch Currently the hadoop wrapper script assumes its the only place that uses JAVA_LIBRARY_PATH and initializes it to a blank line. JAVA_LIBRARY_PATH='' This prevents anyone from setting this outside of the hadoop wrapper (say hadoop-config.sh) for their own native libraries. The fix is pretty simple. Don't initialize it to '' and append the native libs like normal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-6453) Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH
[ https://issues.apache.org/jira/browse/HADOOP-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley resolved HADOOP-6453. Resolution: Fixed Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH Key: HADOOP-6453 URL: https://issues.apache.org/jira/browse/HADOOP-6453 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 0.20.2, 0.21.0, 0.22.0 Reporter: Chad Metcalf Priority: Minor Fix For: 0.22.1 Attachments: HADOOP-6453-0.20.patch, HADOOP-6453-0.20v2.patch, HADOOP-6453-0.20v3.patch, HADOOP-6453.trunk.patch, HADOOP-6453-trunkv2.patch, HADOOP-6453-trunkv3.patch Currently the hadoop wrapper script assumes its the only place that uses JAVA_LIBRARY_PATH and initializes it to a blank line. JAVA_LIBRARY_PATH='' This prevents anyone from setting this outside of the hadoop wrapper (say hadoop-config.sh) for their own native libraries. The fix is pretty simple. Don't initialize it to '' and append the native libs like normal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (HADOOP-6453) Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH
[ https://issues.apache.org/jira/browse/HADOOP-6453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley closed HADOOP-6453. -- Hadoop wrapper script shouldn't ignore an existing JAVA_LIBRARY_PATH Key: HADOOP-6453 URL: https://issues.apache.org/jira/browse/HADOOP-6453 Project: Hadoop Common Issue Type: Bug Components: scripts Affects Versions: 0.20.2, 0.21.0, 0.22.0 Reporter: Chad Metcalf Priority: Minor Fix For: 0.22.1 Attachments: HADOOP-6453-0.20.patch, HADOOP-6453-0.20v2.patch, HADOOP-6453-0.20v3.patch, HADOOP-6453.trunk.patch, HADOOP-6453-trunkv2.patch, HADOOP-6453-trunkv3.patch Currently the hadoop wrapper script assumes its the only place that uses JAVA_LIBRARY_PATH and initializes it to a blank line. JAVA_LIBRARY_PATH='' This prevents anyone from setting this outside of the hadoop wrapper (say hadoop-config.sh) for their own native libraries. The fix is pretty simple. Don't initialize it to '' and append the native libs like normal. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7825) Hadoop wrapper script not picking up native libs correctly
[
https://issues.apache.org/jira/browse/HADOOP-7825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501513#comment-13501513
]
Matt Foley commented on HADOOP-7825:
It appears no work is being done on this, so removing the target version.
Hadoop wrapper script not picking up native libs correctly
--
Key: HADOOP-7825
URL: https://issues.apache.org/jira/browse/HADOOP-7825
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Affects Versions: 0.20.205.0
Environment: Debian 6.0 x64_64
java version 1.6.0_26
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
Reporter: stephen mulcahy
Originally discussed in
https://mail-archives.apache.org/mod_mbox/hadoop-common-user/20.mbox/%3C4EC3A3AE.7060402%40deri.org%3E
I'm testing out native lib support on our test amd64 test cluster
running 0.20.205 running the following
./bin/hadoop jar hadoop-test-0.20.205.0.jar testsequencefile -seed 0
-count 1000 -compressType RECORD xxx -codec
org.apache.hadoop.io.compress.GzipCodec -check 2
it fails with
WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
Looking at
bin/hadoop
it seems to successfully detect that the native libs are available (they
seem to come pre-compiled with 0.20.205 which is nice)
if [ -d ${HADOOP_HOME}/lib/native ]; then
if [ x$JAVA_LIBRARY_PATH != x ]; then
JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:${HADOOP_HOME}/lib/native/${JAVA_PLATFORM}
else
JAVA_LIBRARY_PATH=${HADOOP_HOME}/lib/native/${JAVA_PLATFORM}
fi
fi
and sets JAVA_LIBRARY_PATH to contain them.
Then in the following line, if ${HADOOP_HOME}/lib contains libhadoop.a
(which is seems to in the stock tar) then it proceeds to ignore the
native libs
if [ -e ${HADOOP_PREFIX}/lib/libhadoop.a ]; then
JAVA_LIBRARY_PATH=${HADOOP_PREFIX}/lib
fi
The libhadoop.a in ${HADOOP_HOME}/lib seems to be a copy of the
lib/native/Linux-i386-32 going from the filesizes (and also noted by
https://mail-archives.apache.org/mod_mbox/hadoop-common-user/20.mbox/%3ccaocnvr2azudnn0lfhmtqumayujytvhfkmmm_j0r-bmxw2wu...@mail.gmail.com%3E)
hadoop@testhbase01:~$ ls -la hadoop/lib/libhadoop.*
-rw-r--r-- 1 hadoop hadoop 237244 Oct 7 08:20 hadoop/lib/libhadoop.a
-rw-r--r-- 1 hadoop hadoop877 Oct 7 08:20 hadoop/lib/libhadoop.la
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 hadoop/lib/libhadoop.so
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:19 hadoop/lib/libhadoop.so.1.0.0
hadoop@testhbase01:~$ ls -la hadoop/lib/native/Linux-i386-32/
total 728
drwxr-xr-x 3 hadoop hadoop 4096 Nov 15 14:05 .
drwxr-xr-x 5 hadoop hadoop 4096 Oct 7 08:24 ..
-rw-r--r-- 1 hadoop hadoop 237244 Oct 7 08:20 libhadoop.a
-rw-r--r-- 1 hadoop hadoop877 Oct 7 08:20 libhadoop.la
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 libhadoop.so
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 libhadoop.so.1
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 libhadoop.so.1.0.0
A possible solution includes removing libhadoop.a and friends from
${HADOOP_HOME}/lib and possibly also modifying the hadoop wrapper to remove
if [ -e ${HADOOP_PREFIX}/lib/libhadoop.a ]; then
JAVA_LIBRARY_PATH=${HADOOP_PREFIX}/lib
fi
unless there is some other reason for this to exist.
This was also noted in a comment to HADOOP-6453
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7825) Hadoop wrapper script not picking up native libs correctly
[
https://issues.apache.org/jira/browse/HADOOP-7825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Foley updated HADOOP-7825:
---
Target Version/s: (was: 1.2.0)
Hadoop wrapper script not picking up native libs correctly
--
Key: HADOOP-7825
URL: https://issues.apache.org/jira/browse/HADOOP-7825
Project: Hadoop Common
Issue Type: Bug
Components: scripts
Affects Versions: 0.20.205.0
Environment: Debian 6.0 x64_64
java version 1.6.0_26
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
Reporter: stephen mulcahy
Originally discussed in
https://mail-archives.apache.org/mod_mbox/hadoop-common-user/20.mbox/%3C4EC3A3AE.7060402%40deri.org%3E
I'm testing out native lib support on our test amd64 test cluster
running 0.20.205 running the following
./bin/hadoop jar hadoop-test-0.20.205.0.jar testsequencefile -seed 0
-count 1000 -compressType RECORD xxx -codec
org.apache.hadoop.io.compress.GzipCodec -check 2
it fails with
WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
Looking at
bin/hadoop
it seems to successfully detect that the native libs are available (they
seem to come pre-compiled with 0.20.205 which is nice)
if [ -d ${HADOOP_HOME}/lib/native ]; then
if [ x$JAVA_LIBRARY_PATH != x ]; then
JAVA_LIBRARY_PATH=${JAVA_LIBRARY_PATH}:${HADOOP_HOME}/lib/native/${JAVA_PLATFORM}
else
JAVA_LIBRARY_PATH=${HADOOP_HOME}/lib/native/${JAVA_PLATFORM}
fi
fi
and sets JAVA_LIBRARY_PATH to contain them.
Then in the following line, if ${HADOOP_HOME}/lib contains libhadoop.a
(which is seems to in the stock tar) then it proceeds to ignore the
native libs
if [ -e ${HADOOP_PREFIX}/lib/libhadoop.a ]; then
JAVA_LIBRARY_PATH=${HADOOP_PREFIX}/lib
fi
The libhadoop.a in ${HADOOP_HOME}/lib seems to be a copy of the
lib/native/Linux-i386-32 going from the filesizes (and also noted by
https://mail-archives.apache.org/mod_mbox/hadoop-common-user/20.mbox/%3ccaocnvr2azudnn0lfhmtqumayujytvhfkmmm_j0r-bmxw2wu...@mail.gmail.com%3E)
hadoop@testhbase01:~$ ls -la hadoop/lib/libhadoop.*
-rw-r--r-- 1 hadoop hadoop 237244 Oct 7 08:20 hadoop/lib/libhadoop.a
-rw-r--r-- 1 hadoop hadoop877 Oct 7 08:20 hadoop/lib/libhadoop.la
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 hadoop/lib/libhadoop.so
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:19 hadoop/lib/libhadoop.so.1.0.0
hadoop@testhbase01:~$ ls -la hadoop/lib/native/Linux-i386-32/
total 728
drwxr-xr-x 3 hadoop hadoop 4096 Nov 15 14:05 .
drwxr-xr-x 5 hadoop hadoop 4096 Oct 7 08:24 ..
-rw-r--r-- 1 hadoop hadoop 237244 Oct 7 08:20 libhadoop.a
-rw-r--r-- 1 hadoop hadoop877 Oct 7 08:20 libhadoop.la
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 libhadoop.so
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 libhadoop.so.1
-rw-r--r-- 1 hadoop hadoop 160438 Oct 7 08:20 libhadoop.so.1.0.0
A possible solution includes removing libhadoop.a and friends from
${HADOOP_HOME}/lib and possibly also modifying the hadoop wrapper to remove
if [ -e ${HADOOP_PREFIX}/lib/libhadoop.a ]; then
JAVA_LIBRARY_PATH=${HADOOP_PREFIX}/lib
fi
unless there is some other reason for this to exist.
This was also noted in a comment to HADOOP-6453
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daryn Sharp updated HADOOP-9070:
Attachment: HADOOP-9070.patch
Original patch was not tested correctly... There is also a problem with the
client server knowing in all cases that SASL negotiation is complete. Due to
an earlier change in HADOOP-8999 to stop PLAIN clients from hanging waiting for
a response from the server, kerberos clients may now hang. In HADOOP-9034
there is general agreement that the server should always send a success
response, so I removed the conditional to only return success for PLAIN. I'd
like to leave HADOOP-9034 open to implement a more robust solution. Note this
patch causes RPC incompatibility, but I don't think this is a problem since 2.x
is RPC incompatible with earlier releases anyway.
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7030) Add TableMapping topology implementation to read host to rack mapping from a file
[ https://issues.apache.org/jira/browse/HADOOP-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-7030: --- Target Version/s: 0.23.3, 1.2.0 (was: 1.1.1, 0.23.3) Add TableMapping topology implementation to read host to rack mapping from a file - Key: HADOOP-7030 URL: https://issues.apache.org/jira/browse/HADOOP-7030 Project: Hadoop Common Issue Type: New Feature Affects Versions: 0.20.1, 0.20.2, 0.21.0 Reporter: Patrick Angeles Assignee: Tom White Fix For: 2.0.0-alpha Attachments: HADOOP-7030-2.patch, HADOOP-7030-branch-1.patch, HADOOP-7030.patch, HADOOP-7030.patch, HADOOP-7030.patch, HADOOP-7030.patch, topology.patch The default ScriptBasedMapping implementation of DNSToSwitchMapping for determining cluster topology has some drawbacks. Principally, it forks to an OS-specific script. This issue proposes two new Java implementations of DNSToSwitchMapping. TableMapping reads a two column text file that maps an IP or hostname to a rack ID. Ip4RangeMapping reads a three column text file where each line represents a start and end IP range plus a rack ID. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7030) Add TableMapping topology implementation to read host to rack mapping from a file
[ https://issues.apache.org/jira/browse/HADOOP-7030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501515#comment-13501515 ] Matt Foley commented on HADOOP-7030: moved target version to 1.2.0 upon publishing 1.1.1 RC. Add TableMapping topology implementation to read host to rack mapping from a file - Key: HADOOP-7030 URL: https://issues.apache.org/jira/browse/HADOOP-7030 Project: Hadoop Common Issue Type: New Feature Affects Versions: 0.20.1, 0.20.2, 0.21.0 Reporter: Patrick Angeles Assignee: Tom White Fix For: 2.0.0-alpha Attachments: HADOOP-7030-2.patch, HADOOP-7030-branch-1.patch, HADOOP-7030.patch, HADOOP-7030.patch, HADOOP-7030.patch, HADOOP-7030.patch, topology.patch The default ScriptBasedMapping implementation of DNSToSwitchMapping for determining cluster topology has some drawbacks. Principally, it forks to an OS-specific script. This issue proposes two new Java implementations of DNSToSwitchMapping. TableMapping reads a two column text file that maps an IP or hostname to a rack ID. Ip4RangeMapping reads a three column text file where each line represents a start and end IP range plus a rack ID. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7548) ant binary target fails if native has not been built
[ https://issues.apache.org/jira/browse/HADOOP-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501516#comment-13501516 ] Matt Foley commented on HADOOP-7548: moved target version to 1.2.0 upon publishing 1.1.1 RC. ant binary target fails if native has not been built Key: HADOOP-7548 URL: https://issues.apache.org/jira/browse/HADOOP-7548 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 0.20.205.0 Reporter: Eli Collins The binary target on branch-0.20-security fails with the following, it assumes the native dir exists. BUILD FAILED /home/eli/src/hadoop-branch-0.20-security/build.xml:1572: /home/eli/src/hadoop-branch-0.20-security/build/hadoop-0.20.206.0-SNAPSHOT/native not found. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7548) ant binary target fails if native has not been built
[ https://issues.apache.org/jira/browse/HADOOP-7548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-7548: --- Target Version/s: 1.2.0 (was: 1.1.1) ant binary target fails if native has not been built Key: HADOOP-7548 URL: https://issues.apache.org/jira/browse/HADOOP-7548 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 0.20.205.0 Reporter: Eli Collins The binary target on branch-0.20-security fails with the following, it assumes the native dir exists. BUILD FAILED /home/eli/src/hadoop-branch-0.20-security/build.xml:1572: /home/eli/src/hadoop-branch-0.20-security/build/hadoop-0.20.206.0-SNAPSHOT/native not found. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7675) Ant option to run disabled kerberos authentication tests.
[ https://issues.apache.org/jira/browse/HADOOP-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-7675: --- Target Version/s: 1.2.0 (was: 1.1.1) Ant option to run disabled kerberos authentication tests. - Key: HADOOP-7675 URL: https://issues.apache.org/jira/browse/HADOOP-7675 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jitendra Nath Pandey The kerberos tests, TestKerberosAuthenticator and TestKerberosAuthenticationHandler, are disabled using @Ignore. A better approach would be to have an ant option to run them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7678) Nightly build+test should run with continue on error for automated testing after successful build
[ https://issues.apache.org/jira/browse/HADOOP-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-7678: --- Target Version/s: 0.23.0, 1.2.0 (was: 1.1.1, 0.23.0) Nightly build+test should run with continue on error for automated testing after successful build --- Key: HADOOP-7678 URL: https://issues.apache.org/jira/browse/HADOOP-7678 Project: Hadoop Common Issue Type: Test Components: build Affects Versions: 0.20.205.0, 0.23.0 Reporter: Matt Foley Assignee: Giridharan Kesavan It appears that scripts for nightly build in Apache Jenkins will stop after unit testing if any unit tests fail. Therefore, contribs, schedulers, and some other system-level automated tests don't ever run until the unit tests are clean. This results in two-phase cleanup of broken builds, which wastes developers' time. Please change them to run even in the presence of unit test errors, as long as the compile+packaging build successfully. This jira does not relate to CI builds, which emphasize test-patch execution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7675) Ant option to run disabled kerberos authentication tests.
[ https://issues.apache.org/jira/browse/HADOOP-7675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501517#comment-13501517 ] Matt Foley commented on HADOOP-7675: moved target version to 1.2.0 upon publishing 1.1.1 RC. Ant option to run disabled kerberos authentication tests. - Key: HADOOP-7675 URL: https://issues.apache.org/jira/browse/HADOOP-7675 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.1.0 Reporter: Jitendra Nath Pandey The kerberos tests, TestKerberosAuthenticator and TestKerberosAuthenticationHandler, are disabled using @Ignore. A better approach would be to have an ant option to run them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7678) Nightly build+test should run with continue on error for automated testing after successful build
[ https://issues.apache.org/jira/browse/HADOOP-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501518#comment-13501518 ] Matt Foley commented on HADOOP-7678: moved target version to 1.2.0 upon publishing 1.1.1 RC. Nightly build+test should run with continue on error for automated testing after successful build --- Key: HADOOP-7678 URL: https://issues.apache.org/jira/browse/HADOOP-7678 Project: Hadoop Common Issue Type: Test Components: build Affects Versions: 0.20.205.0, 0.23.0 Reporter: Matt Foley Assignee: Giridharan Kesavan It appears that scripts for nightly build in Apache Jenkins will stop after unit testing if any unit tests fail. Therefore, contribs, schedulers, and some other system-level automated tests don't ever run until the unit tests are clean. This results in two-phase cleanup of broken builds, which wastes developers' time. Please change them to run even in the presence of unit test errors, as long as the compile+packaging build successfully. This jira does not relate to CI builds, which emphasize test-patch execution. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7847) branch-1 has findbugs warnings
[ https://issues.apache.org/jira/browse/HADOOP-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501519#comment-13501519 ] Matt Foley commented on HADOOP-7847: moved target version to 1.2.0 upon publishing 1.1.1 RC. branch-1 has findbugs warnings -- Key: HADOOP-7847 URL: https://issues.apache.org/jira/browse/HADOOP-7847 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.1.0 Reporter: Eli Collins Labels: newbie A month or so ago test-patch used to run cleanly. There are now two findbugs warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7847) branch-1 has findbugs warnings
[ https://issues.apache.org/jira/browse/HADOOP-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-7847: --- Target Version/s: 1.2.0 (was: 1.1.1) branch-1 has findbugs warnings -- Key: HADOOP-7847 URL: https://issues.apache.org/jira/browse/HADOOP-7847 Project: Hadoop Common Issue Type: Bug Affects Versions: 1.1.0 Reporter: Eli Collins Labels: newbie A month or so ago test-patch used to run cleanly. There are now two findbugs warnings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7723) Automatically generate good Release Notes
[ https://issues.apache.org/jira/browse/HADOOP-7723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-7723: --- Target Version/s: 0.23.0, 1.2.0 (was: 1.1.1, 0.23.0) Automatically generate good Release Notes - Key: HADOOP-7723 URL: https://issues.apache.org/jira/browse/HADOOP-7723 Project: Hadoop Common Issue Type: Improvement Affects Versions: 0.20.204.0, 0.23.0 Reporter: Matt Foley Assignee: Matt Foley In branch-0.20-security, there is a tool src/docs/relnotes.py, that automatically generates Release Notes. Fix deficiencies and port it up to trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8402) Add support for generating pdf clover report in 1.1 release
[ https://issues.apache.org/jira/browse/HADOOP-8402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-8402: --- Target Version/s: 1.2.0 (was: 1.1.1) Add support for generating pdf clover report in 1.1 release --- Key: HADOOP-8402 URL: https://issues.apache.org/jira/browse/HADOOP-8402 Project: Hadoop Common Issue Type: Bug Components: build Affects Versions: 1.0.0 Reporter: Suresh Srinivas Priority: Minor Attachments: HADOOP-8402.txt Add support for generating clover PDF report. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8477) Pull in Yahoo! Hadoop Tutorial and update it accordingly.
[ https://issues.apache.org/jira/browse/HADOOP-8477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-8477: --- Target Version/s: 1.2.0, 3.0.0 (was: 1.1.1, 3.0.0) Pull in Yahoo! Hadoop Tutorial and update it accordingly. - Key: HADOOP-8477 URL: https://issues.apache.org/jira/browse/HADOOP-8477 Project: Hadoop Common Issue Type: Improvement Components: documentation Affects Versions: 1.1.0, 2.0.0-alpha Reporter: Robert Joseph Evans Attachments: tutorial.tgz I was able to get the Yahoo! Hadoop tutorial released under an Apache 2.0 license. This allows us to make it a official part of the Hadoop Project. This ticket is to pull the tutorial and update it as needed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8515) Upgrade Jetty to the current Jetty 7 release
[ https://issues.apache.org/jira/browse/HADOOP-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HADOOP-8515: --- Target Version/s: 1.2.0, 3.0.0 (was: 1.1.1, 3.0.0) Upgrade Jetty to the current Jetty 7 release Key: HADOOP-8515 URL: https://issues.apache.org/jira/browse/HADOOP-8515 Project: Hadoop Common Issue Type: Improvement Reporter: Luke Lu Labels: jetty According to http://dev.eclipse.org/mhonarc/lists/jetty-announce/msg00026.html, jetty-6 has been effectively EOL since January. Let's bump jetty to the 7 series. The current jetty 6.1.26 contains at least one known vulnerability: CVE-2011-4461. Note this can be an incompatible change if you reference jetty-6 packages (org.mortbay.*). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8515) Upgrade Jetty to the current Jetty 7 release
[ https://issues.apache.org/jira/browse/HADOOP-8515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501520#comment-13501520 ] Matt Foley commented on HADOOP-8515: moved target version to 1.2.0 upon publishing 1.1.1 RC. Upgrade Jetty to the current Jetty 7 release Key: HADOOP-8515 URL: https://issues.apache.org/jira/browse/HADOOP-8515 Project: Hadoop Common Issue Type: Improvement Reporter: Luke Lu Labels: jetty According to http://dev.eclipse.org/mhonarc/lists/jetty-announce/msg00026.html, jetty-6 has been effectively EOL since January. Let's bump jetty to the 7 series. The current jetty 6.1.26 contains at least one known vulnerability: CVE-2011-4461. Note this can be an incompatible change if you reference jetty-6 packages (org.mortbay.*). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9073) Migrate saveVersion.sh script to python
Radim Kolar created HADOOP-9073: --- Summary: Migrate saveVersion.sh script to python Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Environment: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Reporter: Radim Kolar -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9070) Kerberos SASL server cannot find kerberos key
[
https://issues.apache.org/jira/browse/HADOOP-9070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501545#comment-13501545
]
Hadoop QA commented on HADOOP-9070:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554412/HADOOP-9070.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1782//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1782//console
This message is automatically generated.
Kerberos SASL server cannot find kerberos key
-
Key: HADOOP-9070
URL: https://issues.apache.org/jira/browse/HADOOP-9070
Project: Hadoop Common
Issue Type: Sub-task
Components: ipc
Affects Versions: 3.0.0, 2.0.3-alpha
Reporter: Daryn Sharp
Assignee: Daryn Sharp
Priority: Blocker
Attachments: HADOOP-9070.patch, HADOOP-9070.patch
HADOOP-9015 inadvertently removed a {{doAs}} block around instantiation of
the sasl server which renders a server incapable of accepting kerberized
connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9073: Attachment: version-py.txt Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Environment: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Reporter: Radim Kolar Attachments: version-py.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9073: Status: Patch Available (was: Open) Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Environment: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Reporter: Radim Kolar Attachments: version-py.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9073: Target Version/s: 3.0.0, 2.0.3-alpha Affects Version/s: 3.0.0 Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Environment: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Reporter: Radim Kolar Attachments: version-py.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8958) ViewFs:Non absolute mount name failures when running multiple tests on Windows
[ https://issues.apache.org/jira/browse/HADOOP-8958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501553#comment-13501553 ] Chris Nauroth commented on HADOOP-8958: --- Yes, that looks like a typical mount point on Windows. I've been trying to limit the occurrences of if (WINDOWS) checks in the code. Generally, we've been able to achieve platform-agnostic test paths by using the Path methods instead of platform-specific string editing. I'll investigate rebasing this patch against the recent changes for HADOOP-8589. ViewFs:Non absolute mount name failures when running multiple tests on Windows -- Key: HADOOP-8958 URL: https://issues.apache.org/jira/browse/HADOOP-8958 Project: Hadoop Common Issue Type: Bug Components: viewfs Affects Versions: trunk-win Reporter: Chris Nauroth Assignee: Chris Nauroth Attachments: HADOOP-8958.patch This appears to be an issue with parsing a Windows-specific path. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9073) Migrate saveVersion.sh script to python
[
https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501558#comment-13501558
]
Hadoop QA commented on HADOOP-9073:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554417/version-py.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.yarn.util.TestYarnVersionInfo
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1783//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1783//console
This message is automatically generated.
Migrate saveVersion.sh script to python
---
Key: HADOOP-9073
URL: https://issues.apache.org/jira/browse/HADOOP-9073
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.0.0
Environment: Work based on HADOOP-8924. Changes build system to use
python script instead of sh script for generating hadoop package version.
Changes hadoop-common and yarn-common.
Reporter: Radim Kolar
Attachments: version-py.txt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8934) Shell command ls should include sort options
[
https://issues.apache.org/jira/browse/HADOOP-8934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501575#comment-13501575
]
Daryn Sharp commented on HADOOP-8934:
-
I'm sorry for delay. I lost track of this jira... Comments:
* Let's leave {{PathData}} alone to make the patch have more brevity and to
prevent inconsistency. Another followup jira could switch all the commands to
use accessors rather than fields.
* I think the sort order isn't quite right, at least for size it looks
backwards from the real ls.
* Default sort should use {{PathData#compareTo}}.
* If you haven't already, ensure the patch hasn't broken any of the
{{TestHDFSCLI}} tests. You probably should add a few tests of your own. If
nothing else it's easier to visualize if a command and output are correct.
* Small nits regarding code guidelines:
** please use spaces in conditionals, ex. if (blah) instead of if(blah)
** cuddle the else clauses
** try to stay within 80 char lines
Shell command ls should include sort options
Key: HADOOP-8934
URL: https://issues.apache.org/jira/browse/HADOOP-8934
Project: Hadoop Common
Issue Type: Improvement
Components: fs
Reporter: Jonathan Allen
Assignee: Jonathan Allen
Priority: Minor
Attachments: HADOOP-8934.patch, HADOOP-8934.patch
The shell command ls should include options to sort the output similar to the
unix ls command. The following options seem appropriate:
-t : sort by modification time
-S : sort by file size
-r : reverse the sort order
-u : use access time rather than modification time for sort and display
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9073: Attachment: version-py2.txt Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Environment: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Reporter: Radim Kolar Attachments: version-py2.txt, version-py.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9073) Migrate saveVersion.sh script to python
[
https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501600#comment-13501600
]
Hadoop QA commented on HADOOP-9073:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554424/version-py2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common:
org.apache.hadoop.yarn.util.TestYarnVersionInfo
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1784//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1784//console
This message is automatically generated.
Migrate saveVersion.sh script to python
---
Key: HADOOP-9073
URL: https://issues.apache.org/jira/browse/HADOOP-9073
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.0.0
Environment: Work based on HADOOP-8924. Changes build system to use
python script instead of sh script for generating hadoop package version.
Changes hadoop-common and yarn-common.
Reporter: Radim Kolar
Attachments: version-py2.txt, version-py.txt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9073: Attachment: version-py3.txt Need to extract branch information. Added support for both SVN and GIT Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Environment: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Reporter: Radim Kolar Attachments: version-py2.txt, version-py3.txt, version-py.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9073) Migrate saveVersion.sh script to python
[ https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Radim Kolar updated HADOOP-9073: Description: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. Environment: (was: Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common.) Migrate saveVersion.sh script to python --- Key: HADOOP-9073 URL: https://issues.apache.org/jira/browse/HADOOP-9073 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0 Reporter: Radim Kolar Attachments: version-py2.txt, version-py3.txt, version-py.txt Work based on HADOOP-8924. Changes build system to use python script instead of sh script for generating hadoop package version. Changes hadoop-common and yarn-common. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9073) Migrate saveVersion.sh script to python
[
https://issues.apache.org/jira/browse/HADOOP-9073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501632#comment-13501632
]
Hadoop QA commented on HADOOP-9073:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12554428/version-py3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1785//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/1785//console
This message is automatically generated.
Migrate saveVersion.sh script to python
---
Key: HADOOP-9073
URL: https://issues.apache.org/jira/browse/HADOOP-9073
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 3.0.0
Reporter: Radim Kolar
Attachments: version-py2.txt, version-py3.txt, version-py.txt
Work based on HADOOP-8924. Changes build system to use python script instead
of sh script for generating hadoop package version. Changes hadoop-common and
yarn-common.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9074) Hadoop install scripts for Windows
Ivan Mitic created HADOOP-9074: -- Summary: Hadoop install scripts for Windows Key: HADOOP-9074 URL: https://issues.apache.org/jira/browse/HADOOP-9074 Project: Hadoop Common Issue Type: Bug Affects Versions: 1-win Reporter: Ivan Mitic Assignee: Ivan Mitic Tracking Jira to post Hadoop install scripts for Windows. Scripts will provide means for Windows users/developers to install/uninstall Hadoop on a single-node. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9066) Sorting for FileStatus[]
[ https://issues.apache.org/jira/browse/HADOOP-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501707#comment-13501707 ] david king commented on HADOOP-9066: Hey Harsh J: How did mainipulate this batch ? Did it have any problems? Sorting for FileStatus[] Key: HADOOP-9066 URL: https://issues.apache.org/jira/browse/HADOOP-9066 Project: Hadoop Common Issue Type: Improvement Environment: java7 , RedHat9 , Hadoop 0.20.2 ,eclipse-jee-juno-linux-gtk.tar.gz Reporter: david king Labels: patch Attachments: ConcreteFileStatusAscComparable.java, ConcreteFileStatusDescComparable.java, FileStatusComparable.java, FileStatusTool.java, TestFileStatusTool.java I will submit a batch of FileStatusTool that used to sort FileStatus by the Comparator, the Comparator not only customer to realizate , but alse use the example code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Moved] (HADOOP-9075) FileContext#FSLinkResolver should be made static
[ https://issues.apache.org/jira/browse/HADOOP-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas moved HDFS-4218 to HADOOP-9075: --- Fix Version/s: (was: 2.0.0-alpha) Affects Version/s: (was: 2.0.0-alpha) 2.0.0-alpha Key: HADOOP-9075 (was: HDFS-4218) Project: Hadoop Common (was: Hadoop HDFS) FileContext#FSLinkResolver should be made static Key: HADOOP-9075 URL: https://issues.apache.org/jira/browse/HADOOP-9075 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Attachments: HDFS-4218.patch FSLinkResolver class does not require access to Outer Class and hence should be made static. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9075) FileContext#FSLinkResolver should be made static
[ https://issues.apache.org/jira/browse/HADOOP-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9075: Resolution: Fixed Fix Version/s: 3.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1. I committed the patch to trunk Thank you Arpit. FileContext#FSLinkResolver should be made static Key: HADOOP-9075 URL: https://issues.apache.org/jira/browse/HADOOP-9075 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Fix For: 3.0.0 Attachments: HDFS-4218.patch FSLinkResolver class does not require access to Outer Class and hence should be made static. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9075) FileContext#FSLinkResolver should be made static
[ https://issues.apache.org/jira/browse/HADOOP-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13501718#comment-13501718 ] Hudson commented on HADOOP-9075: Integrated in Hadoop-trunk-Commit #3052 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3052/]) HADOOP-9075. FileContext#FSLinkResolver should be made static. Contributed by Arpit Agarwal. (Revision 1411988) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1411988 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java FileContext#FSLinkResolver should be made static Key: HADOOP-9075 URL: https://issues.apache.org/jira/browse/HADOOP-9075 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Suresh Srinivas Assignee: Arpit Agarwal Priority: Minor Fix For: 3.0.0 Attachments: HDFS-4218.patch FSLinkResolver class does not require access to Outer Class and hence should be made static. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9074) Hadoop install scripts for Windows
[
https://issues.apache.org/jira/browse/HADOOP-9074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Mitic updated HADOOP-9074:
---
Attachment: HADOOP-9074.branch-1-win.installer.patch
Attaching the patch. Patch contributors are Steve Maine, Ramya Nimmagadda,
Chuan Liu, Brad Sarsfield and Ivan Mitic.
To compile the windows package run:
{{ant winpkg}}
This will result in a {{hadoop-1.1.0-SNAPSHOT.winpkg.zip}} target which
contains all binaries/scripts needed for install.
To install Hadoop on the box, unpack the zip and run:
{{hadoop-1.1.0-SNAPSHOT.winpkg\scripts\install.ps1 -username username
-password password}}
where username/password represent an account in which context the services
will be running in. Please refer to install.ps1 for additional details on how
to setup this account.
Hadoop services will run as Windows services meaning that they could be
controlled through the Service Control Manager (services.msc). Alternatively,
one could use %HADOOP_HOME%\bin\ start_daemons.cmd/stop_daemons.cmd to start
and stop the services.
Hadoop install scripts for Windows
--
Key: HADOOP-9074
URL: https://issues.apache.org/jira/browse/HADOOP-9074
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 1-win
Reporter: Ivan Mitic
Assignee: Ivan Mitic
Attachments: HADOOP-9074.branch-1-win.installer.patch
Tracking Jira to post Hadoop install scripts for Windows. Scripts will
provide means for Windows users/developers to install/uninstall Hadoop on a
single-node.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
