[jira] [Commented] (HADOOP-7101) UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context
[
https://issues.apache.org/jira/browse/HADOOP-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598632#comment-13598632
]
Torsten Mielke commented on HADOOP-7101:
Filling in for Claus Ibsen here as well:
{quote}Can you tell me the branch name and commit revision for this?{quote}
The latest 0.23.5 release has this fix included.
However we did not find any 1.x release that contains this code fix. See my
comment from 01/Feb/13.
I checked the source code of all relevant 1.x releases and did not see the fix.
The fix is present in 2.0 branch however there has not been any release of 2.0
yet.
So trying to use the Hadoop client libs of any 1.x version when running inside
another container (e.g. OSGi) fails.
UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS
context
Key: HADOOP-7101
URL: https://issues.apache.org/jira/browse/HADOOP-7101
Project: Hadoop Common
Issue Type: Bug
Components: security
Affects Versions: 0.22.0
Reporter: Todd Lipcon
Assignee: Todd Lipcon
Priority: Blocker
Fix For: 0.22.0
Attachments: hadoop-7101.txt
If a Hadoop client is run from inside a container like Tomcat, and the
current AccessControlContext has a Subject associated with it that is not
created by Hadoop, then UserGroupInformation.getCurrentUser() will throw
NoSuchElementException, since it assumes that any Subject will have a hadoop
User principal.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8973) DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs
[ https://issues.apache.org/jira/browse/HADOOP-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598704#comment-13598704 ] Hudson commented on HADOOP-8973: Integrated in Hadoop-Yarn-trunk #152 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/152/]) HADOOP-8973. DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs. Contributed by Chris Nauroth. (Revision 1454889) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1454889 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DiskChecker.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestDiskChecker.java DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs - Key: HADOOP-8973 URL: https://issues.apache.org/jira/browse/HADOOP-8973 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 3.0.0, 1-win Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 1-win Attachments: DiskChecker.proto.patch, HADOOP-8973.3.patch, HADOOP-8973-branch-1-win.3.patch, HADOOP-8973-branch-trunk-win.2.patch, HADOOP-8973-branch-trunk-win.patch DiskChecker.checkDir uses File.canRead, File.canWrite, and File.canExecute to check if a directory is inaccessible. These APIs are not reliable on Windows with NTFS ACLs due to a known JVM bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9326) maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-common: There a test failures.
[ https://issues.apache.org/jira/browse/HADOOP-9326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598708#comment-13598708 ] JLASSI Aymen commented on HADOOP-9326: -- Hi, Suresh Srinivas, I will transfert my question to the adequate placement, can you tell me where can i ask it? Chris Nauroth, Yes i run as root, i don't now how offert root-right to other user-compte on mac-OS (account and right's administrator have also some restrictions and limit disk access), that's why i'm connecting as a root. I try with other user account and it's failure?? If you have other suggestions? thanks for all for your help. maven-surefire-plugin:2.12.3:test (default-test) on project hadoop-common: There a test failures. - Key: HADOOP-9326 URL: https://issues.apache.org/jira/browse/HADOOP-9326 Project: Hadoop Common Issue Type: Bug Components: build, test Environment: For information, i take hadoop with GIT and i run it on mac OS Reporter: JLASSI Aymen Original Estimate: 336h Remaining Estimate: 336h I'd like to compile hadoop from source code, and when i launch test-step, i have the desciption as follows, when i skip the test-step to the package step, i have the same problem, the same description of bug: Results : Failed tests: testFailFullyDelete(org.apache.hadoop.fs.TestFileUtil): The directory xSubDir *should* not have been deleted. expected:true but was:false testFailFullyDeleteContents(org.apache.hadoop.fs.TestFileUtil): The directory xSubDir *should* not have been deleted. expected:true but was:false testListStatusThrowsExceptionForUnreadableDir(org.apache.hadoop.fs.TestFSMainOperationsLocalFileSystem): Should throw IOException test0[0](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for build/test/temp/RELATIVE1 in build/test/temp/RELATIVE0/block4197707426846287299.tmp - FAILED! testROBufferDirAndRWBufferDir[0](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for build/test/temp/RELATIVE2 in build/test/temp/RELATIVE1/block138767728739012230.tmp - FAILED! testRWBufferDirBecomesRO[0](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for build/test/temp/RELATIVE3 in build/test/temp/RELATIVE4/block4888615109050601773.tmp - FAILED! test0[1](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/ABSOLUTE1 in /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/ABSOLUTE0/block4663369813226761504.tmp - FAILED! testROBufferDirAndRWBufferDir[1](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/ABSOLUTE2 in /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/ABSOLUTE1/block2846944239985650460.tmp - FAILED! testRWBufferDirBecomesRO[1](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/ABSOLUTE3 in /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/ABSOLUTE4/block4367331619344952181.tmp - FAILED! test0[2](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for file:/Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/QUALIFIED1 in /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/QUALIFIED0/block5687619346377173125.tmp - FAILED! testROBufferDirAndRWBufferDir[2](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for file:/Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/QUALIFIED2 in /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/QUALIFIED1/block2235209534902942511.tmp - FAILED! testRWBufferDirBecomesRO[2](org.apache.hadoop.fs.TestLocalDirAllocator): Checking for file:/Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/QUALIFIED3 in /Users/aymenjlassi/Desktop/hadoop_source/releaseGit/hadoop-common/hadoop-common-project/hadoop-common/build/test/temp/QUALIFIED4/block6994640486900109274.tmp - FAILED! testReportChecksumFailure(org.apache.hadoop.fs.TestLocalFileSystem)
[jira] [Updated] (HADOOP-9063) enhance unit-test coverage of class org.apache.hadoop.fs.FileUtil
[ https://issues.apache.org/jira/browse/HADOOP-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Y updated HADOOP-9063: - Attachment: HADOOP-9063-trunk--N2.patch updated patch for trunk enhance unit-test coverage of class org.apache.hadoop.fs.FileUtil - Key: HADOOP-9063 URL: https://issues.apache.org/jira/browse/HADOOP-9063 Project: Hadoop Common Issue Type: Test Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-9063--b.patch, HADOOP-9063-branch-0.23--b.patch, HADOOP-9063-branch-0.23--c.patch, HADOOP-9063.patch, HADOOP-9063-trunk--c.patch, HADOOP-9063-trunk--c.patch, HADOOP-9063-trunk--N2.patch Some methods of class org.apache.hadoop.fs.FileUtil are covered by unit-tests poorly or not covered at all. Enhance the coverage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9166) Cover authentication with Kerberos ticket cache with unit tests
[ https://issues.apache.org/jira/browse/HADOOP-9166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Y updated HADOOP-9166: - Attachment: HADOOP-9166-trunk--N2.patch HADOOP-9166-branch-2--N1.patch HADOOP-9166-branch-0.23--N1.patch updated patches for trunk, branch-2, branch-0.23 Cover authentication with Kerberos ticket cache with unit tests Key: HADOOP-9166 URL: https://issues.apache.org/jira/browse/HADOOP-9166 Project: Hadoop Common Issue Type: Test Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Attachments: HADOOP-9166-branch-0.23--b.patch, HADOOP-9166-branch-0.23--c.patch, HADOOP-9166-branch-0.23--N1.patch, HADOOP-9166-branch-2--b.patch, HADOOP-9166-branch-2--c.patch, HADOOP-9166-branch-2--N1.patch, HADOOP-9166-trunk--b.patch, HADOOP-9166-trunk--c.patch, HADOOP-9166-trunk--N2.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9392) Token based authentication and Single Sign On
Kai Zheng created HADOOP-9392: - Summary: Token based authentication and Single Sign On Key: HADOOP-9392 URL: https://issues.apache.org/jira/browse/HADOOP-9392 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Kai Zheng Fix For: 3.0.0 This is an umbrella entry for one of project Rhino’s topic, for details of project Rhino, please refer to https://github.com/intel-hadoop/project-rhino/. The major goal for this entry as described in project Rhino was “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at the RPC layer, via SASL. However this does not provide valuable attributes such as group membership, classification level, organizational identity, or support for user defined attributes. Hadoop components must interrogate external resources for discovering these attributes and at scale this is problematic. There is also no consistent delegation model. HDFS has a simple delegation capability, and only Oozie can take limited advantage of it. We will implement a common token based authentication framework to decouple internal user and service authentication from external mechanisms used to support it (like Kerberos)” We’d like to start our work from Hadoop-Common and try to provide common facilities by extending existing authentication framework which support: 1. Pluggable token provider interface 2. Pluggable token verification protocol and interface 3. Security mechanism to distribute secrets in cluster nodes 4. Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9392) Token based authentication and Single Sign On
[ https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HADOOP-9392: -- Tags: Project Rhino Token based authentication and Single Sign On - Key: HADOOP-9392 URL: https://issues.apache.org/jira/browse/HADOOP-9392 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Kai Zheng Fix For: 3.0.0 This is an umbrella entry for one of project Rhino’s topic, for details of project Rhino, please refer to https://github.com/intel-hadoop/project-rhino/. The major goal for this entry as described in project Rhino was “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at the RPC layer, via SASL. However this does not provide valuable attributes such as group membership, classification level, organizational identity, or support for user defined attributes. Hadoop components must interrogate external resources for discovering these attributes and at scale this is problematic. There is also no consistent delegation model. HDFS has a simple delegation capability, and only Oozie can take limited advantage of it. We will implement a common token based authentication framework to decouple internal user and service authentication from external mechanisms used to support it (like Kerberos)” We’d like to start our work from Hadoop-Common and try to provide common facilities by extending existing authentication framework which support: 1.Pluggable token provider interface 2.Pluggable token verification protocol and interface 3.Security mechanism to distribute secrets in cluster nodes 4.Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9063) enhance unit-test coverage of class org.apache.hadoop.fs.FileUtil
[
https://issues.apache.org/jira/browse/HADOOP-9063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598762#comment-13598762
]
Hadoop QA commented on HADOOP-9063:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573071/HADOOP-9063-trunk--N2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2306//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2306//console
This message is automatically generated.
enhance unit-test coverage of class org.apache.hadoop.fs.FileUtil
-
Key: HADOOP-9063
URL: https://issues.apache.org/jira/browse/HADOOP-9063
Project: Hadoop Common
Issue Type: Test
Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6
Reporter: Ivan A. Veselovsky
Assignee: Ivan A. Veselovsky
Priority: Minor
Attachments: HADOOP-9063--b.patch, HADOOP-9063-branch-0.23--b.patch,
HADOOP-9063-branch-0.23--c.patch, HADOOP-9063.patch,
HADOOP-9063-trunk--c.patch, HADOOP-9063-trunk--c.patch,
HADOOP-9063-trunk--N2.patch
Some methods of class org.apache.hadoop.fs.FileUtil are covered by unit-tests
poorly or not covered at all. Enhance the coverage.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9166) Cover authentication with Kerberos ticket cache with unit tests
[
https://issues.apache.org/jira/browse/HADOOP-9166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598766#comment-13598766
]
Hadoop QA commented on HADOOP-9166:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573075/HADOOP-9166-trunk--N2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2305//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2305//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2305//console
This message is automatically generated.
Cover authentication with Kerberos ticket cache with unit tests
Key: HADOOP-9166
URL: https://issues.apache.org/jira/browse/HADOOP-9166
Project: Hadoop Common
Issue Type: Test
Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6
Reporter: Ivan A. Veselovsky
Assignee: Ivan A. Veselovsky
Attachments: HADOOP-9166-branch-0.23--b.patch,
HADOOP-9166-branch-0.23--c.patch, HADOOP-9166-branch-0.23--N1.patch,
HADOOP-9166-branch-2--b.patch, HADOOP-9166-branch-2--c.patch,
HADOOP-9166-branch-2--N1.patch, HADOOP-9166-trunk--b.patch,
HADOOP-9166-trunk--c.patch, HADOOP-9166-trunk--N2.patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8973) DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs
[ https://issues.apache.org/jira/browse/HADOOP-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598783#comment-13598783 ] Hudson commented on HADOOP-8973: Integrated in Hadoop-Hdfs-trunk #1341 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1341/]) HADOOP-8973. DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs. Contributed by Chris Nauroth. (Revision 1454889) Result = FAILURE suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1454889 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DiskChecker.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestDiskChecker.java DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs - Key: HADOOP-8973 URL: https://issues.apache.org/jira/browse/HADOOP-8973 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 3.0.0, 1-win Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 1-win Attachments: DiskChecker.proto.patch, HADOOP-8973.3.patch, HADOOP-8973-branch-1-win.3.patch, HADOOP-8973-branch-trunk-win.2.patch, HADOOP-8973-branch-trunk-win.patch DiskChecker.checkDir uses File.canRead, File.canWrite, and File.canExecute to check if a directory is inaccessible. These APIs are not reliable on Windows with NTFS ACLs due to a known JVM bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8973) DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs
[ https://issues.apache.org/jira/browse/HADOOP-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598831#comment-13598831 ] Hudson commented on HADOOP-8973: Integrated in Hadoop-Mapreduce-trunk #1369 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1369/]) HADOOP-8973. DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs. Contributed by Chris Nauroth. (Revision 1454889) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1454889 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DiskChecker.java * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestDiskChecker.java DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs - Key: HADOOP-8973 URL: https://issues.apache.org/jira/browse/HADOOP-8973 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 3.0.0, 1-win Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 1-win Attachments: DiskChecker.proto.patch, HADOOP-8973.3.patch, HADOOP-8973-branch-1-win.3.patch, HADOOP-8973-branch-trunk-win.2.patch, HADOOP-8973-branch-trunk-win.patch DiskChecker.checkDir uses File.canRead, File.canWrite, and File.canExecute to check if a directory is inaccessible. These APIs are not reliable on Windows with NTFS ACLs due to a known JVM bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9166) Cover authentication with Kerberos ticket cache with unit tests
[ https://issues.apache.org/jira/browse/HADOOP-9166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Y updated HADOOP-9166: - Attachment: HADOOP-9166-trunk--N4(1).patch updated trunk patch with license header timeouts Cover authentication with Kerberos ticket cache with unit tests Key: HADOOP-9166 URL: https://issues.apache.org/jira/browse/HADOOP-9166 Project: Hadoop Common Issue Type: Test Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Attachments: HADOOP-9166-branch-0.23--b.patch, HADOOP-9166-branch-0.23--c.patch, HADOOP-9166-branch-0.23--N1.patch, HADOOP-9166-branch-2--b.patch, HADOOP-9166-branch-2--c.patch, HADOOP-9166-branch-2--N1.patch, HADOOP-9166-trunk--b.patch, HADOOP-9166-trunk--c.patch, HADOOP-9166-trunk--N2.patch, HADOOP-9166-trunk--N4(1).patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9258) Add stricter tests to FileSystemContractTestBase
[ https://issues.apache.org/jira/browse/HADOOP-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9258: --- Status: Open (was: Patch Available) Add stricter tests to FileSystemContractTestBase Key: HADOOP-9258 URL: https://issues.apache.org/jira/browse/HADOOP-9258 Project: Hadoop Common Issue Type: Sub-task Components: test Affects Versions: 2.0.3-alpha, 1.1.1 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-9528-2.patch, HADOOP-9528-3.patch, HADOOP-9528-4.patch, HADOOP-9528-5.patch, HADOOP-9528-6.patch, HADOOP-9528-7.patch, HADOOP-9528.patch The File System Contract contains implicit assumptions that aren't checked in the contract test base. Add more tests to define the contract's assumptions more rigorously for those filesystems that are tested by this (not Local, BTW) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9258) Add stricter tests to FileSystemContractTestBase
[
https://issues.apache.org/jira/browse/HADOOP-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-9258:
---
Attachment: HADOOP-9528-7.patch
# this patch incorporates HADOOP-9265
# ignores the return value of {{rename(self,self)}} when
{{self.isDirectory()}}, as HDFS returns success where posix returns false
Add stricter tests to FileSystemContractTestBase
Key: HADOOP-9258
URL: https://issues.apache.org/jira/browse/HADOOP-9258
Project: Hadoop Common
Issue Type: Sub-task
Components: test
Affects Versions: 1.1.1, 2.0.3-alpha
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HADOOP-9528-2.patch, HADOOP-9528-3.patch,
HADOOP-9528-4.patch, HADOOP-9528-5.patch, HADOOP-9528-6.patch,
HADOOP-9528-7.patch, HADOOP-9528.patch
The File System Contract contains implicit assumptions that aren't checked in
the contract test base. Add more tests to define the contract's assumptions
more rigorously for those filesystems that are tested by this (not Local, BTW)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9258) Add stricter tests to FileSystemContractTestBase
[ https://issues.apache.org/jira/browse/HADOOP-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9258: --- Status: Patch Available (was: Open) No @Test timeout attributes as this is a JUnit3 test case. Add stricter tests to FileSystemContractTestBase Key: HADOOP-9258 URL: https://issues.apache.org/jira/browse/HADOOP-9258 Project: Hadoop Common Issue Type: Sub-task Components: test Affects Versions: 2.0.3-alpha, 1.1.1 Reporter: Steve Loughran Assignee: Steve Loughran Attachments: HADOOP-9528-2.patch, HADOOP-9528-3.patch, HADOOP-9528-4.patch, HADOOP-9528-5.patch, HADOOP-9528-6.patch, HADOOP-9528-7.patch, HADOOP-9528.patch The File System Contract contains implicit assumptions that aren't checked in the contract test base. Add more tests to define the contract's assumptions more rigorously for those filesystems that are tested by this (not Local, BTW) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9287) Parallel testing hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Klochkov updated HADOOP-9287: Attachment: HADOOP-9287--N3.patch The patch is updated with a few additional fixes. Parallel testing hadoop-common -- Key: HADOOP-9287 URL: https://issues.apache.org/jira/browse/HADOOP-9287 Project: Hadoop Common Issue Type: Bug Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: HADOOP-9287.1.patch, HADOOP-9287--N3.patch, HADOOP-9287.patch, HADOOP-9287.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9258) Add stricter tests to FileSystemContractTestBase
[
https://issues.apache.org/jira/browse/HADOOP-9258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13598937#comment-13598937
]
Hadoop QA commented on HADOOP-9258:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573102/HADOOP-9528-7.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2308//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2308//console
This message is automatically generated.
Add stricter tests to FileSystemContractTestBase
Key: HADOOP-9258
URL: https://issues.apache.org/jira/browse/HADOOP-9258
Project: Hadoop Common
Issue Type: Sub-task
Components: test
Affects Versions: 1.1.1, 2.0.3-alpha
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HADOOP-9528-2.patch, HADOOP-9528-3.patch,
HADOOP-9528-4.patch, HADOOP-9528-5.patch, HADOOP-9528-6.patch,
HADOOP-9528-7.patch, HADOOP-9528.patch
The File System Contract contains implicit assumptions that aren't checked in
the contract test base. Add more tests to define the contract's assumptions
more rigorously for those filesystems that are tested by this (not Local, BTW)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9043) winutils can create unusable symlinks
[
https://issues.apache.org/jira/browse/HADOOP-9043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599006#comment-13599006
]
Chris Nauroth commented on HADOOP-9043:
---
Tightening up the validation is great, but we still have the problem that
{{TestLocalFSFileContextSymlink}} passes on Linux but fails on Windows. The
current state of the discussion here is that winutils will enforce validation
rules to reject creation of invalid/dangling symlinks. Since this diverges
intentionally from the Linux behavior, do we want to update
{{TestLocalFSFileContextSymlink}} to add {{assumeTrue(!Shell.WINDOWS)}} on the
invalid tests?
winutils can create unusable symlinks
-
Key: HADOOP-9043
URL: https://issues.apache.org/jira/browse/HADOOP-9043
Project: Hadoop Common
Issue Type: Bug
Components: util
Affects Versions: 3.0.0, 1-win
Reporter: Chris Nauroth
Assignee: Arpit Agarwal
Fix For: 3.0.0, 1-win
Attachments: HADOOP-9043.branch-1.2.patch,
HADOOP-9043.branch-1-win.patch, HADOOP-9043.trunk.2.patch,
HADOOP-9043.trunk.patch
In general, the winutils symlink command rejects attempts to create symlinks
targeting a destination file that does not exist. However, if given a
symlink destination with forward slashes pointing at a file that does exist,
then it creates the symlink with the forward slashes, and then attempts to
open the file through the symlink will fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9390) Tests fail when run as root/Administrator.
[
https://issues.apache.org/jira/browse/HADOOP-9390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599014#comment-13599014
]
Alejandro Abdelnur commented on HADOOP-9390:
+1 on won't fix.
Tests fail when run as root/Administrator.
--
Key: HADOOP-9390
URL: https://issues.apache.org/jira/browse/HADOOP-9390
Project: Hadoop Common
Issue Type: Bug
Components: test
Reporter: Chris Nauroth
Fix For: 3.0.0
There is at least one test, {{TestDiskChecker}}, that fails when running as
root on Linux or Administrator on Windows. The test assumes that setting
file permissions can make a file inaccessible, without considering the
possibility that root can access everything.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-9389) test-patch marks -1 due to a context @Test by mistake
[
https://issues.apache.org/jira/browse/HADOOP-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhijie Shen reassigned HADOOP-9389:
---
Assignee: Zhijie Shen
test-patch marks -1 due to a context @Test by mistake
-
Key: HADOOP-9389
URL: https://issues.apache.org/jira/browse/HADOOP-9389
Project: Hadoop Common
Issue Type: Bug
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: HADOOP-9389_1.patch
HADOOP-9112 enables the function of marking -1 when the newly added tests
don't have timeout. However, test-patch will mark -1 due to a context @Test
by mistake. Bellow is the problematic part of the YARN-378_3.patch that I've
created.
{code}
+}
+ }
+
@Test
public void testRMAppSubmitWithQueueAndName() throws Exception {
long now = System.currentTimeMillis();
{code}
There's a @Test without timeout (most existing tests don't have timeout) in
the context. In test-patch, $AWK '\{ printf %s , $0 \}' collapses these
lines into one line, i.e.,
{code}
+} + } +@Testpublic void testRMAppSubmitWithQueueAndName()
throws Exception { long now = System.currentTimeMillis();
{code}
Then, @Test in the context follows a +, and is regarded as a newly added
test by mistake. Consequently, the following regex will accept the context
@Test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9287) Parallel testing hadoop-common
[
https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599073#comment-13599073
]
Hadoop QA commented on HADOOP-9287:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573108/HADOOP-9287--N3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 52 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs
hadoop-tools/hadoop-distcp
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
org.apache.hadoop.hdfs.server.datanode.TestDataDirs
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2309//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2309//console
This message is automatically generated.
Parallel testing hadoop-common
--
Key: HADOOP-9287
URL: https://issues.apache.org/jira/browse/HADOOP-9287
Project: Hadoop Common
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Tsuyoshi OZAWA
Assignee: Andrey Klochkov
Attachments: HADOOP-9287.1.patch, HADOOP-9287--N3.patch,
HADOOP-9287.patch, HADOOP-9287.patch
The maven surefire plugin supports parallel testing feature. By using it, the
tests can be run more faster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9389) test-patch marks -1 due to a context @Test by mistake
[
https://issues.apache.org/jira/browse/HADOOP-9389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599075#comment-13599075
]
Zhijie Shen commented on HADOOP-9389:
-
I agree it's good to cleanup the existing tests without timeout, but my concern
is that this issue is to mislead people to thinking there're missed timeouts in
their newly added tests. Hence people should be aware of this issue.
test-patch marks -1 due to a context @Test by mistake
-
Key: HADOOP-9389
URL: https://issues.apache.org/jira/browse/HADOOP-9389
Project: Hadoop Common
Issue Type: Bug
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: HADOOP-9389_1.patch
HADOOP-9112 enables the function of marking -1 when the newly added tests
don't have timeout. However, test-patch will mark -1 due to a context @Test
by mistake. Bellow is the problematic part of the YARN-378_3.patch that I've
created.
{code}
+}
+ }
+
@Test
public void testRMAppSubmitWithQueueAndName() throws Exception {
long now = System.currentTimeMillis();
{code}
There's a @Test without timeout (most existing tests don't have timeout) in
the context. In test-patch, $AWK '\{ printf %s , $0 \}' collapses these
lines into one line, i.e.,
{code}
+} + } +@Testpublic void testRMAppSubmitWithQueueAndName()
throws Exception { long now = System.currentTimeMillis();
{code}
Then, @Test in the context follows a +, and is regarded as a newly added
test by mistake. Consequently, the following regex will accept the context
@Test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9330) Add custom JUnit4 test runner with configurable timeout
[ https://issues.apache.org/jira/browse/HADOOP-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9330: --- Attachment: HADOOP-9330-timeouts-1.patch first cut: test class (without any tests underneath to verify that it works) Add custom JUnit4 test runner with configurable timeout --- Key: HADOOP-9330 URL: https://issues.apache.org/jira/browse/HADOOP-9330 Project: Hadoop Common Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Steve Loughran Attachments: HADOOP-9330-timeouts-1.patch HADOOP-9112 has added a requirement for all new test methods to declare a timeout, so that jenkins/maven builds will have better information on a timeout. Hard coding timeouts into tests is dangerous as it will generate spurious failures on slower machines/networks and when debugging a test. I propose providing a custom JUnit4 test runner that test cases can declare as their test runner; this can provide timeouts specified at run-time, rather than in-source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9330) Add custom JUnit4 test runner with configurable timeout
[ https://issues.apache.org/jira/browse/HADOOP-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-9330: --- Status: Patch Available (was: Open) Add custom JUnit4 test runner with configurable timeout --- Key: HADOOP-9330 URL: https://issues.apache.org/jira/browse/HADOOP-9330 Project: Hadoop Common Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Steve Loughran Attachments: HADOOP-9330-timeouts-1.patch HADOOP-9112 has added a requirement for all new test methods to declare a timeout, so that jenkins/maven builds will have better information on a timeout. Hard coding timeouts into tests is dangerous as it will generate spurious failures on slower machines/networks and when debugging a test. I propose providing a custom JUnit4 test runner that test cases can declare as their test runner; this can provide timeouts specified at run-time, rather than in-source. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9046) provide unit-test coverage of class org.apache.hadoop.fs.DelegationTokenRenewer.RenewActionT
[ https://issues.apache.org/jira/browse/HADOOP-9046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan A. Veselovsky updated HADOOP-9046: --- Attachment: HADOOP-9046-trunk--over-HDFS-4567.patch HADOOP-9046-branch-0.23--over-HDFS-4567.patch Attaching new versions of patches merged over fix HDFS-4567. provide unit-test coverage of class org.apache.hadoop.fs.DelegationTokenRenewer.RenewActionT -- Key: HADOOP-9046 URL: https://issues.apache.org/jira/browse/HADOOP-9046 Project: Hadoop Common Issue Type: Test Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6 Reporter: Ivan A. Veselovsky Assignee: Ivan A. Veselovsky Priority: Minor Attachments: HADOOP-9046-branch-0.23--c.patch, HADOOP-9046-branch-0.23--d.patch, HADOOP-9046-branch-0.23-over-9049.patch, HADOOP-9046-branch-0.23--over-HDFS-4567.patch, HADOOP-9046-branch-0.23.patch, HADOOP-9046--c.patch, HADOOP-9046--d.patch, HADOOP-9046--e.patch, HADOOP-9046-over-9049.patch, HADOOP-9046.patch, HADOOP-9046-trunk--over-HDFS-4567.patch The class org.apache.hadoop.fs.DelegationTokenRenewer.RenewActionT has zero coverage in entire cumulative test run. Provide test(s) to cover this class. Note: the request submitted to HDFS project because the class likely to be tested by tests in that project. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9393) TestRPC fails with JDK7
Andrew Wang created HADOOP-9393:
---
Summary: TestRPC fails with JDK7
Key: HADOOP-9393
URL: https://issues.apache.org/jira/browse/HADOOP-9393
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Since the test order is different with JDK7, we hit an error in
{{TestRPC#testStopsAllThreads}} on the initial assertEquals. This is because
{{testRPCInterruptedSimple}} and {{testRPCInterrupted}} don't stop the server
after they finish, leaving some threads around.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
Andrew Wang created HADOOP-9394:
---
Summary: Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9330) Add custom JUnit4 test runner with configurable timeout
[
https://issues.apache.org/jira/browse/HADOOP-9330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599191#comment-13599191
]
Hadoop QA commented on HADOOP-9330:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573149/HADOOP-9330-timeouts-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2310//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2310//console
This message is automatically generated.
Add custom JUnit4 test runner with configurable timeout
---
Key: HADOOP-9330
URL: https://issues.apache.org/jira/browse/HADOOP-9330
Project: Hadoop Common
Issue Type: Test
Components: test
Affects Versions: 3.0.0
Reporter: Steve Loughran
Attachments: HADOOP-9330-timeouts-1.patch
HADOOP-9112 has added a requirement for all new test methods to declare a
timeout, so that jenkins/maven builds will have better information on a
timeout.
Hard coding timeouts into tests is dangerous as it will generate spurious
failures on slower machines/networks and when debugging a test.
I propose providing a custom JUnit4 test runner that test cases can declare
as their test runner; this can provide timeouts specified at run-time, rather
than in-source.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9046) provide unit-test coverage of class org.apache.hadoop.fs.DelegationTokenRenewer.RenewActionT
[
https://issues.apache.org/jira/browse/HADOOP-9046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599211#comment-13599211
]
Hadoop QA commented on HADOOP-9046:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573154/HADOOP-9046-trunk--over-HDFS-4567.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2311//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2311//console
This message is automatically generated.
provide unit-test coverage of class
org.apache.hadoop.fs.DelegationTokenRenewer.RenewActionT
--
Key: HADOOP-9046
URL: https://issues.apache.org/jira/browse/HADOOP-9046
Project: Hadoop Common
Issue Type: Test
Affects Versions: 3.0.0, 2.0.3-alpha, 0.23.6
Reporter: Ivan A. Veselovsky
Assignee: Ivan A. Veselovsky
Priority: Minor
Attachments: HADOOP-9046-branch-0.23--c.patch,
HADOOP-9046-branch-0.23--d.patch, HADOOP-9046-branch-0.23-over-9049.patch,
HADOOP-9046-branch-0.23--over-HDFS-4567.patch, HADOOP-9046-branch-0.23.patch,
HADOOP-9046--c.patch, HADOOP-9046--d.patch, HADOOP-9046--e.patch,
HADOOP-9046-over-9049.patch, HADOOP-9046.patch,
HADOOP-9046-trunk--over-HDFS-4567.patch
The class org.apache.hadoop.fs.DelegationTokenRenewer.RenewActionT has zero
coverage in entire cumulative test run. Provide test(s) to cover this class.
Note: the request submitted to HDFS project because the class likely to be
tested by tests in that project.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HADOOP-9395) jetty7 and hadoop 1.1.x
Alexey Babutin created HADOOP-9395: -- Summary: jetty7 and hadoop 1.1.x Key: HADOOP-9395 URL: https://issues.apache.org/jira/browse/HADOOP-9395 Project: Hadoop Common Issue Type: Improvement Components: build, net Affects Versions: 1.1.2 Environment: FreeBSD 9.1+ZFS.2 xeon 5640,12Gb regDDR3,4 HDD 2Tb. Reporter: Alexey Babutin Priority: Minor Fix For: 1.2.0, 1.1.3 Change jetty6 to jetty7 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9395) jetty7 and hadoop 1.1.x
[ https://issues.apache.org/jira/browse/HADOOP-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexey Babutin updated HADOOP-9395: --- Attachment: hadoop_jetty.patch jetty7 and hadoop 1.1.x --- Key: HADOOP-9395 URL: https://issues.apache.org/jira/browse/HADOOP-9395 Project: Hadoop Common Issue Type: Improvement Components: build, net Affects Versions: 1.1.2 Environment: FreeBSD 9.1+ZFS.2 xeon 5640,12Gb regDDR3,4 HDD 2Tb. Reporter: Alexey Babutin Priority: Minor Labels: patch Fix For: 1.2.0, 1.1.3 Attachments: hadoop_jetty.patch Change jetty6 to jetty7 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9395) jetty7 and hadoop 1.1.x
[ https://issues.apache.org/jira/browse/HADOOP-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599231#comment-13599231 ] Alexey Babutin commented on HADOOP-9395: Patch in attach. After patch,hadoop compile and work fine.But only one cheat. In HttpServer.java I have commited defineFilter(webAppContext, KRB5_FILTER ... because it is not working properly with defineFilter(...) string 435 in holder.setInitParameters(parameters). The,you can remove 448-450,i have used it for finding problem, and uncomment 447 string.it should work. After that,A had to comment 100,102,103 string in Krb5AndCertsSslSocketConnector.java. jetty7 and hadoop 1.1.x --- Key: HADOOP-9395 URL: https://issues.apache.org/jira/browse/HADOOP-9395 Project: Hadoop Common Issue Type: Improvement Components: build, net Affects Versions: 1.1.2 Environment: FreeBSD 9.1+ZFS.2 xeon 5640,12Gb regDDR3,4 HDD 2Tb. Reporter: Alexey Babutin Priority: Minor Labels: patch Fix For: 1.2.0, 1.1.3 Attachments: hadoop_jetty.patch Change jetty6 to jetty7 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9287) Parallel testing hadoop-common
[ https://issues.apache.org/jira/browse/HADOOP-9287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599246#comment-13599246 ] Andrey Klochkov commented on HADOOP-9287: - The patch affects a lot of tests and setting timeout for all of them shouldn't be done as part of this patch. TestDataDirs is broken in trunk currently, seems like it's caused by HADOOP-8973 Parallel testing hadoop-common -- Key: HADOOP-9287 URL: https://issues.apache.org/jira/browse/HADOOP-9287 Project: Hadoop Common Issue Type: Bug Components: test Affects Versions: 3.0.0 Reporter: Tsuyoshi OZAWA Assignee: Andrey Klochkov Attachments: HADOOP-9287.1.patch, HADOOP-9287--N3.patch, HADOOP-9287.patch, HADOOP-9287.patch The maven surefire plugin supports parallel testing feature. By using it, the tests can be run more faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9379) capture the ulimit info after printing the log to the console
[ https://issues.apache.org/jira/browse/HADOOP-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HADOOP-9379: Resolution: Fixed Fix Version/s: 2.0.5-beta 1.2.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed the patch to trunk, branch-2, branch-1 and branch-1.2. Thank you Arpit! Thanks to Junping for doing quick validation. capture the ulimit info after printing the log to the console - Key: HADOOP-9379 URL: https://issues.apache.org/jira/browse/HADOOP-9379 Project: Hadoop Common Issue Type: Improvement Affects Versions: 1.2.0, 2.0.4-alpha Reporter: Arpit Gupta Assignee: Arpit Gupta Priority: Trivial Fix For: 1.2.0, 2.0.5-beta Attachments: HADOOP-9379.branch-1.patch, HADOOP-9379.patch Based on the discussions in HADOOP-9253 people prefer if we dont print the ulimit info to the console but still have it in the logs. Just need to move the head statement to before the capture of ulimit code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8973) DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs
[ https://issues.apache.org/jira/browse/HADOOP-8973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599255#comment-13599255 ] Andrey Klochkov commented on HADOOP-8973: - Seems this patch has broken test TestDataDirs in trunk. On a side note, shouldn't we improve patch verifier to catch such issues? The reason the issue wasn't detected is that the robot doesn't run tests for modules which depend on the modified ones. Shouldn't we fix it? DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs - Key: HADOOP-8973 URL: https://issues.apache.org/jira/browse/HADOOP-8973 Project: Hadoop Common Issue Type: Bug Components: util Affects Versions: 3.0.0, 1-win Reporter: Chris Nauroth Assignee: Chris Nauroth Fix For: 3.0.0, 1-win Attachments: DiskChecker.proto.patch, HADOOP-8973.3.patch, HADOOP-8973-branch-1-win.3.patch, HADOOP-8973-branch-trunk-win.2.patch, HADOOP-8973-branch-trunk-win.patch DiskChecker.checkDir uses File.canRead, File.canWrite, and File.canExecute to check if a directory is inaccessible. These APIs are not reliable on Windows with NTFS ACLs due to a known JVM bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9379) capture the ulimit info after printing the log to the console
[ https://issues.apache.org/jira/browse/HADOOP-9379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599264#comment-13599264 ] Hudson commented on HADOOP-9379: Integrated in Hadoop-trunk-Commit #3447 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3447/]) HADOOP-9379. capture the ulimit info after printing the log to the console. Contributed by Arpit Gupta. (Revision 1455316) Result = SUCCESS suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1455316 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/bin/hadoop-daemon.sh * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/bin/yarn-daemon.sh capture the ulimit info after printing the log to the console - Key: HADOOP-9379 URL: https://issues.apache.org/jira/browse/HADOOP-9379 Project: Hadoop Common Issue Type: Improvement Affects Versions: 1.2.0, 2.0.4-alpha Reporter: Arpit Gupta Assignee: Arpit Gupta Priority: Trivial Fix For: 1.2.0, 2.0.5-beta Attachments: HADOOP-9379.branch-1.patch, HADOOP-9379.patch Based on the discussions in HADOOP-9253 people prefer if we dont print the ulimit info to the console but still have it in the logs. Just need to move the head statement to before the capture of ulimit code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-9395) jetty7 and hadoop 1.1.x
[ https://issues.apache.org/jira/browse/HADOOP-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Lu resolved HADOOP-9395. - Resolution: Duplicate Fix Version/s: (was: 1.1.3) (was: 1.2.0) Thanks for the patch Alexey! The usual process here is to patch trunk first then backport to branch-1. I'll assign HADOOP-8515 to you. Let me know if you don't want to work on it :) jetty7 and hadoop 1.1.x --- Key: HADOOP-9395 URL: https://issues.apache.org/jira/browse/HADOOP-9395 Project: Hadoop Common Issue Type: Improvement Components: build, net Affects Versions: 1.1.2 Environment: FreeBSD 9.1+ZFS.2 xeon 5640,12Gb regDDR3,4 HDD 2Tb. Reporter: Alexey Babutin Priority: Minor Labels: patch Attachments: hadoop_jetty.patch Change jetty6 to jetty7 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9393) TestRPC fails with JDK7
[
https://issues.apache.org/jira/browse/HADOOP-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-9393:
Attachment: hadoop-9393-1-whitespace.patch
TestRPC fails with JDK7
---
Key: HADOOP-9393
URL: https://issues.apache.org/jira/browse/HADOOP-9393
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Attachments: hadoop-9393-1-whitespace.patch
Since the test order is different with JDK7, we hit an error in
{{TestRPC#testStopsAllThreads}} on the initial assertEquals. This is because
{{testRPCInterruptedSimple}} and {{testRPCInterrupted}} don't stop the server
after they finish, leaving some threads around.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9393) TestRPC fails with JDK7
[
https://issues.apache.org/jira/browse/HADOOP-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-9393:
Attachment: hadoop-9393-1.patch
Patch attached. It's simply wrapping these tests in a try/finally with
#server.stop(). The whitespace changes make this look huge, but I attached a
{{git diff -w}} version of the patch for easy review.
Tested by running {{TestRPC}} 10 times with JDK7, all passed.
TestRPC fails with JDK7
---
Key: HADOOP-9393
URL: https://issues.apache.org/jira/browse/HADOOP-9393
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Attachments: hadoop-9393-1.patch, hadoop-9393-1-whitespace.patch
Since the test order is different with JDK7, we hit an error in
{{TestRPC#testStopsAllThreads}} on the initial assertEquals. This is because
{{testRPCInterruptedSimple}} and {{testRPCInterrupted}} don't stop the server
after they finish, leaving some threads around.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9393) TestRPC fails with JDK7
[
https://issues.apache.org/jira/browse/HADOOP-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-9393:
Status: Patch Available (was: Open)
TestRPC fails with JDK7
---
Key: HADOOP-9393
URL: https://issues.apache.org/jira/browse/HADOOP-9393
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Attachments: hadoop-9393-1.patch, hadoop-9393-1-whitespace.patch
Since the test order is different with JDK7, we hit an error in
{{TestRPC#testStopsAllThreads}} on the initial assertEquals. This is because
{{testRPCInterruptedSimple}} and {{testRPCInterrupted}} don't stop the server
after they finish, leaving some threads around.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-9394:
Status: Patch Available (was: Open)
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HADOOP-9394:
Attachment: hadoop-9394-1.patch
Patch attached. It's just a cp from hbase trunk to here, I've used this script
multiple times myself successfully.
No tests for obvious reasons.
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599349#comment-13599349
]
David S. Wang commented on HADOOP-9394:
---
+1 (non-binding)
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599371#comment-13599371
]
Ted Yu commented on HADOOP-9394:
One enhancement for the script is that we should detect interleaving of tests
in console output.
e.g. from https://builds.apache.org/job/PreCommit-HBASE-Build/4760/console :
{code}
Running org.apache.hadoop.hbase.io.encoding.TestChangingEncoding
Running org.apache.hadoop.hbase.regionserver.TestStore
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.224 sec
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 188.814 sec
{code}
The two tests actually both passed. However, the script would think that
TestChangingEncoding hung.
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599378#comment-13599378
]
Hadoop QA commented on HADOOP-9394:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573179/hadoop-9394-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2312//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2312//console
This message is automatically generated.
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599379#comment-13599379
]
Andrew Wang commented on HADOOP-9394:
-
Hi Ted,
Could we do that fix in a follow-on JIRA? The script is already quite useful in
its current form.
Feel free to report in HBase and Hadoop and link together.
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9394) Port findHangingTest.sh from HBase to Hadoop
[
https://issues.apache.org/jira/browse/HADOOP-9394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599386#comment-13599386
]
Ted Yu commented on HADOOP-9394:
follow-on JIRA is fine for enhancement.
Port findHangingTest.sh from HBase to Hadoop
Key: HADOOP-9394
URL: https://issues.apache.org/jira/browse/HADOOP-9394
Project: Hadoop Common
Issue Type: Improvement
Reporter: Andrew Wang
Assignee: Andrew Wang
Priority: Minor
Attachments: hadoop-9394-1.patch
HBase has this handy {{dev-support/findHangingTests.sh}} script, which parses
Jenkins consoleText and finds hanging tests for you. This has been especially
useful for identifying balancer test timeouts (see HDFS-4376 and HDFS-4261).
It'd be nice to have this in our own {{dev-support}} directory.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9393) TestRPC fails with JDK7
[
https://issues.apache.org/jira/browse/HADOOP-9393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599403#comment-13599403
]
Hadoop QA commented on HADOOP-9393:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573178/hadoop-9393-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 tests included appear to have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2313//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2313//console
This message is automatically generated.
TestRPC fails with JDK7
---
Key: HADOOP-9393
URL: https://issues.apache.org/jira/browse/HADOOP-9393
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Andrew Wang
Assignee: Andrew Wang
Attachments: hadoop-9393-1.patch, hadoop-9393-1-whitespace.patch
Since the test order is different with JDK7, we hit an error in
{{TestRPC#testStopsAllThreads}} on the initial assertEquals. This is because
{{testRPCInterruptedSimple}} and {{testRPCInterrupted}} don't stop the server
after they finish, leaving some threads around.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9371) Define Semantics of FileSystem and FileContext more rigorously
[
https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599411#comment-13599411
]
Matthew Farrellee commented on HADOOP-9371:
---
Page 2, Concurrency, you mention mkdir/mkdirs is atomic
It seems reasonable that mkdir is atomic.
I've been researching mkdirs(), with a focus on idempotence and atomicity.
ClientProtocol.java:mkdirs() clearly labels it as @Idempotent, and the
documentation and various implementations support that claim. It's also a
property that is relatively straight-forward to implement on many back-end
filesystems.
I'm having more difficulty tracking down the atomicity of mkdirs(). The LocalFS
implementations are not themselves atomic. I tracked the HDFS implementation
back to FSNamesystem.java:mkdirsInt(), which appears to provide an atomic
implementation. However, the atomic nature of mkdirsInt() appears to come from
HDFS-988, which looks to fix a bug by making mkdirs() atomic rather having an
explicit purpose of making mkdirs() atomic by design.
How are you getting to mkdirs() as atomic?
A mild concern of mine is that even if mkdirs() isn't atomic by design, for
HDFS it has been implemented as atomic and who knows who may silently be
relying on the not-by-design atomic property. That said, given mkdirs() is
idempotent it isn't suitable for use as a locking mechanism.
Define Semantics of FileSystem and FileContext more rigorously
--
Key: HADOOP-9371
URL: https://issues.apache.org/jira/browse/HADOOP-9371
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 1.2.0, 3.0.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HadoopFilesystemContract.pdf
Original Estimate: 48h
Remaining Estimate: 48h
The semantics of {{FileSystem}} and {{FileContext}} are not completely
defined in terms of
# core expectations of a filesystem
# consistency requirements.
# concurrency requirements.
# minimum scale limits
Furthermore, methods are not defined strictly enough in terms of their
outcomes and failure modes.
The requirements and method semantics should be defined more strictly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Affects Version/s: 2.0.0-alpha Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381_1.patch, HADOOP-9381_2.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Affects Version/s: 2.0.0-alpha Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382_1.patch, HADOOP-9382_2.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9387) TestDFVariations fails on Windows after the merge
[
https://issues.apache.org/jira/browse/HADOOP-9387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Mitic updated HADOOP-9387:
---
Status: Open (was: Patch Available)
I see regressions in HDFS tests on Windows with this patch because HDFS invokes
DF#getFileSystem() which in turn calls DF#run(). Canceling the patch until I
address both.
TestDFVariations fails on Windows after the merge
-
Key: HADOOP-9387
URL: https://issues.apache.org/jira/browse/HADOOP-9387
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
Attachments: HADOOP-9387.trunk.patch
Test fails with the following errors:
{code}
Running org.apache.hadoop.fs.TestDFVariations
Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.186 sec
FAILURE!
testOSParsing(org.apache.hadoop.fs.TestDFVariations) Time elapsed: 109 sec
ERROR!
java.io.IOException: Fewer lines of output than expected
at org.apache.hadoop.fs.DF.parseOutput(DF.java:203)
at org.apache.hadoop.fs.DF.getMount(DF.java:150)
at
org.apache.hadoop.fs.TestDFVariations.testOSParsing(TestDFVariations.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
testGetMountCurrentDirectory(org.apache.hadoop.fs.TestDFVariations) Time
elapsed: 1 sec ERROR!
java.io.IOException: Fewer lines of output than expected
at org.apache.hadoop.fs.DF.parseOutput(DF.java:203)
at org.apache.hadoop.fs.DF.getMount(DF.java:150)
at
org.apache.hadoop.fs.TestDFVariations.testGetMountCurrentDirectory(TestDFVariations.java:139)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.FailOnTimeout$1.run(FailOnTimeout.java:28)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9392) Token based authentication and Single Sign On
[ https://issues.apache.org/jira/browse/HADOOP-9392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599649#comment-13599649 ] Thomas NGUY commented on HADOOP-9392: - Wonderful Kai !! For starting, I'd like to ask some questions about the subject, as I have no experience with Hadoop, some points are still unclear for me. Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at the RPC layer, via SASL. However this does not provide valuable attributes such as group membership, classification level, organizational identity, or support for user defined attributes. Hadoop components must interrogate external resources for discovering these attributes and at scale this is problematic I've seen that the NameNode and JobTracker get informations about the user by using its username and a pluggable interface that maps the username to a set of groups that the user belongs. Is this method problematic at larger scale? What do we have to do in that case? Include the user informations in the token? We will implement a common token based authentication framework to decouple internal user and service authentication from external mechanisms used to support it (like Kerberos)” Here also, what is the problem with the token based authentification kerberos? What does common token based authentification means? Is there a link with the interactions of Hadoop components (see link http://clustermania.blogspot.jp/2011/11/hadoop-how-it-manages-security.html) ?? These questions seem stupid but I really need to understand more about the subject before starting ^^. Oh BTW I'm doing my master research at the NII (National Institute of Informatic) in Tokyo, Its already late night so I might not be able to answer the same day. Best regards. Thomas Token based authentication and Single Sign On - Key: HADOOP-9392 URL: https://issues.apache.org/jira/browse/HADOOP-9392 Project: Hadoop Common Issue Type: New Feature Components: security Reporter: Kai Zheng Fix For: 3.0.0 This is an umbrella entry for one of project Rhino’s topic, for details of project Rhino, please refer to https://github.com/intel-hadoop/project-rhino/. The major goal for this entry as described in project Rhino was “Core, HDFS, ZooKeeper, and HBase currently support Kerberos authentication at the RPC layer, via SASL. However this does not provide valuable attributes such as group membership, classification level, organizational identity, or support for user defined attributes. Hadoop components must interrogate external resources for discovering these attributes and at scale this is problematic. There is also no consistent delegation model. HDFS has a simple delegation capability, and only Oozie can take limited advantage of it. We will implement a common token based authentication framework to decouple internal user and service authentication from external mechanisms used to support it (like Kerberos)” We’d like to start our work from Hadoop-Common and try to provide common facilities by extending existing authentication framework which support: 1.Pluggable token provider interface 2.Pluggable token verification protocol and interface 3.Security mechanism to distribute secrets in cluster nodes 4.Delegation model of user authentication -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9371) Define Semantics of FileSystem and FileContext more rigorously
[
https://issues.apache.org/jira/browse/HADOOP-9371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599651#comment-13599651
]
Arun C Murthy commented on HADOOP-9371:
---
+1 for this effort - thanks for taking this on Steve!
Define Semantics of FileSystem and FileContext more rigorously
--
Key: HADOOP-9371
URL: https://issues.apache.org/jira/browse/HADOOP-9371
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 1.2.0, 3.0.0
Reporter: Steve Loughran
Assignee: Steve Loughran
Attachments: HadoopFilesystemContract.pdf
Original Estimate: 48h
Remaining Estimate: 48h
The semantics of {{FileSystem}} and {{FileContext}} are not completely
defined in terms of
# core expectations of a filesystem
# consistency requirements.
# concurrency requirements.
# minimum scale limits
Furthermore, methods are not defined strictly enough in terms of their
outcomes and failure modes.
The requirements and method semantics should be defined more strictly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Attachment: HADOOP-9381_3.patch Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381_1.patch, HADOOP-9381_2.patch, HADOOP-9381_3.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Attachment: HADOOP-9382_3.patch Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382_1.patch, HADOOP-9382_2.patch, HADOOP-9382_3.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Attachment: (was: HADOOP-9381_1.patch) Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Attachment: (was: HADOOP-9381_2.patch) Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Attachment: (was: HADOOP-9381_3.patch) Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Attachment: (was: HADOOP-9382_3.patch) Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Attachment: HADOOP-9381.patch Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Attachment: (was: HADOOP-9382_1.patch) Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Attachment: (was: HADOOP-9382_2.patch) Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Attachment: HADOOP-9382.patch Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Status: Patch Available (was: Open) Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381.patch fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Status: Patch Available (was: Open) Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9381) Document dfs cp -f option
[ https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9381: Description: dfs cp should document -f (overwrite) option in the page displayed by -help. Additionally, the HTML documentation page should also document this option and all the options should all be formatted the same. (was: fs shell should document -f (overwrite) option in the page displayed by -help. A patch is included that should correct that. Additionally, the latest web [documentation page|http://hadoop.apache.org/docs/r1.1.2/file_system_shell.html] should also be updated (I don't know where the source for that is to include a patch).) Document dfs cp -f option - Key: HADOOP-9381 URL: https://issues.apache.org/jira/browse/HADOOP-9381 Project: Hadoop Common Issue Type: Bug Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Trivial Attachments: HADOOP-9381.patch dfs cp should document -f (overwrite) option in the page displayed by -help. Additionally, the HTML documentation page should also document this option and all the options should all be formatted the same. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keegan Witt updated HADOOP-9382: Description: Add a -f option to allow overwriting existing destinations in dfs mv command. (was: Add a -f option to allow overwriting existing destinations in mv command.) Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in dfs mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9382) Add dfs mv overwrite option
[ https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599693#comment-13599693 ] Keegan Witt commented on HADOOP-9382: - I finally got the compile working. The patch includes a unit test. Add dfs mv overwrite option --- Key: HADOOP-9382 URL: https://issues.apache.org/jira/browse/HADOOP-9382 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Keegan Witt Priority: Minor Attachments: HADOOP-9382.patch Add a -f option to allow overwriting existing destinations in dfs mv command. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9382) Add dfs mv overwrite option
[
https://issues.apache.org/jira/browse/HADOOP-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599712#comment-13599712
]
Hadoop QA commented on HADOOP-9382:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573255/HADOOP-9382.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 one of tests included doesn't have a timeout.{color}
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.fs.TestFsShellCopy
org.apache.hadoop.cli.TestCLI
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2314//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2314//console
This message is automatically generated.
Add dfs mv overwrite option
---
Key: HADOOP-9382
URL: https://issues.apache.org/jira/browse/HADOOP-9382
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 2.0.0-alpha
Reporter: Keegan Witt
Priority: Minor
Attachments: HADOOP-9382.patch
Add a -f option to allow overwriting existing destinations in dfs mv command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-9381) Document dfs cp -f option
[
https://issues.apache.org/jira/browse/HADOOP-9381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13599713#comment-13599713
]
Hadoop QA commented on HADOOP-9381:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12573254/HADOOP-9381.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.cli.TestCLI
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2315//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/2315//console
This message is automatically generated.
Document dfs cp -f option
-
Key: HADOOP-9381
URL: https://issues.apache.org/jira/browse/HADOOP-9381
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Keegan Witt
Priority: Trivial
Attachments: HADOOP-9381.patch
dfs cp should document -f (overwrite) option in the page displayed by -help.
Additionally, the HTML documentation page should also document this option
and all the options should all be formatted the same.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
