[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[ https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14057526#comment-14057526 ] Hudson commented on HDFS-6638: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1827 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1827/]) shorten test run time with a smaller retry timeout setting -- Key: HDFS-6638 URL: https://issues.apache.org/jira/browse/HDFS-6638 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 3.0.0, 2.5.0 Reporter: Liang Xie Assignee: Liang Xie Fix For: 3.0.0, 2.6.0 Attachments: HDFS-6638.txt similiar with HDFS-6614, i think it's a general test duration optimization tip, so i grep IOException, will wait for from a full test run under hdfs project, found several test cases could be optimized, so made a simple patch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[ https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056077#comment-14056077 ] Hudson commented on HDFS-6638: -- FAILURE: Integrated in Hadoop-Yarn-trunk #608 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/608/]) HDFS-6638. Shorten test run time with a smaller retry timeout setting. Contributed by Liang Xie. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1608905) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockMissingException.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestCrcCorruption.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestFailoverWithBlockTokensEnabled.java shorten test run time with a smaller retry timeout setting -- Key: HDFS-6638 URL: https://issues.apache.org/jira/browse/HDFS-6638 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 3.0.0, 2.5.0 Reporter: Liang Xie Assignee: Liang Xie Fix For: 3.0.0, 2.6.0 Attachments: HDFS-6638.txt similiar with HDFS-6614, i think it's a general test duration optimization tip, so i grep IOException, will wait for from a full test run under hdfs project, found several test cases could be optimized, so made a simple patch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[ https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14056121#comment-14056121 ] Hudson commented on HDFS-6638: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #1799 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1799/]) HDFS-6638. Shorten test run time with a smaller retry timeout setting. Contributed by Liang Xie. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1608905) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockMissingException.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestCrcCorruption.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestFailoverWithBlockTokensEnabled.java shorten test run time with a smaller retry timeout setting -- Key: HDFS-6638 URL: https://issues.apache.org/jira/browse/HDFS-6638 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 3.0.0, 2.5.0 Reporter: Liang Xie Assignee: Liang Xie Fix For: 3.0.0, 2.6.0 Attachments: HDFS-6638.txt similiar with HDFS-6614, i think it's a general test duration optimization tip, so i grep IOException, will wait for from a full test run under hdfs project, found several test cases could be optimized, so made a simple patch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[ https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054813#comment-14054813 ] Liang Xie commented on HDFS-6638: - with attached patch, we could reduce more than 2.5 minutes. i set the retry value to 10ms instead of 0, just considering i am not familiar with those modified testing, so probably will have a crazy loop happened:) ps: TestEncryptedTransfer#testLongLivedReadClientAfterRestart seems failed after i tune the setting to 10ms, so i omit it in current patch. shorten test run time with a smaller retry timeout setting -- Key: HDFS-6638 URL: https://issues.apache.org/jira/browse/HDFS-6638 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 3.0.0 Reporter: Liang Xie Assignee: Liang Xie Attachments: HDFS-6638.txt similiar with HDFS-6614, i think it's a general test duration optimization tip, so i grep IOException, will wait for from a full test run under hdfs project, found several test cases could be optimized, so made a simple patch. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[
https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054929#comment-14054929
]
Hadoop QA commented on HDFS-6638:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12654565/HDFS-6638.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 11 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7289//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7289//console
This message is automatically generated.
shorten test run time with a smaller retry timeout setting
--
Key: HDFS-6638
URL: https://issues.apache.org/jira/browse/HDFS-6638
Project: Hadoop HDFS
Issue Type: Test
Components: test
Affects Versions: 3.0.0
Reporter: Liang Xie
Assignee: Liang Xie
Attachments: HDFS-6638.txt
similiar with HDFS-6614, i think it's a general test duration optimization
tip, so i grep IOException, will wait for from a full test run under hdfs
project, found several test cases could be optimized, so made a simple patch.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[
https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055338#comment-14055338
]
Chris Nauroth commented on HDFS-6638:
-
Before:
{code}
Running org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.404 sec - in
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
Running
org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.742 sec - in
org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
Running org.apache.hadoop.hdfs.server.namenode.TestFsck
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.438 sec -
in org.apache.hadoop.hdfs.server.namenode.TestFsck
Running org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.352 sec - in
org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
Running org.apache.hadoop.hdfs.TestBlockMissingException
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.265 sec - in
org.apache.hadoop.hdfs.TestBlockMissingException
Running org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 5.88 sec - in
org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.445 sec - in
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestCrcCorruption
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.005 sec - in
org.apache.hadoop.hdfs.TestCrcCorruption
Running org.apache.hadoop.hdfs.TestDFSShell
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.658 sec -
in org.apache.hadoop.hdfs.TestDFSShell
Running org.apache.hadoop.hdfs.TestEncryptedTransfer
Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 135.511 sec -
in org.apache.hadoop.hdfs.TestEncryptedTransfer
Running org.apache.hadoop.hdfs.TestMissingBlocksAlert
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.831 sec - in
org.apache.hadoop.hdfs.TestMissingBlocksAlert
{code}
After:
{code}
Running org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 27.858 sec - in
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
Running
org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.841 sec - in
org.apache.hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
Running org.apache.hadoop.hdfs.server.namenode.TestFsck
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.374 sec -
in org.apache.hadoop.hdfs.server.namenode.TestFsck
Running org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.593 sec - in
org.apache.hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
Running org.apache.hadoop.hdfs.TestBlockMissingException
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.155 sec - in
org.apache.hadoop.hdfs.TestBlockMissingException
Running org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
Tests run: 2, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 2.557 sec - in
org.apache.hadoop.hdfs.TestBlockReaderLocalLegacy
Running org.apache.hadoop.hdfs.TestClientReportBadBlock
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.937 sec - in
org.apache.hadoop.hdfs.TestClientReportBadBlock
Running org.apache.hadoop.hdfs.TestCrcCorruption
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.63 sec - in
org.apache.hadoop.hdfs.TestCrcCorruption
Running org.apache.hadoop.hdfs.TestDFSShell
Tests run: 29, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.688 sec -
in org.apache.hadoop.hdfs.TestDFSShell
Running org.apache.hadoop.hdfs.TestEncryptedTransfer
Tests run: 24, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.515 sec -
in org.apache.hadoop.hdfs.TestEncryptedTransfer
Running org.apache.hadoop.hdfs.TestMissingBlocksAlert
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.707 sec - in
org.apache.hadoop.hdfs.TestMissingBlocksAlert
{code}
shorten test run time with a smaller retry timeout setting
--
Key: HDFS-6638
URL: https://issues.apache.org/jira/browse/HDFS-6638
Project: Hadoop HDFS
Issue Type: Test
Components: test
Affects Versions: 3.0.0, 2.5.0
Reporter: Liang Xie
Assignee: Liang Xie
Attachments: HDFS-6638.txt
[jira] [Commented] (HDFS-6638) shorten test run time with a smaller retry timeout setting
[ https://issues.apache.org/jira/browse/HDFS-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14055357#comment-14055357 ] Hudson commented on HDFS-6638: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5844 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5844/]) HDFS-6638. Shorten test run time with a smaller retry timeout setting. Contributed by Liang Xie. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1608905) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockMissingException.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockReaderLocalLegacy.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestCrcCorruption.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockTokenWithDFS.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestListCorruptFileBlocks.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestFailoverWithBlockTokensEnabled.java shorten test run time with a smaller retry timeout setting -- Key: HDFS-6638 URL: https://issues.apache.org/jira/browse/HDFS-6638 Project: Hadoop HDFS Issue Type: Test Components: test Affects Versions: 3.0.0, 2.5.0 Reporter: Liang Xie Assignee: Liang Xie Attachments: HDFS-6638.txt similiar with HDFS-6614, i think it's a general test duration optimization tip, so i grep IOException, will wait for from a full test run under hdfs project, found several test cases could be optimized, so made a simple patch. -- This message was sent by Atlassian JIRA (v6.2#6252)
