[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401207#comment-16401207 ] Chris Douglas commented on HDFS-11600: -- Thanks, [~Sammi]! > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: SammiChen >Priority: Minor > Fix For: 3.1.0, 3.0.2 > > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch, HDFS-11600.005.patch, > HDFS-11600.006.patch, HDFS-11600.007.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398325#comment-16398325 ] Hudson commented on HDFS-11600: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13833 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13833/]) HDFS-11600. Refactor TestDFSStripedOutputStreamWithFailure test classes. (sammi.chen: rev ad1b988a828608b12cafb6382436cd17f95bfcc5) * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/ParameterizedTestDFSStripedOutputStreamWithFailure.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure190.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure030.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure210.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure010.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure060.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/ParameterizedTestDFSStripedOutputStreamWithFailureWithRandomECPolicy.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure120.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure020.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure090.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure160.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure140.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure200.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure000.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure040.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure180.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure150.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure100.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure050.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure170.java * (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailureBase.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure070.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure110.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure080.java * (delete) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure130.java > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: SammiChen >Priority: Minor > Fix For: 3.1.0, 3.0.2 > > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch, HDFS-11600.005.patch, > HDFS-11600.006.patch, HDFS-11600.007.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIR
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396881#comment-16396881 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 47s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new + 394 unchanged - 3 fixed = 394 total (was 397) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 41s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 0s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 43s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}120m 44s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12914239/HDFS-11600.007.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 379a52c90518 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 0355ec2 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23448/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23448/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/23448/testRepo
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396772#comment-16396772 ] SammiChen commented on HDFS-11600: -- Thanks [~xiaochen] for the review. I uploaded 007 patch to address the line length checkstyle issue. Will commit after the pre-commit build comes out. Also thanks [~andrew.wang] for the initial patches and [~rakeshr] for the review. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch, HDFS-11600.005.patch, > HDFS-11600.006.patch, HDFS-11600.007.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396599#comment-16396599 ] Xiao Chen commented on HDFS-11600: -- Thanks Sammi, makes sense. +1 from me on patch 6 pending the line length checkstyle. (The VisibilityModifier stuff feels okay to me, since it's in tests and IMO getters would be harder to read than protected vars) Still think the {{tearDown}} case can be improved and {{testBlockTokenExpired}} modified for that, but that can be done in a separate jira. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch, HDFS-11600.005.patch, > HDFS-11600.006.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396573#comment-16396573 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 51s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new + 394 unchanged - 3 fixed = 394 total (was 397) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 40s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 12 new + 0 unchanged - 0 fixed = 12 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 58s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 74m 13s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 24s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}122m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestDFSClientRetries | | | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12914209/HDFS-11600.006.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 8322e630daa7 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 45d1b0f | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23442/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23442/artifact/out/patch-unit-hadoop-hdfs-project_
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396493#comment-16396493 ] SammiChen commented on HDFS-11600: -- Thanks [~xiaochen] for the comments. bq. Do you know why this range was chosen? I didn't know initial reason. By going through the code, I guess at the moment the TestDFSStripedOutputStreamWithFailure is introduced, the only supported EC policy is RS-6-3-64K. The intent is to test the file with length varies from [0, 1, 2] block groups, each time block group's cell number varies from [0 - (6(data block number)*4(cell per block)-1] , plus [-1,0,1] delta length. So approximately there will be total 3 * ((6 * 4) -1) * 3 = 207 length variants. While now we support more EC policies, especially RS-10-4, so the previous 210 variants doesn't stand any more. Actually the variants should varies when different EC polices is used inTestDFSStripedOutputStreamWithFailureWithRandomECPolicy. bq. This is from existing code, but now may be a good chance to change - could you do tearDown with a @After annotation? This way, each test doesn't have to try-finally. Agree. While there is a loop in testBlockTokenExpired which requires setup and tearDown the cluster every iterate. So It seems better to keep it. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch, HDFS-11600.005.patch, > HDFS-11600.006.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395689#comment-16395689 ] Xiao Chen commented on HDFS-11600: -- Thanks Sammi for the work here, LGTM in general! Just have 1 question and a few minor comments. Questions: - Looking at the original tests, I think TestDFSStripedOutputStreamWithFailure[000-210] covered continuously the range [0, 219], judging from this [code|https://github.com/apache/hadoop/blob/branch-3.0.0/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStreamWithFailure.java#L714-L728]. Do you know why this range was chosen? In current patch, we test randomly in the same range but not continuously. But I think that should be fine. Comments: - import * is usually discouraged - less strict for tests, but please change if possible. - Should use slf4j Logger in the new code as we're (slowly) moving away from apache commons logging - This is from existing code, but now may be a good chance to change - could you do {{tearDown}} with a {{@After}} annotation? This way, each test doesn't have to try-finally. - Can still see the unused {{testCloseWithExceptionsInStreamer}} in {{TestDFSStripedOutputStreamWithFailureBase}} > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch, HDFS-11600.005.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395294#comment-16395294 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 36s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 12s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 5s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new + 394 unchanged - 3 fixed = 394 total (was 397) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 49s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 13 new + 0 unchanged - 0 fixed = 13 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 43s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}109m 58s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}166m 26s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.TestFileAppend | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | | | hadoop.hdfs.server.balancer.TestBalancer | | | hadoop.hdfs.server.datanode.TestDataNodeMXBean | | | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics | | | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA | | | hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12914009/HDFS-11600.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux bb798518e7fb 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394916#comment-16394916 ] SammiChen commented on HDFS-11600: -- Hi [~rakeshr], thanks for the comments. I will upload a new patch to address all the issues. Regarding the class name with "P", you are right, it stands for "parameterized". I was originally using the full word, then found out the class name becomes very very long, especially "TestDFSStripedOutputStreamWithFailureWithRandomECPolicy". > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394067#comment-16394067 ] Rakesh R commented on HDFS-11600: - Thanks [~Sammi] for the useful work. Overall the idea looks good to me. I've few comments, please take a look at the below points. # Please keep all the member variables together at the top of the class for better readbility. {code} private int[][] dnIndexSuite; protected List lengths; protected static final Random RANDOM = new Random(); MiniDFSCluster cluster; DistributedFileSystem dfs; final Path dir = new Path("/" + TestDFSStripedOutputStreamWithFailureBase.class.getSimpleName()); {code} # I hope you have named the class with "P" to represent parameterized class. Can we give a meaningful name instead of appending with letter "P" - {{TestDFSStripedOutputStreamWithFailureP}}, {{TestDFSStripedOutputStreamWithFailurePWithRandomECPolicy}}. # Do we need both timeouts? {code} + @Rule + public Timeout globalTimeout = new Timeout(60); + + @Test(timeout = 24) + public void run() { {code} # {{TestDFSStripedOutputStreamWithFailureBase#testCloseWithExceptionsInStreamer}} function is not used anywhere. Whats the purpose of this? # Please give proper naming to this test. {code} @Test(timeout = 24) public void run() { {code} > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390803#comment-16390803 ] SammiChen commented on HDFS-11600: -- Hi [~rakeshr], do you have time to help review the patch? > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390736#comment-16390736 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 29s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 40s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 47s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new + 400 unchanged - 3 fixed = 400 total (was 403) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 38s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 7s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}113m 16s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}164m 16s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12913359/HDFS-11600.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0e7ed5e7b882 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 583f459 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23341/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23341/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/23341/testRepo
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389486#comment-16389486 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 1s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s{color} | {color:green} hadoop-hdfs-project_hadoop-hdfs generated 0 new + 400 unchanged - 3 fixed = 400 total (was 403) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 42s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 4s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}110m 50s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}159m 10s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12913359/HDFS-11600.004.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux e4215285047a 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 58ea2d7 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23330/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23330/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results |
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389328#comment-16389328 ] SammiChen commented on HDFS-11600: -- Handle javac issues and improve the {{testMultipleDatanodeFailure56}} to make sure will not run out of heap size. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch, HDFS-11600.004.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389121#comment-16389121 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 17m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 47s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 49s{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs generated 3 new + 400 unchanged - 3 fixed = 403 total (was 403) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 43s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 13s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}108m 8s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}174m 3s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12910082/HDFS-11600.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 7a3a8ef7d904 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / edf9445 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | javac | https://builds.apache.org/job/PreCommit-HDFS-Build/23327/artifact/out/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/23327/artifact/out/diff-checkstyle-hado
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387171#comment-16387171 ] SammiChen commented on HDFS-11600: -- The output links in last build expired. Try to trigger the build again. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359913#comment-16359913 ] genericqa commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 33s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 26 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 52s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 51s{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs generated 3 new + 388 unchanged - 3 fixed = 391 total (was 391) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 34s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch generated 11 new + 0 unchanged - 0 fixed = 11 total (was 0) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 15s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}110m 46s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}159m 17s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA | | | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.server.namenode.TestNameNodeMXBean | | | hadoop.hdfs.TestReadStripedFileWithMissingBlocks | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12910082/HDFS-11600.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 049e0b378d9c 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20 11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a08c048 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | javac | https://builds.apache.org/job/PreCommit-HDFS-B
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359850#comment-16359850 ] SammiChen commented on HDFS-11600: -- I uploaded a new patch based on Andrew's 002 patch. The idea is to separate {{TestDFSStripedOutputStreamWithFailure}} into {{TestDFSStripedOutputStreamWithFailureBase}} which carries common routines and variable defines, {{TestDFSStripedOutputStreamWithFailure}} which carries fixed parameter test cases, and {{TestDFSStripedOutputStreamWithFailureP}} which carries parameterized test case. In {{TestDFSStripedOutputStreamWithFailureP}}, I refine the current test case. Each time it will randomly choose 10 file length to run the test case. Given that the largest built-in EC policy currently support is RS-10-4-1MB, 10 rounds of same test case with random 1 data node failure seems enough. [~andrew.wang], would you take a look at the new patch at your convenient time? > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch, > HDFS-11600.003.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359055#comment-16359055 ] Andrew Wang commented on HDFS-11600: Sure, go for it Sammi :) > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16358077#comment-16358077 ] SammiChen commented on HDFS-11600: -- Hi [~andrew.wang], the idea of using JUnit parameterize to really good, it helps to cleanup the messy test cases. I took a further look into the TestDFSStripedOutputStreamWithFailure. Many test cases are constant, not related with parameter. So I think we can further split the TestDFSStripedOutputStreamWithFailure into 2 files, one with constant test case, another is parameterized. What do you think? By the way, If you don't have too much time lately, I can take over it. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Priority: Minor > Attachments: HDFS-11600-1.patch, HDFS-11600.002.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162192#comment-16162192 ] Chris Douglas commented on HDFS-11600: -- bq. I posted an alternative disable patch on HDFS-12417 Sure. No reason to block CI stability on a test refactor. Cancelled patch. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600.002.patch, HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162122#comment-16162122 ] Andrew Wang commented on HDFS-11600: I posted an alternative disable patch on HDFS-12417. I don't think we can do that much better when it comes to refactoring. We can parameterize using JUnit instead of the classname and add better docs, but I don't know how to get around having all these different test classes. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600.002.patch, HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162074#comment-16162074 ] Chris Douglas commented on HDFS-11600: -- Evidently there's a whole suite of tests using this pattern. {{TestDFSStripedOutputStreamWithFailureWithRandomECPolicy}} is completely broken. [~andrew.wang], these tests need to be disabled until they can be fixed. Can we delete the subclassses as part of HDFS-12417 and follow up with this later? > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600.002.patch, HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162045#comment-16162045 ] Hadoop QA commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 23 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 45s{color} | {color:red} hadoop-hdfs-project_hadoop-hdfs generated 3 new + 405 unchanged - 3 fixed = 408 total (was 408) {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 31s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 17s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}123m 27s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestReconstructStripedFile | | | hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy | | | hadoop.hdfs.TestClientProtocolForPipelineRecovery | | | hadoop.hdfs.TestLeaseRecoveryStriped | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy | | Timed out junit tests | org.apache.hadoop.hdfs.TestWriteReadStripedFile | | | org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12886483/HDFS-11600.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 5c14e8c500aa 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / de9994b | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | javac | https://builds.apache.org/job/PreCommit-HDFS-Build/21073/artifact/patchprocess/diff-compile-javac-hadoop-hdfs-project_hadoop-hdfs.txt | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/21073/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/21073/testReport/ | | modules | C: hadoop-hdfs
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161940#comment-16161940 ] Chris Douglas commented on HDFS-11600: -- bq. The proliferation of test classes still sucks from a code cleanliness POV, so I'm open to alternative suggestions. I haven't looked at the implementation. Could the scaffolding could be factored into a separate utility class, and each testcase given a more descriptive name/implementation? Whatever form this ultimately takes, we should disable this suite of tests. As in HDFS-12349, failing unit tests are now expected. I've stopped even looking at the causes for TestDFSStripedOutputStreamWithFailure. Instead of adding {{\@Ignore}} annotations to all its subclasses, would you mind if we committed this and HDFS-12417? > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600.002.patch, HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161888#comment-16161888 ] Andrew Wang commented on HDFS-11600: One possible issue with combining these into the same class is that we're likely to hit the Surefire timeout for a test class. I had to split up an existing EC test in HDFS-12377 for this reason. The proliferation of test classes still sucks from a code cleanliness POV, so I'm open to alternative suggestions. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600.002.patch, HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161842#comment-16161842 ] Chris Douglas commented on HDFS-11600: -- Attached a naive rebase of the original patch. [~arpitagarwal], could you take a look? > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600.002.patch, HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16161576#comment-16161576 ] Hadoop QA commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s{color} | {color:red} HDFS-11600 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12866623/HDFS-11600-1.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/21071/console | | Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Refactor TestDFSStripedOutputStreamWithFailure test classes > --- > > Key: HDFS-11600 > URL: https://issues.apache.org/jira/browse/HDFS-11600 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Affects Versions: 3.0.0-alpha2 >Reporter: Andrew Wang >Assignee: Elek, Marton >Priority: Minor > Attachments: HDFS-11600-1.patch > > > TestDFSStripedOutputStreamWithFailure has a great number of subclasses. The > tests are parameterized based on the name of these subclasses. > Seems like we could parameterize these tests with JUnit and then not need all > these separate test classes. > Another note, the tests will randomly return instead of running the test. > Using {{Assume}} instead would make it more clear in the test output that > these tests were skipped. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998496#comment-15998496 ] Hadoop QA commented on HDFS-11600: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 23 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 43s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 48s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 72m 15s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}107m 47s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.server.namenode.TestFileTruncate | | | hadoop.hdfs.TestMaintenanceState | | | hadoop.hdfs.server.namenode.TestDecommissioningStatus | | | hadoop.hdfs.TestDFSXORStripedOutputStreamWithFailure | | | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.TestDFSRSDefault10x4StripedOutputStreamWithFailure | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | HDFS-11600 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12866623/HDFS-11600-1.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 109a35d2d76a 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 4e6bbd0 | | Default Java | 1.8.0_121 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-HDFS-Build/19331/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/19331/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/19331/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/19331/console | | Powered by |