[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412838#comment-17412838 ] Steve Loughran commented on HADOOP-14693: - (it's not junit5; its hadoop-azure surefire set to run different test methods in parallel *and out of order*) > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412820#comment-17412820 ] Steve Loughran commented on HADOOP-14693: - I'm having fun with my tests in MAPREDUCE-7341 & parallel surefire running. In the hadoop-azure parallel run, the junit4 @FixMethodOrder(MethodSorters.NAME_ASCENDING) attribute is not working; methods are running out of order. Looks like the mechanism has changed https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-execution-order and, presumably, it means somehow sequential vs parallel runs work differently. Will investigate further. If JUnit5 have gone and changed something else I shall be deeply disappointed. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404429#comment-17404429 ] Akira Ajisaka commented on HADOOP-14693: If we are going to create JUnit 5 based KerberosSecurityTestCase, I think it should be placed in the test directory. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404427#comment-17404427 ] Akira Ajisaka commented on HADOOP-14693: Hi [~smeng] I appreciated that you are trying to upgrade JUnit in [https://github.com/apache/hadoop/pull/3316] I tried the plugin in some other modules and found a problem: * KerberosSecurityTestCase in hadoop-minikdc module has JUnit 4 dependency in compile (not test) scope, and it is used from many modules. Therefore we need to upgrade all the unit tests that extend KerberosSecurityTestCase at once. To mitigate the situation, we can create JUnit 5 based KerberosSecurityTestCase so that we don't need to upgrade the unit tests at the same time. There are many other test utils, and most of them have the same problem. Hi [~ste...@apache.org] I'm +1 for gradually moving to AssertJ. The error message in AssertJ is more verbose and it has many useful methods. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402174#comment-17402174 ] Steve Loughran commented on HADOOP-14693: - There's an IntelliJ article on the topic too https://blog.jetbrains.com/idea/2020/08/migrating-from-junit-4-to-junit-5/ One big PITA is that the Assertion classes have changed. I think it's a real shame that this has changed as it is generating needless work. Similarly, @Ignore attributes now need to move to @Disabled. Again, I think this is "unfortunate" Even with the automated migration and IDE support this is going to make back porting really painful. Normally all test suites which avoid Mockito go back to any of version without problems; now we (I!) are going to have to change them, recompile, and rerun. I note that we have alternatives to some of the JUnit methods which will backport seamlessly * AssertJ assertions * {{ContractTestUtils.skip(message)}} over {{@Ignore}} * LambdaTestUtils.intercept over @Expected (also, it provides better diagnostics with the toString() of what the l-expression was, etc etc.) It's makes me think: can we enforce the use of those over the JUnit5 versions? Or at least have checkstyle recommending them. AssertJ is a bit verbose for simple asserts, and I would not advocate replacing all existing assertTrue, assertEquals etc with them, but we could # warn on new code with the classic JUnit asserts # warn on @Disabled, @Expected # reject hamcrest asserts entirely Given that AssertJ and intercept are, IMO, better, we can justify this on test Qquality rather than just ease of back porting. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399186#comment-17399186 ] Akira Ajisaka commented on HADOOP-14693: Thank you [~smeng]. I tried the plugin and it worked: https://github.com/apache/hadoop/pull/3304 Though the number of whitespaces for tab is not correct, I think we can save a lot of time with the plugin. Also, I think we can pass our checkstyle rule to the plugin to fix the whitespace issue. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399121#comment-17399121 ] Siyao Meng commented on HADOOP-14693: - fyi I just stumbled into this tool called OpenRewrite. apparently it can seemingly automate some code refactoring including migration from junit 4 -> 5 and others. might be helpful? https://docs.openrewrite.org/tutorials/migrate-from-junit-4-to-junit-5 > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890689#comment-16890689 ] Akira Ajisaka commented on HADOOP-14693: Global timeout is supported in JUnit 5.5.0. Filed HADOOP-16447 to upgrade JUnit5. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695934#comment-16695934 ] Steve Loughran commented on HADOOP-14693: - Global timeout control is a critical part of long-lived tests: we want to stop jenkins runs hanging. I put a lot of work into the S3A scale tests to make those timeouts configurable from the CLI, so a 4GB test run could have a longer timeout than a 32MB dataset, and {{org.apache.hadoop.test.HadoopTestBase}} tries to provide the same for all its subclasses. Looking at the github issue, it looks like we just need to write and ship with our own timing extension, which we can sneak into hadoop-common test for all our tests to pick up > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695205#comment-16695205 ] Ewan Higgs commented on HADOOP-14693: - In various tests (e.g. TestECSchema) we have {{@Rule Timeout globalTimeout = new Timeout(3000);}} which means the entire test and all child tests have a timeout set. Junit5 doesn't have this (yet). Discussion is here: https://github.com/junit-team/junit5/issues/80 I'm not sure if this is a blocker for anyone. If the tests behave well enough then maybe we can remove the timeouts? > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662009#comment-16662009 ] Takanobu Asanuma commented on HADOOP-14693: --- HADOOP-14775 resolved. Now we can start working on this. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16378421#comment-16378421 ] Akira Ajisaka commented on HADOOP-14693: Hi [~ste...@apache.org], attached a patch in HADOOP-14775 to run junit4 tests by junit-vintage-engine. You can start playing with it. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16377168#comment-16377168 ] Steve Loughran commented on HADOOP-14693: - How ready are we to start playing with this? > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128286#comment-16128286 ] Akira Ajisaka commented on HADOOP-14693: +1 for the option 2. Thanks. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16126796#comment-16126796 ] Ajay Kumar commented on HADOOP-14693: - [~steve_l] agree!! will link jira to upgrade the junit dependency in parent pom with junit-vintage-engine. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120335#comment-16120335 ] Steve Loughran commented on HADOOP-14693: - Option 2, incremental. it's not just the smaller changes, but we know that some tests (and I'm thinking of all the FS contract tests) are used downstream as part of the code to verify their filesystems are consistent with what Hadoop expects. We shouldn't break things unless we need to. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16120328#comment-16120328 ] Ajay Yadav commented on HADOOP-14693: - [~ajisakaa],[~andrew.wang] , As i understand , we can do it in two ways. 1. Update the junit dependency in hadoop-main to junit5 with junit-jupiter-engine. which will require changes in most of the test cases to move them to new junit5 api and platform. (More riskier) 2. Update the junit dependency in hadoop-main to junit5 while maintaining backward compatibility to test cases built in junit4 using (junit-vintage-engine). As a next step we can create new test cases using junit 5 api and move old test cases to junit5 in steps. This will be incremental change with less risk to breaking old test cases. Any ideas, suggestions on this? > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16112355#comment-16112355 ] Akira Ajisaka commented on HADOOP-14693: Yes. We need to finish the migration from 3 to 4 first. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-14693) Upgrade JUnit from 4 to 5
[ https://issues.apache.org/jira/browse/HADOOP-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108031#comment-16108031 ] Andrew Wang commented on HADOOP-14693: -- Bold JIRA :) We haven't even finished our migration from JUnit 3 to 4. > Upgrade JUnit from 4 to 5 > - > > Key: HADOOP-14693 > URL: https://issues.apache.org/jira/browse/HADOOP-14693 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka > > JUnit 4 does not support Java 9. We need to upgrade this. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org