I'm ok with "flacky tests" vs "flacked tests" I'm ok that "tests in error" is not the best description, but "erroneous tests" does not convince me...
If I look at how JUnit itself report issues, which is IMHO the origin of this description, JUnit 3 made the distinction between "failures" and "errors" What about just removing "tests" and get a consistent descripton in all cases? Regards, Hervé Le jeudi 22 décembre 2016, 23:13:11 CET [email protected] a écrit : > Repository: maven-surefire > Updated Branches: > refs/heads/master 304f454ed -> dcd3b64ac > > > [SUREFIRE-1315] Fix stylistic errors in DefaultReporterFactory > > * Update labels for proper English style > * Use proper title case > > > Project: http://git-wip-us.apache.org/repos/asf/maven-surefire/repo > Commit: > http://git-wip-us.apache.org/repos/asf/maven-surefire/commit/dcd3b64a Tree: > http://git-wip-us.apache.org/repos/asf/maven-surefire/tree/dcd3b64a Diff: > http://git-wip-us.apache.org/repos/asf/maven-surefire/diff/dcd3b64a > > Branch: refs/heads/master > Commit: dcd3b64acf129e630c6aaec9b2e1f48476855bad > Parents: 304f454 > Author: Michael Osipov <[email protected]> > Authored: Fri Dec 23 00:11:25 2016 +0100 > Committer: Michael Osipov <[email protected]> > Committed: Fri Dec 23 00:11:25 2016 +0100 > > ---------------------------------------------------------------------- > .../plugin/surefire/report/DefaultReporterFactory.java | 6 +++--- > .../surefire/report/DefaultReporterFactoryTest.java | 4 ++-- > .../src/site/markdown/newerrorsummary.md | 4 ++-- > .../maven/surefire/its/JUnit4RerunFailingTestsIT.java | 10 +++++----- > .../jiras/Surefire1146RerunFailedAndParameterized.java | 2 +- > 5 files changed, 13 insertions(+), 13 deletions(-) > ---------------------------------------------------------------------- > > > http://git-wip-us.apache.org/repos/asf/maven-surefire/blob/dcd3b64a/maven-su > refire-common/src/main/java/org/apache/maven/plugin/surefire/report/DefaultR > eporterFactory.java > ---------------------------------------------------------------------- diff > --git > a/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactory.java > b/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactory.java index 2332856..24f5b3b 100644 > --- > a/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactory.java +++ > b/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactory.java @@ -421,9 +421,9 @@ public class > DefaultReporterFactory > enum TestResultType > { > > - error( "Tests in error: " ), > - failure( "Failed tests: " ), > - flake( "Flaked tests: " ), > + error( "Erroneous Tests: " ), > + failure( "Failed Tests: " ), > + flake( "Flaky Tests: " ), > success( "Success: " ), > skipped( "Skipped: " ), > unknown( "Unknown: " ); > > http://git-wip-us.apache.org/repos/asf/maven-surefire/blob/dcd3b64a/maven-su > refire-common/src/test/java/org/apache/maven/plugin/surefire/report/DefaultR > eporterFactoryTest.java > ---------------------------------------------------------------------- diff > --git > a/maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactoryTest.java > b/maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactoryTest.java index e51c993..98f97bd 100644 > --- > a/maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactoryTest.java +++ > b/maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/repo > rt/DefaultReporterFactoryTest.java @@ -120,7 +120,7 @@ public class > DefaultReporterFactoryTest > // Now test the result will be printed out correctly > factory.printTestFailures( flake ); > String[] expectedFlakeOutput = > - { "Flaked tests: ", TEST_FOUR, " Run 1: " + ASSERTION_FAIL, " > Run 2: PASS", "", TEST_ONE, + { "Flaky Tests: ", TEST_FOUR, " > Run 1: " + ASSERTION_FAIL, " Run 2: PASS", "", TEST_ONE, " Run 1: " + > ERROR, " Run 2: " + ASSERTION_FAIL, " Run 3: PASS", "", TEST_TWO, " Run > 1: " + ERROR, " Run 2: PASS", "" }; > assertEquals( asList( expectedFlakeOutput ), reporter.getMessages() > ); @@ -128,7 +128,7 @@ public class DefaultReporterFactoryTest > reporter.reset(); > factory.printTestFailures( error ); > String[] expectedFailureOutput = > - { "Tests in error: ", TEST_THREE, " Run 1: " + ASSERTION_FAIL, > " Run 2: " + ERROR, " Run 3: " + ERROR, "" + { "Erroneous > Tests: ", TEST_THREE, " Run 1: " + ASSERTION_FAIL, " Run 2: " + ERROR, " > Run 3: " + ERROR, "" }; > assertEquals( asList( expectedFailureOutput ), > reporter.getMessages() ); > > > http://git-wip-us.apache.org/repos/asf/maven-surefire/blob/dcd3b64a/maven-su > refire-plugin/src/site/markdown/newerrorsummary.md > ---------------------------------------------------------------------- diff > --git a/maven-surefire-plugin/src/site/markdown/newerrorsummary.md > b/maven-surefire-plugin/src/site/markdown/newerrorsummary.md index > 7467aa9..09537e3 100644 > --- a/maven-surefire-plugin/src/site/markdown/newerrorsummary.md > +++ b/maven-surefire-plugin/src/site/markdown/newerrorsummary.md > @@ -26,11 +26,11 @@ report of the run or the files on disk. > > ### Example output: > > - Failed tests: > + Failed Tests: > Test1.assertion1:59 Bending maths expected:<[123]> but was:<[312]> > Test1.assertion2:64 True is false > > - Tests in error: > + Erroneous Tests: > Test1.nullPointerInLibrary:38 �� NullPointer > Test1.failInMethod:43->innerFailure:68 NullPointer Fail here > Test1.failInLibInMethod:48 �� NullPointer > > http://git-wip-us.apache.org/repos/asf/maven-surefire/blob/dcd3b64a/surefire > -integration-tests/src/test/java/org/apache/maven/surefire/its/JUnit4RerunFa > ilingTestsIT.java > ---------------------------------------------------------------------- diff > --git > a/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/JU > nit4RerunFailingTestsIT.java > b/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/JU > nit4RerunFailingTestsIT.java index c6c6b8f..f216e08 100644 > --- > a/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/JU > nit4RerunFailingTestsIT.java +++ > b/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/JU > nit4RerunFailingTestsIT.java @@ -216,11 +216,11 @@ public class > JUnit4RerunFailingTestsIT > private void verifyFailuresOneRetry( OutputValidator outputValidator, > int run, int failures, int errors, int flakes ) > { > - outputValidator.verifyTextInLog( "Failed tests" ); > + outputValidator.verifyTextInLog( "Failed Tests:" ); > outputValidator.verifyTextInLog( "Run 1: > FlakyFirstTimeTest.testFailingTestOne" ); outputValidator.verifyTextInLog( > "Run 2: FlakyFirstTimeTest.testFailingTestOne" ); > > - outputValidator.verifyTextInLog( "Tests in error" ); > + outputValidator.verifyTextInLog( "Erroneous Tests:" ); > outputValidator.verifyTextInLog( "Run 1: > FlakyFirstTimeTest.testErrorTestOne" ); outputValidator.verifyTextInLog( > "Run 2: FlakyFirstTimeTest.testErrorTestOne" ); > > @@ -230,7 +230,7 @@ public class JUnit4RerunFailingTestsIT > private void verifyOnlyFailuresOneRetry( OutputValidator > outputValidator, int run, int failures, int errors, int flakes ) > { > - outputValidator.verifyTextInLog( "Failed tests" ); > + outputValidator.verifyTextInLog( "Failed Tests:" ); > outputValidator.verifyTextInLog( "Run 1: > FlakyFirstTimeTest.testFailingTestOne" ); outputValidator.verifyTextInLog( > "Run 2: FlakyFirstTimeTest.testFailingTestOne" ); > > @@ -240,7 +240,7 @@ public class JUnit4RerunFailingTestsIT > private void verifyFailuresTwoRetry( OutputValidator outputValidator, > int run, int failures, int errors, int flakes ) > { > - outputValidator.verifyTextInLog( "Flaked tests" ); > + outputValidator.verifyTextInLog( "Flaky Tests:" ); > outputValidator.verifyTextInLog( "Run 1: > FlakyFirstTimeTest.testFailingTestOne" ); outputValidator.verifyTextInLog( > "Run 2: FlakyFirstTimeTest.testFailingTestOne" ); > outputValidator.verifyTextInLog( "Run 3: PASS" ); > @@ -253,7 +253,7 @@ public class JUnit4RerunFailingTestsIT > > private void verifyFailuresNoRetry( OutputValidator outputValidator, > int run, int failures, int errors, int flakes ) { > - outputValidator.verifyTextInLog( "Failed tests" ); > + outputValidator.verifyTextInLog( "Failed Tests:" ); > outputValidator.verifyTextInLog( > "testFailingTestOne(junit4.FlakyFirstTimeTest)" ); > outputValidator.verifyTextInLog( "ERROR" ); > outputValidator.verifyTextInLog( > "testErrorTestOne(junit4.FlakyFirstTimeTest)" ); > > http://git-wip-us.apache.org/repos/asf/maven-surefire/blob/dcd3b64a/surefire > -integration-tests/src/test/java/org/apache/maven/surefire/its/jiras/Surefir > e1146RerunFailedAndParameterized.java > ---------------------------------------------------------------------- diff > --git > a/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/ji > ras/Surefire1146RerunFailedAndParameterized.java > b/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/ji > ras/Surefire1146RerunFailedAndParameterized.java index 24d39bc..8ebc6ec > 100644 > --- > a/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/ji > ras/Surefire1146RerunFailedAndParameterized.java +++ > b/surefire-integration-tests/src/test/java/org/apache/maven/surefire/its/ji > ras/Surefire1146RerunFailedAndParameterized.java @@ -39,7 +39,7 @@ public > class Surefire1146RerunFailedAndParameterized > > private void verify( OutputValidator outputValidator, int run, int > failures, int errors, int skipped, int flakes ) { > - outputValidator.verifyTextInLog( "Flaked tests:" ); > + outputValidator.verifyTextInLog( "Flaky Tests:" ); > outputValidator.verifyTextInLog( > "jiras.surefire1146.CustomDescriptionParameterizedTest.flakyTest[0: > (Test11); Test12; > Test13;](jiras.surefire1146.CustomDescriptionParameterizedTest)" ); > outputValidator.verifyTextInLog( "Run 1: > CustomDescriptionParameterizedTest.flakyTest:" ); > outputValidator.verifyTextInLog( "Run 2: > CustomDescriptionParameterizedTest.flakyTest:" ); --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
