[GitHub] Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console
Tibor17 commented on a change in pull request #112: Adding support for
externally passed random seed and printing used seed on console
URL: https://github.com/apache/maven-surefire/pull/112#discussion_r212600811
##
File path:
surefire-its/src/test/resources/testng-parallel-suites/src/test/java/testng/suiteXml/TestNGSuiteTest.java
##
@@ -28,16 +28,23 @@
* @author mailto:tibordig...@apache.org;>Tibor Digana (tibor17)
* @since 2.19
*/
-public class TestNGSuiteTest {
- private static final AtomicInteger counter = new AtomicInteger();
+public class TestNGSuiteTest
+{
+private static final AtomicInteger counter = new AtomicInteger();
- @Test
- public void shouldRunAndPrintItself()
- throws Exception
- {
- System.out.println( getClass().getSimpleName() +
"#shouldRunAndPrintItself() "
- +
counter.incrementAndGet() + ".");
+@Test
+public void shouldRunAndPrintItself() throws Exception
+{
+String message = String.format(
+"%s#shouldRunAndPrintItself() %d.",
+getClass().getSimpleName(),
+counter.incrementAndGet()
+);
+TimeUnit.SECONDS.sleep( 1 );
- TimeUnit.SECONDS.sleep( 2 );
- }
+synchronized ( System.out )
Review comment:
@cardil
Thx for pointing out the synchronization issue with `System.out`.
I have noticed the same problem yesterday, see
https://github.com/cbeust/testng/issues/1908
Treating the stream in test would fix our builds but we still have this
issue on Ubuntu and not on Windows. Does it mean that the Sysout should be
synchronized in the application run? We are setting the delegate via
`System.setOut()` and the delegate in `ForkingRunListener` is synchronized. Why
we do not see same issue in JUnit tests?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console
Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console URL: https://github.com/apache/maven-surefire/pull/112#discussion_r189106324 ## File path: maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/CommonReflectorTest.java ## @@ -0,0 +1,103 @@ +package org.apache.maven.plugin.surefire; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import java.io.File; +import java.io.IOException; + +import junit.framework.TestCase; +import org.apache.maven.plugin.surefire.report.DefaultReporterFactory; +import org.apache.maven.shared.utils.io.FileUtils; +import org.apache.maven.surefire.testset.RunOrderParameters; +import org.apache.maven.surefire.util.Randomizer; +import org.apache.maven.surefire.util.RunOrder; + +/** + * @author Krzysztof Suszyński + * @since 2016-04-03 + */ +public class CommonReflectorTest extends TestCase Review comment: Have you registered every new unit test in `JUnit4SuiteTest.java`? If you have not, please do it and run the build `mvn install`. The integration test may take one hour to complete. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console
Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console URL: https://github.com/apache/maven-surefire/pull/112#discussion_r189106324 ## File path: maven-surefire-common/src/test/java/org/apache/maven/plugin/surefire/CommonReflectorTest.java ## @@ -0,0 +1,103 @@ +package org.apache.maven.plugin.surefire; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import java.io.File; +import java.io.IOException; + +import junit.framework.TestCase; +import org.apache.maven.plugin.surefire.report.DefaultReporterFactory; +import org.apache.maven.shared.utils.io.FileUtils; +import org.apache.maven.surefire.testset.RunOrderParameters; +import org.apache.maven.surefire.util.Randomizer; +import org.apache.maven.surefire.util.RunOrder; + +/** + * @author Krzysztof Suszyński + * @since 2016-04-03 + */ +public class CommonReflectorTest extends TestCase Review comment: Have you registered every new unit test in `JUnit4SuiteTest.java`? If you have not, please do it and run the build `mvn install`. The integration test may take obe hour to complete. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console
Tibor17 commented on a change in pull request #112: Adding support for externally passed random seed and printing used seed on console URL: https://github.com/apache/maven-surefire/pull/112#discussion_r189099902 ## File path: maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java ## @@ -283,6 +288,22 @@ @Parameter( property = "failsafe.runOrder", defaultValue = "filesystem" ) private String runOrder; +/** + * When using a random order with runOrder parameter it is useful to be able to pass a seed. + * + * + * This is because, when tests fails to execution order binding, one can use the seed printed in + * reports to execute tests with exact same order as they where when they failed. This effectively reproduce + * the error. To reproduce a given seed most useful is to use command line option: + * + * + * -Dfailsafe.randomSeed=325119 + * + * @since 2.19.2 + */ +@Parameter( property = "failsafe.randomSeed", defaultValue = Randomizer.DEFAULT_SEED ) +private String randomSeed; Review comment: Can we include this in the already existing parameter `runOrder`? For instance, the usage can be `-Dfailsafe.runOrder=random:325119`. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
