[
https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14238666#comment-14238666
]
Nicholas Chammas commented on SPARK-3431:
-----------------------------------------
OK, here's a patch for {{pom.xml}} that represents my first attempt at having
Maven use Surefire.
{code}
diff --git a/pom.xml b/pom.xml
index b7df53d..78a5b8a 100644
--- a/pom.xml
+++ b/pom.xml
@@ -533,6 +533,12 @@
<version>${scala.version}</version>
</dependency>
<dependency>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <version>2.17</version>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_${scala.binary.version}</artifactId>
<version>2.2.1</version>
@@ -946,15 +952,6 @@
<artifactId>maven-surefire-plugin</artifactId>
<version>2.17</version>
<configuration>
- <!-- Uses scalatest instead -->
- <skipTests>true</skipTests>
- </configuration>
- </plugin>
- <plugin>
- <groupId>org.scalatest</groupId>
- <artifactId>scalatest-maven-plugin</artifactId>
- <version>1.0</version>
- <configuration>
<reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
<junitxml>.</junitxml>
<filereports>SparkTestSuite.txt</filereports>
@@ -969,6 +966,12 @@
<spark.executor.extraClassPath>${test_classpath}</spark.executor.extraClassPath>
<spark.driver.allowMultipleContexts>true</spark.driver.allowMultipleContexts>
</systemProperties>
+ <includes>
+ <include>**/*Suite.java</include>
+ <include>**/*Test.java</include>
+ <include>**/*Suite.scala</include>
+ <include>**/*Test.scala</include>
+ </includes>
</configuration>
<executions>
<execution>
{code}
I'm building and running tests as follows:
{code}
mvn -Pyarn -Phadoop-2.3 -DskipTests -Phive -Phive-thriftserver clean package
mvn -Pyarn -Phadoop-2.3 -Phive -Phive-thriftserver test
{code}
Does that look sensible to you?
The build runs fine, but when I run tests in this way, I get errors in the
{{org.apache.spark.network.shuffle.BlockTransferMessagesSuite}} and a few other
{{network.shuffle}} suites.
> Parallelize execution of tests
> ------------------------------
>
> Key: SPARK-3431
> URL: https://issues.apache.org/jira/browse/SPARK-3431
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Reporter: Nicholas Chammas
> Assignee: Nicholas Chammas
>
> Running all the tests in {{dev/run-tests}} takes up to 2 hours. A common
> strategy to cut test time down is to parallelize the execution of the tests.
> Doing that may in turn require some prerequisite changes to be made to how
> certain tests run.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
