Please specify '-DskipTests' on commandline. Cheers
On Dec 5, 2014, at 3:52 AM, Emre Sevinc <emre.sev...@gmail.com> wrote: > Hello, > > I'm currently developing a Spark Streaming application and trying to write my > first unit test. I've used Java for this application, and I also need use > Java (and JUnit) for writing unit tests. > > I could not find any documentation that focuses on Spark Streaming unit > testing, all I could find was the Java based unit tests in Spark Streaming > source code: > > > https://github.com/apache/spark/blob/branch-1.1/streaming/src/test/java/org/apache/spark/streaming/JavaAPISuite.java > > that depends on a Scala file: > > > https://github.com/apache/spark/blob/branch-1.1/streaming/src/test/java/org/apache/spark/streaming/JavaTestUtils.scala > > which, in turn, depends on the Scala test files in > > > https://github.com/apache/spark/tree/branch-1.1/streaming/src/test/scala/org/apache/spark/streaming > > So I thought that I could grab the Spark source code, switch to branch-1.1 > branch and then only compile 'core' and 'streaming' modules, hopefully ending > up with the compiled classes (or jar files) of the Streaming test utilities, > so that I can import them in my Java based Spark Streaming application. > > However, trying to build it via the following command line failed: > > mvn -pl core,streaming package > > You can see the full output at the end of this message. > > Any ideas how to progress? > > Full output of the build: > > emre@emre-ubuntu:~/code/spark$ mvn -pl core,streaming package > [INFO] Scanning for projects... > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Build Order: > [INFO] > [INFO] Spark Project Core > [INFO] Spark Project Streaming > [INFO] > > [INFO] > ------------------------------------------------------------------------ > [INFO] Building Spark Project Core 1.1.2-SNAPSHOT > [INFO] > ------------------------------------------------------------------------ > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.pom > (5 KB at 5.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.jar > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/plugins/maven-antrun-plugin/1.7/maven-antrun-plugin-1.7.jar > (31 KB at 200.4 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.pom > (24 KB at 178.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.pom > Downloaded: > https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.pom > (3 KB at 22.5 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/scala-lang/scalap/2.10.4/scalap-2.10.4.pom > Downloaded: > https://repo1.maven.org/maven2/org/scala-lang/scalap/2.10.4/scalap-2.10.4.pom > (2 KB at 19.2 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/derby/derby/10.4.2.0/derby-10.4.2.0.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/derby/derby/10.4.2.0/derby-10.4.2.0.pom > (2 KB at 14.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/mockito/mockito-all/1.9.0/mockito-all-1.9.0.pom > Downloaded: > https://repo1.maven.org/maven2/org/mockito/mockito-all/1.9.0/mockito-all-1.9.0.pom > (1010 B at 4.1 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/easymock/easymockclassextension/3.1/easymockclassextension-3.1.pom > Downloaded: > https://repo1.maven.org/maven2/org/easymock/easymockclassextension/3.1/easymockclassextension-3.1.pom > (5 KB at 42.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/easymock/easymock-parent/3.1/easymock-parent-3.1.pom > Downloaded: > https://repo1.maven.org/maven2/org/easymock/easymock-parent/3.1/easymock-parent-3.1.pom > (13 KB at 133.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/easymock/easymock/3.1/easymock-3.1.pom > Downloaded: > https://repo1.maven.org/maven2/org/easymock/easymock/3.1/easymock-3.1.pom (6 > KB at 38.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/cglib/cglib-nodep/2.2.2/cglib-nodep-2.2.2.pom > Downloaded: > https://repo1.maven.org/maven2/cglib/cglib-nodep/2.2.2/cglib-nodep-2.2.2.pom > (2 KB at 9.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.jar > Downloading: > https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.jar > Downloading: > https://repo1.maven.org/maven2/org/scala-lang/scalap/2.10.4/scalap-2.10.4.jar > Downloading: > https://repo1.maven.org/maven2/org/apache/derby/derby/10.4.2.0/derby-10.4.2.0.jar > Downloading: > https://repo1.maven.org/maven2/org/mockito/mockito-all/1.9.0/mockito-all-1.9.0.jar > Downloaded: > https://repo1.maven.org/maven2/org/scala-lang/scalap/2.10.4/scalap-2.10.4.jar > (834 KB at 146.7 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/easymock/easymockclassextension/3.1/easymockclassextension-3.1.jar > Downloaded: > https://repo1.maven.org/maven2/org/easymock/easymockclassextension/3.1/easymockclassextension-3.1.jar > (11 KB at 65.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/easymock/easymock/3.1/easymock-3.1.jar > Downloaded: > https://repo1.maven.org/maven2/org/easymock/easymock/3.1/easymock-3.1.jar > (110 KB at 125.5 KB/sec) > Downloading: > https://repo1.maven.org/maven2/cglib/cglib-nodep/2.2.2/cglib-nodep-2.2.2.jar > Downloaded: > https://repo1.maven.org/maven2/cglib/cglib-nodep/2.2.2/cglib-nodep-2.2.2.jar > (320 KB at 205.4 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/spark-project/akka/akka-testkit_2.10/2.2.3-shaded-protobuf/akka-testkit_2.10-2.2.3-shaded-protobuf.jar > (233 KB at 23.9 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/commons/commons-math3/3.3/commons-math3-3.3.jar > (1907 KB at 192.1 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/mockito/mockito-all/1.9.0/mockito-all-1.9.0.jar > (1461 KB at 113.2 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/derby/derby/10.4.2.0/derby-10.4.2.0.jar > (2390 KB at 131.1 KB/sec) > [INFO] > [INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-versions) @ > spark-core_2.10 --- > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-api/1.3.1/enforcer-api-1.3.1.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-api/1.3.1/enforcer-api-1.3.1.pom > (3 KB at 34.0 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-rules/1.3.1/enforcer-rules-1.3.1.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-rules/1.3.1/enforcer-rules-1.3.1.pom > (4 KB at 66.6 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-api/1.3.1/enforcer-api-1.3.1.jar > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-rules/1.3.1/enforcer-rules-1.3.1.jar > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-api/1.3.1/enforcer-api-1.3.1.jar > (10 KB at 75.7 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/enforcer/enforcer-rules/1.3.1/enforcer-rules-1.3.1.jar > (86 KB at 292.3 KB/sec) > [INFO] > [INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) @ > spark-core_2.10 --- > [INFO] Source directory: /home/emre/code/spark/core/src/main/scala added. > [INFO] > [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ > spark-core_2.10 --- > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/shared/maven-artifact-resolver/1.0/maven-artifact-resolver-1.0.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/shared/maven-artifact-resolver/1.0/maven-artifact-resolver-1.0.pom > (6 KB at 65.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-resources/1.0-alpha-7/plexus-resources-1.0-alpha-7.pom > Downloaded: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-resources/1.0-alpha-7/plexus-resources-1.0-alpha-7.pom > (2 KB at 21.4 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/velocity/velocity/1.7/velocity-1.7.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/velocity/velocity/1.7/velocity-1.7.pom > (11 KB at 143.3 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/shared/maven-artifact-resolver/1.0/maven-artifact-resolver-1.0.jar > Downloading: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interpolation/1.12/plexus-interpolation-1.12.jar > Downloading: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-resources/1.0-alpha-7/plexus-resources-1.0-alpha-7.jar > Downloading: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/1.5.15/plexus-utils-1.5.15.jar > Downloading: > https://repo1.maven.org/maven2/org/apache/velocity/velocity/1.7/velocity-1.7.jar > Dec 05, 2014 12:30:22 PM > org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec > execute > INFO: I/O exception > (org.apache.maven.wagon.providers.http.httpclient.NoHttpResponseException) > caught when processing request: The target server failed to respond > Dec 05, 2014 12:30:22 PM > org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec > execute > INFO: Retrying request > Downloaded: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-resources/1.0-alpha-7/plexus-resources-1.0-alpha-7.jar > (23 KB at 45.6 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/shared/maven-artifact-resolver/1.0/maven-artifact-resolver-1.0.jar > (14 KB at 13.3 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-utils/1.5.15/plexus-utils-1.5.15.jar > (223 KB at 122.9 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interpolation/1.12/plexus-interpolation-1.12.jar > (60 KB at 29.6 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/velocity/velocity/1.7/velocity-1.7.jar > (439 KB at 159.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/apache-jar-resource-bundle/1.4/apache-jar-resource-bundle-1.4.jar > Downloaded: > https://repo1.maven.org/maven2/org/apache/apache-jar-resource-bundle/1.4/apache-jar-resource-bundle-1.4.jar > (13 KB at 98.6 KB/sec) > [INFO] > [INFO] --- maven-antrun-plugin:1.7:run (default) @ spark-core_2.10 --- > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-plugin-api/2.0.11/maven-plugin-api-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-plugin-api/2.0.11/maven-plugin-api-2.0.11.pom > (2 KB at 19.6 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven/2.0.11/maven-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven/2.0.11/maven-2.0.11.pom > (22 KB at 187.1 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-parent/12/maven-parent-12.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-parent/12/maven-parent-12.pom > (22 KB at 58.0 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-project/2.0.11/maven-project-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-project/2.0.11/maven-project-2.0.11.pom > (3 KB at 46.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-settings/2.0.11/maven-settings-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-settings/2.0.11/maven-settings-2.0.11.pom > (3 KB at 40.2 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-model/2.0.11/maven-model-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-model/2.0.11/maven-model-2.0.11.pom > (4 KB at 59.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interpolation/1.1/plexus-interpolation-1.1.pom > Downloaded: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interpolation/1.1/plexus-interpolation-1.1.pom > (2 KB at 26.5 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-profile/2.0.11/maven-profile-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-profile/2.0.11/maven-profile-2.0.11.pom > (2 KB at 31.7 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-artifact-manager/2.0.11/maven-artifact-manager-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-artifact-manager/2.0.11/maven-artifact-manager-2.0.11.pom > (3 KB at 10.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-repository-metadata/2.0.11/maven-repository-metadata-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-repository-metadata/2.0.11/maven-repository-metadata-2.0.11.pom > (3 KB at 27.2 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-artifact/2.0.11/maven-artifact-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-artifact/2.0.11/maven-artifact-2.0.11.pom > (2 KB at 22.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/maven-plugin-registry/2.0.11/maven-plugin-registry-2.0.11.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/maven-plugin-registry/2.0.11/maven-plugin-registry-2.0.11.pom > (2 KB at 30.9 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/ant/ant/1.8.2/ant-1.8.2.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/ant/ant/1.8.2/ant-1.8.2.pom (10 KB > at 100.5 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/ant/ant-parent/1.8.2/ant-parent-1.8.2.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/ant/ant-parent/1.8.2/ant-parent-1.8.2.pom > (5 KB at 84.8 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.8.2/ant-launcher-1.8.2.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.8.2/ant-launcher-1.8.2.pom > (3 KB at 43.0 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/ant/ant/1.8.2/ant-1.8.2.jar > Downloading: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interpolation/1.1/plexus-interpolation-1.1.jar > Downloading: > https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.8.2/ant-launcher-1.8.2.jar > Downloaded: > https://repo1.maven.org/maven2/org/apache/ant/ant-launcher/1.8.2/ant-launcher-1.8.2.jar > (13 KB at 138.3 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/codehaus/plexus/plexus-interpolation/1.1/plexus-interpolation-1.1.jar > (35 KB at 256.7 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/ant/ant/1.8.2/ant-1.8.2.jar (1889 > KB at 416.0 KB/sec) > [WARNING] Parameter tasks is deprecated, use target instead > [INFO] Executing tasks > > main: > [unzip] Expanding: /home/emre/code/spark/python/lib/py4j-0.8.2.1-src.zip > into /home/emre/code/spark/python/build > [INFO] Executed tasks > [INFO] > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > spark-core_2.10 --- > [INFO] Using 'UTF-8' encoding to copy filtered resources. > [INFO] Copying 9 resources > [INFO] Copying 21 resources > [INFO] Copying 7 resources > [INFO] Copying 3 resources > [INFO] > [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @ > spark-core_2.10 --- > [WARNING] Zinc server is not available at port 3030 - reverting to normal > incremental compile > [INFO] Using incremental compilation > [INFO] 'compiler-interface' not yet compiled for Scala 2.10.4. Compiling... > [INFO] Compilation completed in 10.873 s > [INFO] compiler plugin: > BasicArtifact(org.scalamacros,paradise_2.10.4,2.0.1,null) > Downloading: > https://repo1.maven.org/maven2/org/scalamacros/paradise_2.10.4/2.0.1/paradise_2.10.4-2.0.1.jar > Downloaded: > https://repo1.maven.org/maven2/org/scalamacros/paradise_2.10.4/2.0.1/paradise_2.10.4-2.0.1.jar > (1855 KB at 485.7 KB/sec) > [INFO] Compiling 390 Scala sources and 28 Java sources to > /home/emre/code/spark/core/target/scala-2.10/classes... > [WARNING] warning: [options] bootstrap class path not set in conjunction with > -source 1.6 > [WARNING] 1 warning > [INFO] > [INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ > spark-core_2.10 --- > [INFO] Changes detected - recompiling the module! > [INFO] Compiling 28 source files to > /home/emre/code/spark/core/target/scala-2.10/classes > [INFO] > [INFO] --- build-helper-maven-plugin:1.8:add-test-source > (add-scala-test-sources) @ spark-core_2.10 --- > [INFO] Test Source directory: /home/emre/code/spark/core/src/test/scala added. > [INFO] > [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ > spark-core_2.10 --- > [INFO] Using 'UTF-8' encoding to copy filtered resources. > [INFO] Copying 4 resources > [INFO] Copying 3 resources > [INFO] > [INFO] --- scala-maven-plugin:3.2.0:testCompile (scala-test-compile-first) @ > spark-core_2.10 --- > [WARNING] Zinc server is not available at port 3030 - reverting to normal > incremental compile > [INFO] Using incremental compilation > [INFO] compiler plugin: > BasicArtifact(org.scalamacros,paradise_2.10.4,2.0.1,null) > [INFO] Compiling 111 Scala sources and 3 Java sources to > /home/emre/code/spark/core/target/scala-2.10/test-classes... > [WARNING] warning: [options] bootstrap class path not set in conjunction with > -source 1.6 > [WARNING] Note: > /home/emre/code/spark/core/src/test/java/org/apache/spark/JavaAPISuite.java > uses unchecked or unsafe operations. > [WARNING] Note: Recompile with -Xlint:unchecked for details. > [WARNING] 1 warning > [INFO] > [INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ > spark-core_2.10 --- > [INFO] Changes detected - recompiling the module! > [INFO] Compiling 3 source files to > /home/emre/code/spark/core/target/scala-2.10/test-classes > [INFO] > [INFO] --- maven-surefire-plugin:2.17:test (default-test) @ spark-core_2.10 > --- > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/surefire/maven-surefire-common/2.17/maven-surefire-common-2.17.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/surefire/maven-surefire-common/2.17/maven-surefire-common-2.17.pom > (6 KB at 14.4 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-api/2.17/surefire-api-2.17.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-api/2.17/surefire-api-2.17.pom > (3 KB at 13.3 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-booter/2.17/surefire-booter-2.17.pom > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-booter/2.17/surefire-booter-2.17.pom > (3 KB at 40.1 KB/sec) > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/surefire/maven-surefire-common/2.17/maven-surefire-common-2.17.jar > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-booter/2.17/surefire-booter-2.17.jar > Downloading: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-api/2.17/surefire-api-2.17.jar > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-booter/2.17/surefire-booter-2.17.jar > (39 KB at 184.9 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/surefire/surefire-api/2.17/surefire-api-2.17.jar > (144 KB at 141.3 KB/sec) > Downloaded: > https://repo1.maven.org/maven2/org/apache/maven/surefire/maven-surefire-common/2.17/maven-surefire-common-2.17.jar > (260 KB at 166.2 KB/sec) > [INFO] Tests are skipped. > [INFO] > [INFO] --- scalatest-maven-plugin:1.0-RC2:test (test) @ spark-core_2.10 --- > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; > support was removed in 8.0 > Discovery starting. > Discovery completed in 7 seconds, 167 milliseconds. > Run starting. Expected test count is: 809 > ExternalSorterSuite: > - empty data stream > - few elements per partition > - empty partitions with spilling > - empty partitions with spilling, bypass merge-sort > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > - spilling in local cluster *** FAILED *** > org.apache.spark.SparkException: Job aborted due to stage failure: Master > removed our application: FAILED > at > org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173) > at > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) > at > org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at scala.Option.foreach(Option.scala:236) > at > org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688) > ... > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > - spilling in local cluster with many reduce tasks *** FAILED *** > org.apache.spark.SparkException: Job aborted due to stage failure: Master > removed our application: FAILED > at > org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1185) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1174) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1173) > at > scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59) > at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47) > at > org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1173) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at > org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:688) > at scala.Option.foreach(Option.scala:236) > at > org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:688) > ... > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > - cleanup of intermediate files in sorter > - cleanup of intermediate files in sorter, bypass merge-sort > - cleanup of intermediate files in sorter if there are errors > - cleanup of intermediate files in sorter if there are errors, bypass > merge-sort > - cleanup of intermediate files in shuffle > - cleanup of intermediate files in shuffle with errors > - no partial aggregation or sorting > - partial aggregation without spill > - partial aggregation with spill, no ordering > - partial aggregation with spill, with ordering > - sorting without aggregation, no spill > - sorting without aggregation, with spill > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > ls: cannot access /home/emre/code/spark/assembly/target/scala-2.10: No such > file or directory > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Spark Project Core ................................. FAILURE [03:36 > min] > [INFO] Spark Project Streaming ............................ SKIPPED > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 03:37 min > [INFO] Finished at: 2014-12-05T12:33:36+01:00 > [INFO] Final Memory: 37M/597M > [INFO] > ------------------------------------------------------------------------ > [ERROR] Failed to execute goal > org.scalatest:scalatest-maven-plugin:1.0-RC2:test (test) on project > spark-core_2.10: There are test failures -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException > > -- > Emre Sevinç
