felixcheung commented on issue #24888: [SPARK-28040][SPARK-28070][R] Write type object s3 URL: https://github.com/apache/spark/pull/24888#issuecomment-531638807 Anything in the build log about: [INFO] Spark Project ML Library ........................... FAILURE [01:26 min] ________________________________ From: Michael Chirico <[email protected]> Sent: Sunday, September 15, 2019 8:15 PM To: apache/spark Cc: Felix Cheung; Mention Subject: Re: [apache/spark] [SPARK-28040][SPARK-28070][R] Write type object s3 (#24888) @MichaelChirico commented on this pull request. ________________________________ In R/pkg/R/serialize.R<https://github.com/apache/spark/pull/24888#discussion_r324500771>: > } -writeTime <- function(con, time) {- writeDouble(con, as.double(time))+# Used to pass arrays where all the elements are of the same type+writeArray <- function(arr, con) {+ # TODO: Empty lists are given type "character" right now.+ # This may not work if the Java side expects array of any other type.+ writeType(if (length(arr) > 0L) arr[[1L]] else "somestring", con)+ writeObject(length(arr), con, writeType = FALSE) Hey @felixcheung<https://github.com/felixcheung> have finally sort-of got a Docker image able to build Spark from source to help me iterate locally 🎉 I'm able to build Spark on master with ./build/mvn -DskipTests clean package -X, but on the branch here I get: [INFO] ------------------------------------------------------------------------[INFO] Reactor Summary for Spark Project Parent POM 3.0.0-SNAPSHOT:[INFO] [INFO] Spark Project Parent POM ........................... SUCCESS [04:47 min][INFO] Spark Project Tags ................................. SUCCESS [ 30.888 s][INFO] Spark Project Sketch ............................... SUCCESS [ 4.329 s][INFO] Spark Project Local DB ............................. SUCCESS [ 15.605 s][INFO] Spark Project Networking ........................... SUCCESS [ 14.312 s][INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 5.987 s][INFO] Spark Project Unsafe ............................... SUCCESS [ 11.286 s][INFO] Spark Project Launcher ............................. SUCCESS [ 46.401 s][INFO] Spark Project Core ................................. SUCCESS [02:46 min][INFO] Spark Project ML Local Library ..................... SUCCESS [ 14.346 s][INFO] Spark Project GraphX ............................... SUCCESS [ 12.020 s][INFO] Spark Project Streaming ............................ SUCCESS [ 28.049 s][INFO] Spark Project Catalyst ............................. SUCCESS [01:52 min][INFO] Spark Project SQL .................................. SUCCESS [02:41 min][INFO] Spark Project ML Library ........................... FAILURE [01:26 min][INFO] Spark Project Tools ................................ SKIPPED[INFO] Spark Project Hive ................................. SKIPPED[INFO] Spark Project Graph API ............................ SKIPPED[INFO] Spark Project Cypher ............................... SKIPPED[INFO] Spark Project Graph ................................ SKIPPED[INFO] Spark Project REPL ................................. SKIPPED[INFO] Spark Project Assembly ............................. SKIPPED[INFO] Kafka 0.10+ Token Provider for Streaming ........... SKIPPED[INFO] Spark Integration for Kafka 0.10 ................... SKIPPED[INFO] Kafka 0.10+ Source for Structured Streaming ........ SKIPPED[INFO] Spark Project Examples ............................. SKIPPED[INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED[INFO] Spark Avro ......................................... SKIPPED[INFO] ------------------------------------------------------------------------[INFO] BUILD FAILURE[INFO] ------------------------------------------------------------------------[INFO] Total time: 16:41 min[INFO] Finished at: 2019-09-16T01:47:41Z[INFO] ------------------------------------------------------------------------[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.4.4:testCompile (scala-test-compile-first) on project spark-mllib_2.12: Execution scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.4.4:testCompile failed.: CompileFailed -> [Help 1]org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.4.4:testCompile (scala-test-compile-first) on project spark-mllib_2.12: Execution scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.4.4:testCompile failed. at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:566) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.4.4:testCompile failed. at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:566) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)Caused by: sbt.compiler.CompileFailed at sbt_inc.SbtIncrementalCompiler.zincCompile (SbtIncrementalCompiler.java:146) at sbt_inc.SbtIncrementalCompiler.compile (SbtIncrementalCompiler.java:88) at scala_maven.ScalaCompilerSupport.incrementalCompile (ScalaCompilerSupport.java:320) at scala_maven.ScalaCompilerSupport.compile (ScalaCompilerSupport.java:136) at scala_maven.ScalaCompilerSupport.doExecute (ScalaCompilerSupport.java:116) at scala_maven.ScalaMojoSupport.execute (ScalaMojoSupport.java:574) at scala_maven.ScalaTestCompileMojo.execute (ScalaTestCompileMojo.java:50) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156) at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305) at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192) at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105) at org.apache.maven.cli.MavenCli.execute (MavenCli.java:956) at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288) at org.apache.maven.cli.MavenCli.main (MavenCli.java:192) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke (Method.java:566) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282) at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406) at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)[ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles:[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException[ERROR] [ERROR] After correcting the problems, you can resume the build with the command[ERROR] mvn <goals> -rf :spark-mllib_2.12 It doesn't seem like particularly helpful diagnostics to me? Any insight on how to proceed? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://github.com/apache/spark/pull/24888?email_source=notifications&email_token=ACENZ65GZDOWGBNU3LWWEV3QJ325HA5CNFSM4HYS7WZKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCEYLLJI#discussion_r324500771>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ACENZ65FESD66PZP5U3OFLTQJ325HANCNFSM4HYS7WZA>.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
