Re: problem building spark on centos
w.r.t. the second error, have you read this ? http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <jade@nor1.com> wrote: > I’m using 3.3.9. Thanks! > > Jade > > From: Ted Yu <yuzhih...@gmail.com> > Date: Tuesday, January 5, 2016 at 4:57 PM > To: Jade Liu <jade@nor1.com> > Cc: "user@spark.apache.org" <user@spark.apache.org> > Subject: Re: problem building spark on centos > > Which version of maven are you using ? > > It should be 3.3.3+ > > On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <jade@nor1.com> wrote: > >> Hi, All: >> >> I’m trying to build spark 1.5.2 from source using maven with the >> following command: >> >> ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 >> -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests >> >> >> I got the following error:I' >> >> + VERSION='[ERROR] [Help 2] >> http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException >> ' >> >> >> When I try: >> >> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean >> package >> >> >> I got the following error: >> >> [FATAL] Non-resolvable parent POM for >> org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact >> org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): >> java.security.ProviderException: java.security.KeyException and >> 'parent.relativePath' points at wrong local POM @ line 22, column 11 >> >> >> Does anyone know how to change the settings in maven to fix this? >> >> >> Thanks in advance, >> >> >> Jade >> > >
Re: problem building spark on centos
I’ve changed the scala version to 2.10. With this command: build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package Build was successful. But make a runnable version: /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests Still fails with the following error: [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 20 more Caused by: Compile failed via zinc server at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136) at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86) at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303) at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119) at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99) at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) Not sure what’s causing it. Does anyone have any idea? Thanks! Jade From: Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> Date: Wednesday, January 6, 2016 at 10:40 AM To: Jade Liu <jade@nor1.com<mailto:jade@nor1.com>>, user <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: problem building spark on centos w.r.t. the second error, have you read this ? http://www.captaindebug.com/2013/03/mavens-non-resolvable-parent-pom-problem.html#.Vo1fFGSrSuo On Wed, Jan 6, 2016 at 9:49 AM, Jade Liu <jade@nor1.com<mailto:jade@nor1.com>> wrote: I’m using 3.3.9. Thanks! Jade From: Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>> Date: Tuesday, January 5, 2016 at 4:57 PM To: Jade Liu <jade@nor1.com<mailto:jade@nor1.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: problem building spark on centos Which version of maven are you using ? It should be 3.3.3+ On Tue, Jan 5, 2016 at 4:54 PM, Jade Liu <jade@nor1.com<mailto:jade@nor1.com>> wrote: Hi, All: I’m trying to build spark 1.5.2 from source using maven with the following command: ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests I got the following error:I' +
Re: problem building spark on centos
That should read "I think your missing the --name option". Sorry about that. On Wed, Jan 6, 2016 at 3:03 PM, Todd Nist <tsind...@gmail.com> wrote: > Hi Jade, > > I think you "--name" option. The makedistribution should look like this: > > ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 > -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests. > > As for why it failed to build with scala 2.11, did you run the > ./dev/change-scala-version.sh 2.11 script to set the version of the > artifacts to 2.11? If you do that then issue the build like this I think > you will be ok: > > ./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn > -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11 > -DskipTests > > HTH. > > -Todd > > On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <jade@nor1.com> wrote: > >> I’ve changed the scala version to 2.10. >> >> With this command: >> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean >> package >> Build was successful. >> >> But make a runnable version: >> /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 >> -Phive -Phive-thriftserver –DskipTests >> Still fails with the following error: >> [ERROR] Failed to execute goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) >> on project spark-launcher_2.10: Execution scala-compile-first of goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed >> -> [Help 1] >> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute >> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile >> (scala-compile-first) on project spark-launcher_2.10: Execution >> scala-compile-first of goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) >> at >> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) >> at >> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) >> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) >> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) >> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) >> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) >> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) >> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) >> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution >> scala-compile-first of goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. >> at >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145) >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) >> ... 20 more >> Caused by: Compile failed via zinc server >> at >> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136) >> at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86) >> at >> scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303) >> at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119) >> at >> scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99) >> at scala_maven.ScalaMojoSupport.execute(
Re: problem building spark on centos
Hi Jade, I think you "--name" option. The makedistribution should look like this: ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests. As for why it failed to build with scala 2.11, did you run the ./dev/change-scala-version.sh 2.11 script to set the version of the artifacts to 2.11? If you do that then issue the build like this I think you will be ok: ./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11 -DskipTests HTH. -Todd On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <jade@nor1.com> wrote: > I’ve changed the scala version to 2.10. > > With this command: > build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean > package > Build was successful. > > But make a runnable version: > /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 > -Phive -Phive-thriftserver –DskipTests > Still fails with the following error: > [ERROR] Failed to execute goal > net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) > on project spark-launcher_2.10: Execution scala-compile-first of goal > net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed > -> [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile > (scala-compile-first) on project spark-launcher_2.10: Execution > scala-compile-first of goal > net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) > at > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) > at > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) > at > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) > at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) > at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) > at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) > at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) > at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > at > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > at > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > Caused by: org.apache.maven.plugin.PluginExecutionException: Execution > scala-compile-first of goal > net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. > at > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145) > at > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) > ... 20 more > Caused by: Compile failed via zinc server > at > sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136) > at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86) > at > scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303) > at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119) > at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99) > at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482) > at > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) > > Not sure what’s causing it. Does anyone have any idea? > > Thanks! > > Jade > From: Ted Yu <yuzhih...@gmail.com> > Date: Wednesday, January 6, 2016 at 10:40 AM > To: Jade Liu <jade@nor1.com>, user <user@spark.apache.org> > > Subject: Re: problem building spark on centos > > w.r.t. the second error, have you read this ? > > http://www.captaindebug.com/201
Re: problem building spark on centos
Hi, Todd: Thanks for your suggestion. Yes I did run the ./dev/change-scala-version.sh 2.11 script when using scala version 2.11. I just tried this as you suggested: ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests Still got the same error: [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 20 more Caused by: Compile failed via zinc server at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136) at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86) at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303) at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119) at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99) at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) ... 21 more [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :spark-launcher_2.10 Do you think it’s java problem? I’m using oracle JDK 1.7. Should I update it to 1.8 instead? I just checked the conf and it says 1.7. Thanks, Jade From: Todd Nist <tsind...@gmail.com<mailto:tsind...@gmail.com>> Date: Wednesday, January 6, 2016 at 12:03 PM To: Jade Liu <jade@nor1.com<mailto:jade@nor1.com>> Cc: Ted Yu <yuzhih...@gmail.com<mailto:yuzhih...@gmail.com>>, "user@spark.apache.org<mailto:user@spark.apache.org>" <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: problem building spark on centos Hi Jade, I think you "--name" option. The makedistribution should look like this: ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests. As for why it failed to build with scala 2.11, did you run the ./dev/change-scala-version.sh 2.11 script to set the version of the artifacts to 2.11? If you do that then issue the build like this I think you will be ok: ./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn
Re: problem building spark on centos
er_2.10 > > Do you think it’s java problem? I’m using oracle JDK 1.7. Should I update > it to 1.8 instead? I just checked the conf and it says 1.7. > > Thanks, > > Jade > > From: Todd Nist <tsind...@gmail.com> > Date: Wednesday, January 6, 2016 at 12:03 PM > To: Jade Liu <jade@nor1.com> > Cc: Ted Yu <yuzhih...@gmail.com>, "user@spark.apache.org" < > user@spark.apache.org> > > Subject: Re: problem building spark on centos > > Hi Jade, > > I think you "--name" option. The makedistribution should look like this: > > ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 > -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -DskipTests. > > As for why it failed to build with scala 2.11, did you run the > ./dev/change-scala-version.sh 2.11 script to set the version of the > artifacts to 2.11? If you do that then issue the build like this I think > you will be ok: > > ./make-distribution.sh --name hadoop-2.6_scala-2.11 --tgz -Pyarn > -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver -Dscala-2.11 > -DskipTests > > HTH. > > -Todd > > On Wed, Jan 6, 2016 at 2:20 PM, Jade Liu <jade@nor1.com> wrote: > >> I’ve changed the scala version to 2.10. >> >> With this command: >> build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean >> package >> Build was successful. >> >> But make a runnable version: >> /make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 >> -Phive -Phive-thriftserver –DskipTests >> Still fails with the following error: >> [ERROR] Failed to execute goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) >> on project spark-launcher_2.10: Execution scala-compile-first of goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed >> -> [Help 1] >> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute >> goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile >> (scala-compile-first) on project spark-launcher_2.10: Execution >> scala-compile-first of goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) >> at >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) >> at >> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) >> at >> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) >> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) >> at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) >> at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) >> at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) >> at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) >> at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) >> at >> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) >> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution >> scala-compile-first of goal >> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. >> at >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145) >> at >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) >> ... 20 more >> Caused by: Compile failed via zinc server >> at >> sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136) >> at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompil
Re: problem building spark on centos
If you're trying to compile against Scala 2.11, you're missing "-Dscala-2.11" in that command. On Wed, Jan 6, 2016 at 12:27 PM, Jade Liuwrote: > Hi, Todd: > > Thanks for your suggestion. Yes I did run the ./dev/change-scala-version.sh > 2.11 script when using scala version 2.11. > > I just tried this as you suggested: > ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 > -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests -- Marcelo - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: problem building spark on centos
Yes I’m using maven 3.3.9. From: Todd Nist <tsind...@gmail.com<mailto:tsind...@gmail.com>> Date: Wednesday, January 6, 2016 at 12:33 PM To: Jade Liu <jade@nor1.com<mailto:jade@nor1.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" <user@spark.apache.org<mailto:user@spark.apache.org>> Subject: Re: problem building spark on centos Not sure, I just built it with java 8, but 7 is supported so that should be fine. Are you using maven 3.3.3 + ? RADTech:spark-1.5.2 tnist$ mvn -version Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0 Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T07:57:37-04:00) Maven home: /usr/local/maven Java version: 1.8.0_51, vendor: Oracle Corporation Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_51.jdk/Contents/Home/jre Default locale: en_US, platform encoding: UTF-8 OS name: "mac os x", version: "10.10.5", arch: "x86_64", family: "mac" On Wed, Jan 6, 2016 at 3:27 PM, Jade Liu <jade@nor1.com<mailto:jade@nor1.com>> wrote: Hi, Todd: Thanks for your suggestion. Yes I did run the ./dev/change-scala-version.sh 2.11 script when using scala version 2.11. I just tried this as you suggested: ./make-distribution.sh --name hadoop-2.6 --tgz -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver –DskipTests Still got the same error: [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first) on project spark-launcher_2.10: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288) at org.apache.maven.cli.MavenCli.main(MavenCli.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207) ... 20 more Caused by: Compile failed via zinc server at sbt_inc.SbtIncrementalCompiler.zincCompile(SbtIncrementalCompiler.java:136) at sbt_inc.SbtIncrementalCompiler.compile(SbtIncrementalCompiler.java:86) at scala_maven.ScalaCompilerSupport.incrementalCompile(ScalaCompilerSupport.java:303) at scala_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:119) at scala_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:99) at scala_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:482) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134) ... 21 more [ERROR] [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException [ERROR] [ERROR] After correcting the problems, you can resume th
Re: problem building spark on centos
Which version of maven are you using ? It should be 3.3.3+ On Tue, Jan 5, 2016 at 4:54 PM, Jade Liuwrote: > Hi, All: > > I’m trying to build spark 1.5.2 from source using maven with the following > command: > > ./make-distribution.sh --tgz -Phadoop-2.6 -Pyarn -Dhadoop.version=2.6.0 > -Dscala-2.11 -Phive -Phive-thriftserver –DskipTests > > > I got the following error: > > + VERSION='[ERROR] [Help 2] > http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException > ' > > > When I try: > > build/mvn -X -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean > package > > > I got the following error: > > [FATAL] Non-resolvable parent POM for > org.apache.spark:spark-parent_2.11:1.5.2: Could not transfer artifact > org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): > java.security.ProviderException: java.security.KeyException and > 'parent.relativePath' points at wrong local POM @ line 22, column 11 > > > Does anyone know how to change the settings in maven to fix this? > > > Thanks in advance, > > > Jade >