Re: Building Spark behind a proxy
Hi Somya, I meant when you configure the JAVA_OPTS and when you don't configure the JAVA_OPTS is there any difference in the error message? Are you facing the same issue when you built using maven? Thanks Arush On Thu, Jan 29, 2015 at 10:22 PM, Soumya Simanta soumya.sima...@gmail.com wrote: I can do a wget http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom and get the file successfully on a shell. On Thu, Jan 29, 2015 at 11:51 AM, Boromir Widas vcsub...@gmail.com wrote: At least a part of it is due to connection refused, can you check if curl can reach the URL with proxies - [FATAL] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:14 from/to central ( http://repo.maven.apache.org/maven2): Error transferring file: Connection refused from http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom On Thu, Jan 29, 2015 at 11:35 AM, Soumya Simanta soumya.sima...@gmail.com wrote: On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: Does the error change on build with and without the built options? What do you mean by build options? I'm just doing ./sbt/sbt assembly from $SPARK_HOME Did you try using maven? and doing the proxy settings there. No I've not tried maven yet. However, I did set proxy settings inside my .m2/setting.xml, but it didn't make any difference. -- [image: Sigmoid Analytics] http://htmlsig.com/www.sigmoidanalytics.com *Arush Kharbanda* || Technical Teamlead ar...@sigmoidanalytics.com || www.sigmoidanalytics.com
Re: Building Spark behind a proxy
I can do a wget http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom and get the file successfully on a shell. On Thu, Jan 29, 2015 at 11:51 AM, Boromir Widas vcsub...@gmail.com wrote: At least a part of it is due to connection refused, can you check if curl can reach the URL with proxies - [FATAL] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:14 from/to central ( http://repo.maven.apache.org/maven2): Error transferring file: Connection refused from http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom On Thu, Jan 29, 2015 at 11:35 AM, Soumya Simanta soumya.sima...@gmail.com wrote: On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: Does the error change on build with and without the built options? What do you mean by build options? I'm just doing ./sbt/sbt assembly from $SPARK_HOME Did you try using maven? and doing the proxy settings there. No I've not tried maven yet. However, I did set proxy settings inside my .m2/setting.xml, but it didn't make any difference.
Re: Building Spark behind a proxy
On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: Does the error change on build with and without the built options? What do you mean by build options? I'm just doing ./sbt/sbt assembly from $SPARK_HOME Did you try using maven? and doing the proxy settings there. No I've not tried maven yet. However, I did set proxy settings inside my .m2/setting.xml, but it didn't make any difference.
Re: Building Spark behind a proxy
At least a part of it is due to connection refused, can you check if curl can reach the URL with proxies - [FATAL] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:14 from/to central ( http://repo.maven.apache.org/maven2): Error transferring file: Connection refused from http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom On Thu, Jan 29, 2015 at 11:35 AM, Soumya Simanta soumya.sima...@gmail.com wrote: On Thu, Jan 29, 2015 at 11:05 AM, Arush Kharbanda ar...@sigmoidanalytics.com wrote: Does the error change on build with and without the built options? What do you mean by build options? I'm just doing ./sbt/sbt assembly from $SPARK_HOME Did you try using maven? and doing the proxy settings there. No I've not tried maven yet. However, I did set proxy settings inside my .m2/setting.xml, but it didn't make any difference.
Re: Building Spark behind a proxy
Does the error change on build with and without the built options? Did you try using maven? and doing the proxy settings there. On Thu, Jan 29, 2015 at 9:17 PM, Soumya Simanta soumya.sima...@gmail.com wrote: I'm trying to build Spark (v1.1.1 and v1.2.0) behind a proxy using ./sbt/sbt assembly and I get the following error. I've set the http and https proxy as well as the JAVA_OPTS. Any idea what am I missing ? [warn] one warning found org.apache.maven.model.building.ModelBuildingException: 1 problem was encountered while building the effective model for org.apache.spark:spark-parent:1.1.1 [FATAL] Non-resolvable parent POM: Could not transfer artifact org.apache:apache:pom:14 from/to central ( http://repo.maven.apache.org/maven2): Error transferring file: Connection refused from http://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom and 'parent.relativePath' points at wrong local POM @ line 21, column 11 at org.apache.maven.model.building.DefaultModelProblemCollector.newModelBuildingException(DefaultModelProblemCollector.java:195) at org.apache.maven.model.building.DefaultModelBuilder.readParentExternally(DefaultModelBuilder.java:841) at org.apache.maven.model.building.DefaultModelBuilder.readParent(DefaultModelBuilder.java:664) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:310) at org.apache.maven.model.building.DefaultModelBuilder.build(DefaultModelBuilder.java:232) at com.typesafe.sbt.pom.MvnPomResolver.loadEffectivePom(MavenPomResolver.scala:61) at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41) at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:128) at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:49) at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:28) at SparkBuild$.projectDefinitions(SparkBuild.scala:165) at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:458) at sbt.Load$$anonfun$24.apply(Load.scala:415) at sbt.Load$$anonfun$24.apply(Load.scala:415) at scala.collection.immutable.Stream.flatMap(Stream.scala:442) at sbt.Load$.loadUnit(Load.scala:415) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at sbt.Load$$anonfun$15$$anonfun$apply$11.apply(Load.scala:256) at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:93) at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:92) at sbt.BuildLoader.apply(BuildLoader.scala:143) at sbt.Load$.loadAll(Load.scala:312) at sbt.Load$.loadURI(Load.scala:264) at sbt.Load$.load(Load.scala:260) at sbt.Load$.load(Load.scala:251) at sbt.Load$.apply(Load.scala:134) at sbt.Load$.defaultLoad(Load.scala:37) at sbt.BuiltinCommands$.doLoadProject(Main.scala:473) at sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:467) at sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:467) at sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:60) at sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:60) at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:62) at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:62) at sbt.Command$.process(Command.scala:95) at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:100) at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:100) at sbt.State$$anon$1.process(State.scala:179) at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:100) at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:100) at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18) at sbt.MainLoop$.next(MainLoop.scala:100) at sbt.MainLoop$.run(MainLoop.scala:93) at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:71) at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:66) at sbt.Using.apply(Using.scala:25) at sbt.MainLoop$.runWithNewLog(MainLoop.scala:66) at sbt.MainLoop$.runAndClearLast(MainLoop.scala:49) at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:33) at sbt.MainLoop$.runLogged(MainLoop.scala:25) at sbt.StandardMain$.runManaged(Main.scala:57) at sbt.xMain.run(Main.scala:29) at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109) at xsbt.boot.Launch$.withContextLoader(Launch.scala:129) at xsbt.boot.Launch$.run(Launch.scala:109) at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:36)
