I haven't checked this, but I believe the Mesos+ZooKeeper URI format has changed to require "mesos://" to be prepended to the URI, e.g: mesos://zk://zk-01:2181,zk-02:2181,zk-03:2181/masters
On Fri, Dec 20, 2013 at 12:01 PM, Gary Malouf <[email protected]> wrote: > For the past few months, when using Spark or the shell I have been > explicitly pointing to a single Mesos master. In reality, we have Mesos > configured with a primary and backup master which another application > (Chronos) respects. A url like the following is used in Chronos but results > in an error when creating a SparkContext: > > org.apache.spark.SparkException: Could not parse Master URL: > 'zk://zk-01:2181,zk-02:2181,zk-03:2181/masters' > > at org.apache.spark.SparkContext.<init>(SparkContext.scala:242) > > at org.apache.spark.repl.SparkILoop.createSparkContext(SparkILoop.scala:863) > > at $line1.$read$$iwC$$iwC.<init>(<console>:10) > > at $line1.$read$$iwC.<init>(<console>:22) > > at $line1.$read.<init>(<console>:24) > > at $line1.$read$.<init>(<console>:28) > > at $line1.$read$.<clinit>(<console>) > > at $line1.$eval$.<init>(<console>:7) > > at $line1.$eval$.<clinit>(<console>) > > at $line1.$eval.$export(<console>) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:601) > > at org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:629) > > at > org.apache.spark.repl.SparkIMain$Request$$anonfun$10.apply(SparkIMain.scala:897) > > at scala.tools.nsc.interpreter.Line$$anonfun$1.apply$mcV$sp(Line.scala:43) > > at scala.tools.nsc.io.package$$anon$2.run(package.scala:25) > > at java.lang.Thread.run(Thread.java:722) > > Failed to create SparkContext, exiting... > > > I tried this address based on the documentation in > http://spark.incubator.apache.org/docs/latest/running-on-mesos.html > > > After looking at the master regex parsing in SparkContext within master > however, I do not see how it could be supporting the zookeeper master > lookup. > > https://github.com/apache/incubator-spark/blob/4b895013cc965b37d44fd255656da470a3d2c222/core/src/main/scala/org/apache/spark/SparkContext.scala > > Can anyone clarify whether the masters being stored within zookeeper is > actually supported and if so how to connect this way?
