What is the Best practice , I have everything running as docker container in single host ( mesos and marathon also as docker container ) and everything comes up fine but when i try to launch the spark shell i get below error
SQL context available as sqlContext. scala> val data = sc.parallelize(1 to 100) data: org.apache.spark.rdd.RDD[Int] = ParallelCollectionRDD[0] at parallelize at <console>:27 scala> data.count [Stage 0:> (0 + 0) / 2]16/02/29 18:21:12 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 16/02/29 18:21:27 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources On Mon, Feb 29, 2016 at 12:04 PM, Tim Chen <t...@mesosphere.io> wrote: > No you don't have to run Mesos in docker containers to run Spark in docker > containers. > > Once you have Mesos cluster running you can then specfiy the Spark > configurations in your Spark job (i.e: > spark.mesos.executor.docker.image=mesosphere/spark:1.6) > and Mesos will automatically launch docker containers for you. > > Tim > > On Mon, Feb 29, 2016 at 7:36 AM, Ashish Soni <asoni.le...@gmail.com> > wrote: > >> Yes i read that and not much details here. >> >> Is it true that we need to have spark installed on each mesos docker >> container ( master and slave ) ... >> >> Ashish >> >> On Fri, Feb 26, 2016 at 2:14 PM, Tim Chen <t...@mesosphere.io> wrote: >> >>> https://spark.apache.org/docs/latest/running-on-mesos.html should be >>> the best source, what problems were you running into? >>> >>> Tim >>> >>> On Fri, Feb 26, 2016 at 11:06 AM, Yin Yang <yy201...@gmail.com> wrote: >>> >>>> Have you read this ? >>>> https://spark.apache.org/docs/latest/running-on-mesos.html >>>> >>>> On Fri, Feb 26, 2016 at 11:03 AM, Ashish Soni <asoni.le...@gmail.com> >>>> wrote: >>>> >>>>> Hi All , >>>>> >>>>> Is there any proper documentation as how to run spark on mesos , I am >>>>> trying from the last few days and not able to make it work. >>>>> >>>>> Please help >>>>> >>>>> Ashish >>>>> >>>> >>>> >>> >> >
