Hi all,
(Waking up an old thread just for future reference)
We've had a very similar issue just a couple of days ago: executing a spark
driver on the same host as where the mesos master runs succeeds, but
executing it on our remote dev station hangs fails after mesos report the
spark driver dis
-0/
Jacob
Jacob D. Eisinger
IBM Emerging Technologies
jeis...@us.ibm.com - (512) 286-6075
From: Gerard Maas
To: user@spark.apache.org
Date: 05/16/2014 10:26 AM
Subject: Re: Local Dev Env with Mesos + Spark Streaming on Docker: Can't
submit jobs.
Hi Jacob,
Than
I
On May 5, 2014 1:12 PM, "Gerard Maas" wrote:
>
> Hi all,
>
> I'm currently working on creating a set of docker images to facilitate
local development with Spark/streaming on Mesos (+zk, hdfs, kafka)
>
> After solving the initial hurdles to get things working together in
docker containers, now ev
Hi Gerard, thank you for your feedbacks.
On Mon, May 5, 2014 at 11:17 PM, Gerard Maas wrote:
> Hi Benjamin,
>
> Yes, we initially used a modified version of the AmpLabs docker scripts
> [1]. The amplab docker images are a good starting point.
> One of the biggest hurdles has been HDFS, which re
y used a
> modified version of the AmpLabs docker scripts
>
> From: Gerard Maas
> To: user@spark.apache.org
> Date: 05/05/2014 04:18 PM
> Subject: Re: Local Dev Env with Mesos + Spark Streaming on Docker: Can't
> submit jobs.
> --
>
>
&g
quot;
> To: user@spark.apache.org
> Sent: Tuesday, May 6, 2014 8:30:23 AM
> Subject: Re: Local Dev Env with Mesos + Spark Streaming on Docker: Can't
> submit jobs.
> Howdy,
> You might find the discussion Andrew and I have been having about Docker and
> network security [1] applicab
/2014 04:18 PM
Subject:Re: Local Dev Env with Mesos + Spark Streaming on Docker: Can't
submit jobs.
Hi Benjamin,
Yes, we initially used a modified version of the AmpLabs docker scripts
[1]. The amplab docker images are a good starting point.
One of the biggest hurdles has
Hi Benjamin,
Yes, we initially used a modified version of the AmpLabs docker scripts
[1]. The amplab docker images are a good starting point.
One of the biggest hurdles has been HDFS, which requires reverse-DNS and I
didn't want to go the dnsmasq route to keep the containers relatively
simple to u
Hi,
Before considering running on Mesos, did you try to submit the application
on Spark deployed without Mesos on Docker containers ?
Currently investigating this idea to deploy quickly a complete set of
clusters with Docker, I'm interested by your findings on sharing the
settings of Kafka and Zo
Hi all,
I'm currently working on creating a set of docker images to facilitate
local development with Spark/streaming on Mesos (+zk, hdfs, kafka)
After solving the initial hurdles to get things working together in docker
containers, now everything seems to start-up correctly and the mesos UI
show
10 matches
Mail list logo
