Hi Dick, There is a PR on Spark to enable launching Spark executors with Docker containers, and pending a committer to review and merge ( https://github.com/apache/spark/pull/3074)
If you like to try it out please take the patch and try it! Tim On Wed, Dec 3, 2014 at 6:31 PM, Kyle Ellrott <[email protected]> wrote: > I'd like to tag a question onto this; has anybody attempted to deploy > spark under Kubernetes > https://github.com/googlecloudplatform/kubernetes or Kubernetes mesos ( > https://github.com/mesosphere/kubernetes-mesos ) . > On Wednesday, December 3, 2014, Matei Zaharia <[email protected]> > wrote: > >> I'd suggest asking about this on the Mesos list (CCed). As far as I know, >> there was actually some ongoing work for this. >> >> Matei >> >> > On Dec 3, 2014, at 9:46 AM, Dick Davies <[email protected]> wrote: >> > >> > Just wondered if anyone had managed to start spark >> > jobs on mesos wrapped in a docker container? >> > >> > At present (i.e. very early testing) I'm able to submit executors >> > to mesos via spark-submit easily enough, but they fall over >> > as we don't have a JVM on our slaves out of the box. >> > >> > I can push one out via our CM system if push comes to shove, >> > but it'd be nice to have that as part of the job (I'm thinking it might >> > be a way to get some of the dependencies deployed too). >> > >> > bear in mind I'm a total clueless newbie at this so please be gentle >> > if I'm doing this completely wrong. >> > >> > Thanks! >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: [email protected] >> > For additional commands, e-mail: [email protected] >> > >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >>
