If you'd like to use Spark with Docker, the AMPLab's Docker scripts might be a nice starting point:
https://amplab.cs.berkeley.edu/2013/10/23/got-a-minute-spin-up-a-spark-cluster-on-your-laptop-with-docker/ https://github.com/amplab/docker-scripts On Mon, Jan 13, 2014 at 8:51 AM, Mayur Rustagi <[email protected]>wrote: > They are in active development but themselves ask not be used in > production.I can create one to play around I guess. > Regards > Mayur > > Mayur Rustagi > Ph: +919632149971 > h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com > https://twitter.com/mayur_rustagi > > > > On Mon, Jan 13, 2014 at 9:40 PM, Aureliano Buendia > <[email protected]>wrote: > >> Why not using docker for spark? GCE recently supports custom kernels >> allowing docker to work on it. >> >> A docket image can be shared between GCE and EC2. >> >> >> On Mon, Jan 13, 2014 at 8:56 AM, Mayur Rustagi >> <[email protected]>wrote: >> >>> Hi, >>> I have setup a basic guide here. >>> >>> >>> http://docs.sigmoidanalytics.com/index.php/How_to_Install_Spark_on_Google_Compute_Engine >>> >>> GCE unfortunately doesnt provide a easy way to manage images( It has >>> interface to download the tar.gz image and then reload it, I didnt find it >>> as convinient as AMI). If you create one let me know and I'll update the >>> guide accordingly. If you create a sharable image let me know i'll host it >>> on S3 and make it available for all. >>> >>> Regards >>> Mayur >>> >>> >>> Mayur Rustagi >>> Ph: +919632149971 >>> h <https://twitter.com/mayur_rustagi>ttp://www.sigmoidanalytics.com >>> https://twitter.com/mayur_rustagi >>> >>> >>> >>> On Mon, Jan 13, 2014 at 11:01 AM, Debasish Das <[email protected] >>> > wrote: >>> >>>> Hi Aureliano, >>>> >>>> Look for google compute engine scripts from typesafe repo. They >>>> recently tested Akka Cluster on 2400 nodes from Google Compute Engine. You >>>> should be able to reuse the scripts. >>>> >>>> Thanks. >>>> Deb >>>> >>>> >>>> On Sun, Jan 12, 2014 at 8:00 PM, Aureliano Buendia < >>>> [email protected]> wrote: >>>> >>>>> Hi, >>>>> >>>>> Has anyone worked on a script similar to spark-ec2 for google compute >>>>> engine? >>>>> >>>>> Google compute engine claims that they have faster instance start up >>>>> time, and that together with by minute charging makes it a desirable >>>>> choice >>>>> for spark. >>>>> >>>> >>>> >>> >> >
