Thanks Jonathan. I should totally move to EMR. Spark on EMR was announced at Spark Summit!
There's no easy way to resize the cluster on EC2. You basically have to destroy it and launch a new one. Right? -Vadim ᐧ On Fri, Jun 19, 2015 at 3:41 PM, Kelly, Jonathan <jonat...@amazon.com> wrote: > Would you be able to use Spark on EMR rather than on EC2? EMR clusters > allow easy resizing of the cluster, and EMR also now supports Spark 1.3.1 > as of EMR AMI 3.8.0. See http://aws.amazon.com/emr/spark > > ~ Jonathan > > From: Vadim Bichutskiy <vadim.bichuts...@gmail.com> > Date: Friday, June 19, 2015 at 7:15 AM > To: user <user@spark.apache.org> > Subject: [ERROR] Insufficient Space > > Hello Spark Experts, > > I've been running a standalone Spark cluster on EC2 for a few months > now, and today I get this error: > > "IOError: [Errno 28] No space left on device > Spark assembly has been built with Hive, including Datanucleus jars on > classpath > OpenJDK 64-Bit Server VM warning: Insufficient space for shared memory > file" > > I guess I need to resize the cluster. What's the best way to do that? > > Thanks, > Vadim > ᐧ >
