Re: OOM When Running with Mesos Fine-grained Mode

2016-03-05 Thread SLiZn Liu
Hi Tamas, I feel terribly sorry that I forgot to mention we are currently running Spark 1.6. Thanks for your reply though. BR, Todd Leo Tamas Szuromi 于2016年3月5日 周六下午4:33写道： > Hey, > We had the same with Spark 1.5.x and disappeared after we upgraded to 1.6. > > Tamas > > > On Saturday, 5 March 20

Re: OOM When Running with Mesos Fine-grained Mode

2016-03-05 Thread Tamas Szuromi
Hey, We had the same with Spark 1.5.x and disappeared after we upgraded to 1.6. Tamas On Saturday, 5 March 2016, SLiZn Liu wrote: > Hi Spark Mailing List, > > I’m running terabytes of text files with Spark on Mesos, the job runs fine > until we decided to switch to Mesos fine-grained mode. > >

OOM When Running with Mesos Fine-grained Mode

2016-03-04 Thread SLiZn Liu
Hi Spark Mailing List, I’m running terabytes of text files with Spark on Mesos, the job runs fine until we decided to switch to Mesos fine-grained mode. At first glance, we spotted massive number of task lost errors in logs: 16/03/05 04:01:20 ERROR TaskSchedulerImpl: Ignoring update with state L