RE: how to specify cores and executor to run spark jobs simultaneously

Hi Divya I think, you did try to run spark jobs on yarn. And also I think, you would like to submit jobs to different queues on yarn for each. And maybe you need to prepare queues on yarn to run spark jobs by configuring scheduler. Best regards,KyeongHee - Original Message -

Re: how to specify cores and executor to run spark jobs simultaneously

I am not sure about EMR , but seems multi tenancy is not enabled in your case. Multi tenancy means all the applications has to be submitted to different queues. Thanks Deepak On Wed, Sep 14, 2016 at 11:37 AM, Divya Gehlot wrote: > Hi, > > I am on EMR cluster and My

how to specify cores and executor to run spark jobs simultaneously

Hi, I am on EMR cluster and My cluster configuration is as below: Number of nodes including master node - 3 Memory:22.50 GB VCores Total : 16 Active Nodes : 2 Spark version- 1.6.1 Parameter set in spark-default.conf spark.executor.instances 2 > spark.executor.cores 8 >