Just to add you can also look into SPARK_WORKER_INSTANCES configuration in the spark-env.sh file. On Aug 17, 2015 3:44 AM, "Daniel Darabos" <daniel.dara...@lynxanalytics.com> wrote:
> Hi Praveen, > > On Mon, Aug 17, 2015 at 12:34 PM, praveen S <mylogi...@gmail.com> wrote: > >> What does this mean in .setMaster("local[2]") >> > Local mode (executor in the same JVM) with 2 executor threads. > >> Is this applicable only for standalone Mode? >> > It is not applicable for standalone mode, only for local. > >> Can I do this in a cluster setup, eg: >> . setMaster("<hostname:port>[2]").. >> > No. It's faster to try than to ask a mailing list, actually. Also it's > documented at > http://spark.apache.org/docs/latest/submitting-applications.html#master-urls > . > >> Is it number of threads per worker node? >> > You can control the number of total threads with > spark-submit's --total-executor-cores parameter, if that's what you're > looking for. >