There are 4 cores on my system.
Running spark with setMaster("local[2]") results in:
PID USER PR NIVIRTRESSHR S %CPU %MEM TIME+
COMMAND
7 root 20 0 4748836
Hi On,
When you run in Spark mode there is only one SparkSubmit with one executor
only. How many cores do you have?
Each core will allow the same code to run concurrently so with local{8} you
will have 8 tasks running the same code on subset of your data
So do
cat /proc/cpuinfo|grep
OK, sorry, I am running in local mode.
Just a very small setup...
(changed the subject)
On 25.07.2016 20:01, Mich Talebzadeh wrote:
> Hi,
>
> From your reference I can see that you are running in local mode with
> two cores. But that is not standalone.
>
> Can you please clarify whether you
