[
https://issues.apache.org/jira/browse/SPARK-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651882#comment-14651882
]
partha bishnu commented on SPARK-9559:
--------------------------------------
Hi
I am running some tests on spark in stand-alone mode with 3 nodes cluster.
spark master is running on n-1, and slaves are on n-2 and n-3. Each machine is
with 8G RAM and 4 core cpu. I am trying to test worker redundancy.
I wanted to set up the cluster such a way so that there will be two worker JVM,
one on each slave (n-2 and n-3) after I start up the cluster.
Then one of the slave's worker JVM will launch the executor jvm to process the
tasks when I submit the job with the following flags:
---total-executor-cores 1 and --executor-memory 1G
(1) Job submitted successfully in client mode. n-2 had worker jvm launched a
executor jvm. So now n-2 had one worker jvm and one executor jvm running and
n-3 just had the worker jvm running as before.
(2) I killed the worker jvm and the executor jvm on n-2
(3) I expected spark master on n-1 will then ask the worker jvm on n-3 to
launch a new executor to start processing jobs but that did not happen. driver
just got hung on the screen. n-2 disappeared from spark cluster as expected.
n-3 just had the worker jvm running as before and no new executor was launched
as expected after n-2 disappeared.
> Worker redundancy/failover in spark stand-alone mode
> ----------------------------------------------------
>
> Key: SPARK-9559
> URL: https://issues.apache.org/jira/browse/SPARK-9559
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.3.0
> Reporter: partha bishnu
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
