Anand Changediya created SPARK-44417:
----------------------------------------
Summary: Spark Driver and executors were not transferring from
standby master to Alive Master.
Key: SPARK-44417
URL: https://issues.apache.org/jira/browse/SPARK-44417
Project: Spark
Issue Type: Bug
Components: Deploy
Affects Versions: 3.3.2
Environment: RedHat linux
Zookeeper HA Cluster
Reporter: Anand Changediya
Issue with the Spark driver and executors transferring from Standby Master to
Alive Master.
Spark is deployed with Zookeeper as cluster manager. Ideally if standby
node(server) goes down it should transfer applications running on standby node
to alive node.
Application transformations are killed in the Standby Master.
# Just consider we have two Application drivers were running on the Standby
Master server.
# If we forcefully shutdown, This should trigger the failover scenario and
application should move to Alive node. But, The Standby Master node both the
application drivers were not transferring to Active master instead those were
killing.
Check attachment for reference.
Steps to reproduce
* Deploy Spark HA cluster with Zookeper
* Deploy multiple application until we have drivers and executor assigned on
standby node.
* Now reboot the server
* check if all the applications were moving to Active Master.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
