[
https://issues.apache.org/jira/browse/SPARK-44417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anand Changediya updated SPARK-44417:
-------------------------------------
Description:
Issue with the Spark driver and executors transferring from Standby Master to
Alive Master.
Spark is deployed with Zookeeper as cluster manager. Ideally if standby
node(server) goes down it should transfer applications running on standby node
to alive node.
Application transformations are killed in the Standby Master.
# Just consider we have two Application drivers were running on the Standby
Master server.
# If we forcefully shutdown, This should trigger the failover scenario and
application should move to Alive node. But, The Standby Master node both the
application drivers were not transferring to Active master instead those were
killing.
Check attachment for reference.
Steps to reproduce
* Deploy Spark HA cluster with Zookeeper
* Deploy multiple application until we have drivers and executor assigned on
standby node.
* Now reboot the server
* Check if all the applications were moving to Active Master.
was:
Issue with the Spark driver and executors transferring from Standby Master to
Alive Master.
Spark is deployed with Zookeeper as cluster manager. Ideally if standby
node(server) goes down it should transfer applications running on standby node
to alive node.
Application transformations are killed in the Standby Master.
# Just consider we have two Application drivers were running on the Standby
Master server.
# If we forcefully shutdown, This should trigger the failover scenario and
application should move to Alive node. But, The Standby Master node both the
application drivers were not transferring to Active master instead those were
killing.
Check attachment for reference.
Steps to reproduce
* Deploy Spark HA cluster with Zookeper
* Deploy multiple application until we have drivers and executor assigned on
standby node.
* Now reboot the server
* check if all the applications were moving to Active Master.
> Spark Driver and executors are not transferring from standby master to Alive
> Master.
> ------------------------------------------------------------------------------------
>
> Key: SPARK-44417
> URL: https://issues.apache.org/jira/browse/SPARK-44417
> Project: Spark
> Issue Type: Bug
> Components: Deploy
> Affects Versions: 3.3.2
> Environment: RedHat linux
> Zookeeper HA Cluster
> Reporter: Anand Changediya
> Priority: Critical
> Attachments: Issue_steps.docx
>
>
> Issue with the Spark driver and executors transferring from Standby Master to
> Alive Master.
> Spark is deployed with Zookeeper as cluster manager. Ideally if standby
> node(server) goes down it should transfer applications running on standby
> node to alive node.
> Application transformations are killed in the Standby Master.
> # Just consider we have two Application drivers were running on the Standby
> Master server.
> # If we forcefully shutdown, This should trigger the failover scenario and
> application should move to Alive node. But, The Standby Master node both the
> application drivers were not transferring to Active master instead those were
> killing.
> Check attachment for reference.
>
> Steps to reproduce
> * Deploy Spark HA cluster with Zookeeper
> * Deploy multiple application until we have drivers and executor assigned on
> standby node.
> * Now reboot the server
> * Check if all the applications were moving to Active Master.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
