Devesh Agrawal created SPARK-32215: -------------------------------------- Summary: Expose end point on Master so that it can be informed about decommissioned workers out of band Key: SPARK-32215 URL: https://issues.apache.org/jira/browse/SPARK-32215 Project: Spark Issue Type: Sub-task Components: Spark Core Affects Versions: 3.1.0 Environment: Standalone Scheduler Reporter: Devesh Agrawal Fix For: 3.1.0
The use case here is to allow some external entity that has made a decommissioning decision to inform the Master (in case of Standalone scheduling mode) The current decommissioning is triggered by the Worker getting getting a SIGPWR (out of band possibly by some cleanup hook), which then informs the Master about it. This approach may not be feasible in some environments that cannot trigger a clean up hook on the Worker. Add a new post endpoint {{/workers/kill}} on the MasterWebUI that allows an external agent to inform the master about all the nodes being decommissioned in bulk. The workers are identified by either their {{host:port}} or just the host -- in which case all workers on the host would be decommissioned. This API is merely a new entry point into the existing decommissioning logic. It does not change how the decommissioning request is handled in its core. The path /workers/kill is so chosen to be consistent with the other endpoint names on the MasterWebUI. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org