dongjoon-hyun opened a new pull request, #59178: URL: https://github.com/apache/spark/pull/59178
### What changes were proposed in this pull request? This PR adds `hold` and `resume` actions to the REST Submission API of the Spark Standalone Master: - `POST /v1/submissions/hold/<app-id>` - `POST /v1/submissions/resume/<app-id>` They do what the Master web UI's **Hold** and **Resume** buttons (SPARK-59061) do, for scripts: - https://github.com/apache/spark/pull/58361 Unlike `kill` and `status`, these actions take an application ID rather than a submission ID, so they also work for applications submitted in client mode. This is the first REST Submission API action keyed by an application ID. A submission ID cannot be used here: the Master does not link a driver to the application it registers, and a client-mode application has no submission ID. ```mermaid sequenceDiagram participant R as REST client participant S as StandaloneRestServer participant M as Master participant C as StandaloneAppClient (driver) R->>S: POST /v1/submissions/hold/<app-id> S->>M: RequestApplicationHold (ask) Note over M: check the Master's and the application's<br/>spark.ui.holdEnabled, and the reported hold support M->>C: SetApplicationHold (ask, not awaited, unchanged) M-->>S: ApplicationHoldResponse(success, message) S-->>R: HoldApplicationResponse (JSON) C-->>M: ApplicationHoldUpdated (held/draining in /json/) ``` - `RequestApplicationHold` can now be asked as well as sent. When it is asked, the Master replies with a new `ApplicationHoldResponse(success, message)`. The Master web UI keeps `send`ing it, so the UI behaves as before. Like `kill`, a Master that is not `ALIVE` rejects the request. - The shared `Master.handleApplicationHold` now returns that response. It rejects a request when either the Master or the application disables `spark.ui.holdEnabled`, when the application has not reported that it can be held, or when the application is unknown or finished. Otherwise it forwards the request to the driver without waiting for the driver's answer, as before. - Like `kill`, the response only tells whether the request was forwarded to the driver, not its outcome. The driver drains its executors asynchronously, and the outcome shows up in the `held` and `draining` fields of the Master's `/json/` endpoint. - No new configuration. The Master-wide `spark.ui.holdEnabled` gates both the web UI controls and these actions. - On the REST side, `HoldRequestServlet` and `StandaloneHoldRequestServlet` relay the request to the Master the way the `kill` servlets do. A new `HoldApplicationResponse` message carries `appId`, `success`, and `message`. ```bash $ curl -XPOST http://IP:PORT/v1/submissions/hold/app-20260930120000-0000 { "action" : "HoldApplicationResponse", "appId" : "app-20260930120000-0000", "message" : "Requesting application app-20260930120000-0000 to hold.", "serverSparkVersion" : "4.4.0", "success" : true } ``` The docs add the two actions to the REST API table and describe them in **Held Applications** next to the scripted web UI requests documented by SPARK-59915: - https://github.com/apache/spark/pull/59176 ### Why are the changes needed? Since SPARK-59353, the Master web UI's hold and resume endpoints require the per-UI CSRF token, so a script has to read the token from the page before sending the request: - https://github.com/apache/spark/pull/58640 The REST Submission API is the Master's API for scripts and already offers `kill`, `killall`, and `clear`. Holding is the graceful alternative to killing: the application gives back its executors but keeps its driver and the shuffle output already written. It belongs in the same API, without the token round trip. These actions also reach client-mode applications, which the existing REST `kill` cannot, because `kill` takes a submission ID. ### Does this PR introduce _any_ user-facing change? Yes. The Standalone Master REST Submission API gets the new `hold` and `resume` actions. Since hold and resume are new in the unreleased Apache Spark 4.4.0, there is no behavior change compared to the released versions. Within the unreleased branches, the Master now rejects a hold request for an application that has not reported that it can be held, instead of forwarding it to the driver. The Master web UI never offers the controls for such an application, so the UI is unaffected. Like the other REST actions, `hold` and `resume` check no ACLs. The docs recommend protecting the REST API with `JWSFilter` or disabling it with `spark.master.rest.enabled=false`. ### How was this patch tested? Pass the CIs with the newly added test cases: - `StandaloneRestSubmitSuite`: `SPARK-59916: hold and resume an application` - Both actions relay the Master's reply together with the application ID. - A request without an application ID is rejected with 400. - `AppClientSuite`: `SPARK-59916: hold and resume an application through an ask to the Master` - A real Master rejects the request, without reaching the driver, for an application that has not reported that it can be held and for an unknown application. - Once the application reports that it can be held, the Master accepts both requests and both reach the driver. - `MasterSuite`: `SPARK-59916: Reject the hold and resume requests when the Master disables holding` The existing `SPARK-59061: hold and resume an application from the Master` test now reports the hold support first, since the Master requires it. The JSON in the documentation example is the actual output of `HoldApplicationResponse`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
