This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new ac2831c5f75c [SPARK-59915][DOC] Document how to hold and resume
applications via scripted HTTP requests
ac2831c5f75c is described below
commit ac2831c5f75ca3abf86c811ddde64a40a3e12564
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Wed Sep 30 17:46:26 2026 -0700
[SPARK-59915][DOC] Document how to hold and resume applications via
scripted HTTP requests
### What changes were proposed in this pull request?
This PR documents how to hold and resume an application through scripted
HTTP requests (e.g. `curl`). The endpoints are the same ones the
**Hold**/**Resume** buttons already submit to:
- `docs/web-ui.md`: the driver web UI endpoints `/jobs/hold/` and
`/jobs/resume/`, plus the existing `holdstatus` REST API for checking the
result.
- `docs/spark-standalone.md` (**Held Applications**): the Master web UI
endpoints `/app/hold/` and `/app/resume/`, and the `held`/`draining` fields of
`/json/` for checking the result, since the Master forwards the request to the
driver asynchronously.
For each endpoint, the docs describe:
- the request must be a POST (on the driver UI, GET is also accepted when
`spark.ui.actionsViaGetEnabled` is on);
- it must carry the per-UI `csrfToken`, which is the value of the hidden
`csrfToken` field on the corresponding page and stays fixed for the lifetime of
that UI;
- the trailing slash is required: without it, the request is only
redirected and does not take effect;
- a request without a valid token is rejected with 403;
- when the UI requires authentication, reading the token and sending the
request both need the same credentials.
The examples use placeholders (`<csrf-token>`, `<app-id>`) in the same
style as the existing REST API examples.
```bash
curl -X POST -d "csrfToken=<csrf-token>"
http://<driver-host>:4040/jobs/hold/
curl -X POST -d "id=<app-id>" -d "csrfToken=<csrf-token>"
http://<master-host>:8080/app/hold/
```
### Why are the changes needed?
Since SPARK-59353 (#58640), state-changing UI endpoints accept only POST
requests that carry a per-UI CSRF token. The token check cannot be disabled. So
the plain calls operators used before, such as `curl -X POST
http://<driver-host>:4040/jobs/hold`, now fail with 403, and nothing in the
hold/resume docs explains how to call these endpoints from a script.
`docs/configuration.md` only says "Scripted clients can read the token from the
jobs page".
- https://github.com/apache/spark/pull/58640
This PR fills that gap without weakening the CSRF protection: no token-less
endpoint is added, and the endpoints behave exactly as they do today.
### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only change.
### How was this patch tested?
I manually checked the documented requests on a local Standalone cluster:
one Master, one Worker with `spark.shuffle.service.enabled=true`, and a
`spark-shell` application with `spark.decommission.enabled=true`.
| Request | Driver UI | Master UI |
|---|---|---|
| POST with a valid `csrfToken` | 302, then `holdstatus` shows `held: true`
| 302, then `/json/` shows `held` toggled by resume/hold |
| POST without a token | 403 | 403 |
| POST with a wrong token | 403 | - |
| POST without the trailing slash | 301, no effect | 301, no effect |
I also built the docs locally and checked that the new sections render
correctly.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #59176 from dongjoon-hyun/SPARK-59915.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/spark-standalone.md | 15 +++++++++++++++
docs/web-ui.md | 15 +++++++++++++++
2 files changed, 30 insertions(+)
diff --git a/docs/spark-standalone.md b/docs/spark-standalone.md
index 41a4e104deac..1024f0f67052 100644
--- a/docs/spark-standalone.md
+++ b/docs/spark-standalone.md
@@ -804,6 +804,21 @@ whose own setting travels with its registration -- an
application that disabled
and its hold requests are rejected, while its hold status stays visible. The
same controls remain
available on the driver web UI.
+Scripts can hold and resume an application through the `/app/hold/` and
`/app/resume/` endpoints
+the buttons submit to. Like the buttons, a request must be a POST carrying the
application ID as
+`id` and the random per-UI `csrfToken`, the value of the hidden `csrfToken`
field in the Master
+web UI page, which does not change for the lifetime of the Master. A request
without a valid
+token is rejected with 403, and one without the trailing slash is only
redirected, without
+taking effect. For example:
+
+```bash
+curl -X POST -d "id=<app-id>" -d "csrfToken=<csrf-token>"
http://<master-host>:8080/app/hold/
+```
+
+If the Master web UI requires authentication, reading the token and sending
the request both
+need the same credentials. The request is forwarded to the driver
asynchronously, so its outcome
+shows up afterwards in the `held` and `draining` fields of the Master's
`/json/` endpoint.
+
# Running Alongside Hadoop
diff --git a/docs/web-ui.md b/docs/web-ui.md
index 0e4aef1496e9..7521fc3f03bc 100644
--- a/docs/web-ui.md
+++ b/docs/web-ui.md
@@ -88,6 +88,21 @@ drain cannot complete. The control only appears when
`spark.ui.holdEnabled` is t
cluster manager can hold executors (Standalone, YARN, and Kubernetes with the
`direct` pods
allocator); see [Configuration](configuration.html#spark-ui).
+Scripts can hold and resume the application through the same `/jobs/hold/` and
`/jobs/resume/`
+endpoints the buttons submit to. Like the buttons, a request must be a POST
(GET is accepted as
+well only when `spark.ui.actionsViaGetEnabled` is on) carrying the random
per-UI `csrfToken`,
+the value of the hidden `csrfToken` field in the jobs page, which does not
change for the
+lifetime of the UI. A request without a valid token is rejected with 403, and
one without the
+trailing slash is only redirected, without taking effect. For example:
+
+```bash
+curl -X POST -d "csrfToken=<csrf-token>" http://<driver-host>:4040/jobs/hold/
+curl http://<driver-host>:4040/api/v1/applications/<app-id>/holdstatus
+```
+
+If the web UI requires authentication, reading the token and sending the
request both need
+the same credentials.
+
<p style="text-align: center;">
<img src="img/AllJobsPage.png" title="All Jobs page" alt="All Jobs page"
width="100%"/>
</p>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
