This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/spark-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new fa1f279c [SPARK-60087] Add Kueue feature support table to
`spark_custom_resources.md`
fa1f279c is described below
commit fa1f279cafa762f578fa46e341cd0b093633eef6
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Sat Oct 10 00:30:53 2026 +0100
[SPARK-60087] Add Kueue feature support table to `spark_custom_resources.md`
### What changes were proposed in this pull request?
This PR aims to add a table to the `Kueue` section of
`docs/spark_custom_resources.md` that summarizes which Kueue features are
supported. The existing items of the section are unchanged. The table also
covers partial admission, Topology Aware Scheduling, MultiKueue and elastic
workloads, which the docs did not mention before. None of them are supported.
### Why are the changes needed?
The section has 22 prose items and no table, so readers cannot see at a
glance how much of Kueue the integration supports.
### Does this PR introduce _any_ user-facing change?
No. This is a documentation-only change.
### How was this patch tested?
Manual review against the code.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #960 from dongjoon-hyun/SPARK-60087.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
docs/spark_custom_resources.md | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
diff --git a/docs/spark_custom_resources.md b/docs/spark_custom_resources.md
index 36270370..f2771ac1 100644
--- a/docs/spark_custom_resources.md
+++ b/docs/spark_custom_resources.md
@@ -651,6 +651,37 @@ spec:
sparkVersion: "4.2.0"
```
+The following table summarizes which Kueue features are supported. The items
after it describe the
+behavior in detail, and
+[Kueue Admission and Workload
Lifecycle](architecture.md#kueue-admission-and-workload-lifecycle)
+relates it to the reconciliation.
+
+| Feature |
Support | Notes
|
+|---------------------------------------------------------------------------|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Kueue `v1beta2` API |
Supported | Kueue v0.20.0 or later is recommended, see
[Compatibility](operations.md#compatibility).
|
+| Queueing by the `kueue.x-k8s.io/queue-name` label |
Supported | Both `SparkApplication` and `SparkCluster` are queued if
`spark.kubernetes.operator.kueue.enabled` is set, e.g. by
`operatorRbac.kueue.enabled` of the Helm chart, see [Optional
Prerequisites](operations.md#optional-prerequisites). |
+| Waiting for the admission |
Supported | The resource stays in its initializing state without its
driver (or master and workers), and the `KueueAdmissionPending` and
`KueueAdmitted` [events](configuration.md#kubernetes-events) report the
progress. |
+| Changing the queue label |
Supported | A change is followed until the `Workload` reserves quota,
while the `ValidatingAdmissionPolicy` of the Helm chart rejects one once the
resource started, unless it is `Suspended` or `ScheduledToRestart`.
|
+| `Workload` priority |
Supported | The `WorkloadPriorityClass` of the
`kueue.x-k8s.io/priority-class` label takes precedence over the
`priorityClassName` of the pod templates and then the `globalDefault`
`PriorityClass`, like Kueue built-in integrations. |
+| `ResourceFlavor` node labels and tolerations |
Supported | The `nodeLabels` and `tolerations` of the flavors which
Kueue assigns to a pod set are added to its pods, like Kueue built-in
integrations.
|
+| Admission checks, e.g. `ProvisioningRequest` |
Supported | The labels, annotations, node selector and tolerations in
the `podSetUpdates` of the checks are added to the pods once Kueue admits the
`Workload`.
|
+| Preemption and other evictions |
Supported | An evicted resource, e.g. a preempted one or one whose
`ClusterQueue` is stopped with `HoldAndDrain`, is queued again with a new
`Workload`, and a running one is released through `Suspended` first.
|
+| `Workload` deactivation, e.g. `kueuectl stop workload` |
Supported | A resource waits while its `Workload` is deactivated, and
a running one is released through `Suspended` and queued again once the
`Workload` is reactivated.
|
+| `waitForPodsReady` |
Supported | The operator records the `PodsReady` condition once the
driver and `spark.executor.instances` executors, or the master and workers, are
ready, and handles a `PodsReadyTimeout` like any other eviction.
|
+| `spec.suspend` |
Supported | It takes precedence over the queueing, so that a
suspended resource releases its `Workload`, see [Suspend](#suspend).
|
+| Releasing the quota of an application attempt |
Supported | The `Workload` is deleted once the driver and executor
pods of the attempt are gone, or gets the `Finished` condition when a
`Succeeded`, `Failed` or `DriverEvicted` application [retains its
resources](#delete-resources-on-termination). |
+| Requeuing backoff after an eviction |
Partially supported | The resource waits for the backoff which Kueue records,
but its new `Workload` starts the requeue count over, so the backoff does not
grow and a `backoffLimitCount` above 0 never deactivates it.
|
+| `recoveryTimeout` of `waitForPodsReady` |
Not supported | The `PodsReady` condition stays `True` when a pod is lost
later, e.g. an executor which Spark replaces.
|
+| Maximum execution time (`kueue.x-k8s.io/max-exec-time-seconds`) |
Not supported | Use the Spark native `spark.driver.timeout` instead, as
in [pi-with-driver-timeout.yaml](../examples/pi-with-driver-timeout.yaml).
|
+| Partial admission |
Not supported | The pod sets of a `Workload` have no `minCount`, so Kueue
admits all the requested pods or none.
|
+| Topology Aware Scheduling |
Not supported | The operator applies neither the topology assignment nor
the scheduling gate of Kueue to the pods.
|
+| MultiKueue |
Not supported | `SparkApplication` and `SparkCluster` have no
`spec.managedBy` field, which MultiKueue needs to run them on a worker cluster.
|
+| Elastic workloads (`Workload` slices) |
Not supported | A resource has one `Workload` whose pod set counts are
fixed.
|
+| `manageJobsWithoutQueueName` and `managedJobsNamespaceSelector` |
Not supported | Kueue does not manage the Spark custom resources itself,
so a resource without the `kueue.x-k8s.io/queue-name` label is not queued and
starts right away.
|
+| Dynamic allocation (`spark.dynamicAllocation.enabled`) |
Not supported | Such a `SparkApplication` fails with `SchedulingFailure`
instead of being queued.
|
+| `SparkCluster` with `minWorkers < maxWorkers` |
Not supported | Such a `SparkCluster` fails with `SchedulingFailure`
instead of being queued.
|
+| Pod template files (`spark.kubernetes.{driver,executor}.podTemplateFile`) |
Not supported | A `SparkApplication` whose pod template is set only in
such a file fails with `SchedulingFailure`, so set it in the
[spec](#pod-template-support) instead.
|
+
* The Helm chart sets `spark.kubernetes.operator.kueue.enabled` together with
the Kueue RBAC rules
when `operatorRbac.kueue.enabled` is set. Kueue and its `LocalQueue` must
exist as well. See
[Optional Prerequisites](operations.md#optional-prerequisites). Without the
integration, the
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
