This is an automated email from the ASF dual-hosted git repository.
ccondit pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yunikorn-site.git
The following commit(s) were added to refs/heads/master by this push:
new df5ddac3f4 [YUNIKORN-2439] Deprecate state-aware sorting policy (#403)
df5ddac3f4 is described below
commit df5ddac3f4bd543241554d33d14eee5d9206d871
Author: Craig Condit <[email protected]>
AuthorDate: Mon Feb 26 13:27:50 2024 -0600
[YUNIKORN-2439] Deprecate state-aware sorting policy (#403)
Closes: #403
---
docs/user_guide/gang_scheduling.md | 3 ++-
docs/user_guide/labels_and_annotations_in_yunikorn.md | 14 +++++++-------
docs/user_guide/queue_config.md | 6 +++++-
docs/user_guide/resource_quota_mgmt.md | 5 ++---
docs/user_guide/sorting_policies.md | 6 ++++++
5 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/docs/user_guide/gang_scheduling.md
b/docs/user_guide/gang_scheduling.md
index 21739bce2d..1b8dd24bf2 100644
--- a/docs/user_guide/gang_scheduling.md
+++ b/docs/user_guide/gang_scheduling.md
@@ -52,7 +52,7 @@ To configure queue sorting policy, please refer to doc: [app
sorting policies](s
#### Why the `FIFO` sorting policy
When Gang Scheduling is enabled, the scheduler proactively reserves resources
-for each application. If the queue sorting policy is not FIFO based
(StateAware is FIFO based sorting policy),
+for each application. If the queue sorting policy is not FIFO-based
(StateAware is FIFO based sorting policy),
the scheduler might reserve partial resources for each app and causing
resource segmentation issues.
#### Side effects of `StateAware` sorting policy
@@ -61,6 +61,7 @@ We do not recommend using `StateAware`, even-though it is a
FIFO based policy. A
This is specifically an issue with Spark jobs when the driver performs a lot
of pre-processing before requesting the executors.
The `StateAware` timeout in those cases would slow down processing to just one
application per timeout.
This in effect will overrule the gang reservation and cause slowdowns and
excessive resource usage.
+`StateAware` sorting is **deprecated** in YuniKorn 1.5.0 and will be
**removed** from YuniKorn 1.6.0.
### App Configuration
diff --git a/docs/user_guide/labels_and_annotations_in_yunikorn.md
b/docs/user_guide/labels_and_annotations_in_yunikorn.md
index efd9ee8df6..550feaef03 100644
--- a/docs/user_guide/labels_and_annotations_in_yunikorn.md
+++ b/docs/user_guide/labels_and_annotations_in_yunikorn.md
@@ -25,13 +25,13 @@ under the License.
YuniKorn utilizes several Kubernetes labels and annotations to support various
features:
### Labels in YuniKorn
-| Name | Description
|
-|----------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `applicationId` | Associates this pod with an application.
|
-| `queue` | Selects the YuniKorn queue this application should be
scheduled in. This may be ignored if a placement policy is in effect.
|
-| `spark-app-selector` | Alternative method of specifying `applicationId` used
by Spark Operator if the label `applicationId` and annotation
`yunikorn.apache.org/app-id` unset. |
-| `disableStateAware` | If present, disables the YuniKorn state-aware
scheduling policy for this pod. Set internally by the YuniKorn admission
controller. |
-| `placeholder` | Set if this pod represents a placeholder for gang
scheduling. Set internally by YuniKorn.
|
+| Name | Description
|
+|-----------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `applicationId` | Associates this pod with an application.
|
+| `queue` | Selects the YuniKorn queue this
application should be scheduled in. This may be ignored if a placement policy
is in effect. |
+| `spark-app-selector` | Alternative method of specifying
`applicationId` used by Spark Operator if the label `applicationId` and
annotation `yunikorn.apache.org/app-id` unset. |
+| [DEPRECATED] `disableStateAware` | If present, disables the YuniKorn
state-aware scheduling policy for this pod. Set internally by the YuniKorn
admission controller. |
+| `placeholder` | Set if this pod represents a placeholder
for gang scheduling. Set internally by YuniKorn.
|
### Annotations in YuniKorn
All annotations are under the namespace `yunikorn.apache.org`. For example
`yunikorn.apache.org/app-id`.
diff --git a/docs/user_guide/queue_config.md b/docs/user_guide/queue_config.md
index 5362785729..75bd24133a 100644
--- a/docs/user_guide/queue_config.md
+++ b/docs/user_guide/queue_config.md
@@ -299,6 +299,10 @@ Default value: `fifo`
Sets the policy to be used when sorting applications within a queue. This
setting has no effect on a _parent_ queue.
+**NOTE:** The `stateaware` policy is **deprecated** in YuniKorn 1.5.0 and will
+be **removed** from YuniKorn 1.6.0, where it will be treated as an alias for
+`fifo`.
+
See the documentation on [application
sorting](sorting_policies.md#application-sorting)
for more information.
@@ -436,7 +440,7 @@ As an example:
childtemplate:
maxapplications: 10
properties:
- application.sort.policy: stateaware
+ application.sort.policy: fifo
resources:
guaranteed:
vcore: 1
diff --git a/docs/user_guide/resource_quota_mgmt.md
b/docs/user_guide/resource_quota_mgmt.md
index 1c8bf11983..a29bbbd47a 100644
--- a/docs/user_guide/resource_quota_mgmt.md
+++ b/docs/user_guide/resource_quota_mgmt.md
@@ -207,7 +207,7 @@ partitions:
- name: root
submitacl: '*'
properties:
- application.sort.policy: stateaware
+ application.sort.policy: fifo
```
This configuration places an application based on the `tag` rule.
@@ -215,8 +215,7 @@ The tag selected is the `namespace` tag which is
automatically added by the k8s
The `create` flag is set to true which will trigger the creation of the queue
with the same name as the namespace if it does not exist.
Applications within the automatically created child queues will be sorted
based sorting policy set on the parent queue.
-In this case the property `application.sort.policy` is in this configuration
set to `stateaware`.
-This is a simple app sorting policy applicable for batch jobs, you can find
more document [here](sorting_policies.md#stateawarepolicy).
+In this case the property `application.sort.policy` is in this configuration
set to `fifo`.
You can change the configuration using the helm charts during the installation
by overwriting the configuration in the
[helm chart
template](https://github.com/apache/yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
diff --git a/docs/user_guide/sorting_policies.md
b/docs/user_guide/sorting_policies.md
index 982877a138..f5de40af04 100644
--- a/docs/user_guide/sorting_policies.md
+++ b/docs/user_guide/sorting_policies.md
@@ -91,6 +91,12 @@ Short description: limit of one (1) application in Starting
or Accepted state
Config value: `stateaware`
+**DEPRECATED:** The `stateaware` policy is **deprecated** in YuniKorn 1.5.0 and
+will be **removed** in YuniKorn 1.6.0. To preserve backwards compatibility,
+`stateaware` will become an alias for `fifo` in YuniKorn 1.6.0 and later.
+Users are encouraged to migrate to `fifo` and utilize either gang scheduling or
+`maxapplications` to limit concurrency instead.
+
This sorting policy requires an understanding of the application states.
Applications states are described in the [application
states](developer_guide/scheduler_object_states.md#application-state)
documentation.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
