This is an automated email from the ASF dual-hosted git repository.
manirajv06 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/yunikorn-site.git
The following commit(s) were added to refs/heads/master by this push:
new 23d3f7eb18 [YUNIKORN-2634] Remove deprecated user label documentation
(#561)
23d3f7eb18 is described below
commit 23d3f7eb1800cd13a6299f495b037f4ae092af14
Author: Wilfred Spiegelenburg <[email protected]>
AuthorDate: Tue Jul 7 21:15:39 2026 +0530
[YUNIKORN-2634] Remove deprecated user label documentation (#561)
Update the documentation to deprecate the user label processing.
Update the group resolver details to clarify that the annotation does
not use the group resolver. Add the partition level setting for the user
group resolver.
Closes: #561
Signed-off-by: Manikandan R <[email protected]>
---
docs/user_guide/queue_config.md | 34 +++++++------
docs/user_guide/usergroup_resolution.md | 90 ++++++++++++++++++---------------
2 files changed, 68 insertions(+), 56 deletions(-)
diff --git a/docs/user_guide/queue_config.md b/docs/user_guide/queue_config.md
index 726b0070cd..9314bd006f 100644
--- a/docs/user_guide/queue_config.md
+++ b/docs/user_guide/queue_config.md
@@ -35,7 +35,7 @@ The current shim identifies the user and the groups the user
belongs to using th
## Configuration
The configuration file for the scheduler that is described here only provides
the configuration for the partitions and queues.
-By default the scheduler reads the ConfigMap section `queues.yaml` for
partition and queue configuration. The section name can
+By default, the scheduler reads the ConfigMap section `queues.yaml` for
partition and queue configuration. The section name can
be changed by updating the `service.policyGroup` ConfigMap entry to be
something other than `queues`.
The example reference for the configuration is located in the scheduler core's
[queues.yaml](https://github.com/apache/yunikorn-core/blob/master/config/queues.yaml)
file.
@@ -65,8 +65,9 @@ Optionally the following keys can be defined for a partition:
* [limits](#limits)
* nodesortpolicy
* preemption
+* [usergroupresolver](usergroup_resolution.md#group-resolution)
-Placement rules and limits are explained in their own chapters
+Placement rules, limits and the user group resolver are explained in their own
chapters.
The `nodesortpolicy` key defines the way the nodes are sorted for the
partition.
Details on the values that can be used are in the [sorting
policy](sorting_policies.md#node-sorting) documentation.
@@ -82,7 +83,7 @@ Allowed values: _true_ or _false_, any other value will cause
a parse error.
The default value for _quotapreemptionenabled_ is _false_.
Allowed values: _true_ or _false_, any other value will cause a parse error.
-Example `partition` yaml entry with a `nodesortpolicy` of _fair_ and all
preemption disabled:
+Example `partition` yaml entry with a `nodesortpolicy` of _fair_, all
preemption disabled and the LDAP user group resolver:
```yaml
partitions:
- name: <name of the partition>
@@ -90,10 +91,13 @@ partitions:
preemption:
enabled: false
quotapreemptionenabled: false
+ usergroupresolver:
+ type: ldap
```
-NOTE:
-Currently the Kubernetes unique shim does not support any other partition than
the `default` partition.
-This has been logged as an
[jira](https://issues.apache.org/jira/browse/YUNIKORN-22) for the shim.
+:::note
+Currently the Kubernetes shim does not support any other partition than the
`default` partition.
+This has been logged as a
[jira](https://issues.apache.org/jira/browse/YUNIKORN-22) for the shim.
+:::
### Queues
@@ -107,8 +111,8 @@ YuniKorn manages resources by leveraging resource queues.
The resource queue has
:::info
The difference between YuniKorn queue and [Kubernetes
namespace](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/):
-Kubernetes namespace provides the scope for the Kubernetes resources,
including the security context (i.e who can access the objects), and resource
-boundary when [resource
quota](https://kubernetes.io/docs/concepts/policy/resource-quotas/) is defined
(i.e how many resources can be used by the objects).
+Kubernetes namespace provides the scope for the Kubernetes resources,
including the security context (i.e. who can access the objects), and resource
+boundary when [resource
quota](https://kubernetes.io/docs/concepts/policy/resource-quotas/) is defined
(i.e. how many resources can be used by the objects).
On the other hand, YuniKorn queue is only used how many resources can be used
by a group of jobs, and in which order. It provides
a more fine-grained control on resource sharing across multiple tenants with
considering of resource fairness, job ordering, etc. In most of the cases,
YuniKorn queue can be used to replace the namespace resource quota, in order
to provide more scheduling features.
@@ -117,11 +121,11 @@ YuniKorn queue can be used to replace the namespace
resource quota, in order to
The _queues_ entry is the main configuration element.
It defines a hierarchical structure for the queues.
-It can have a `root` queue defined but it is not a required element.
+It can have a `root` queue defined, but it is not a required element.
If the `root` queue is not defined the configuration parsing will insert the
root queue for consistency.
The insertion of the root queue is triggered by:
* If the configuration has more than one queue defined at the top level a root
queue is inserted.
-* If there is only one queue defined at the top level and it is not called
`root` a root queue is inserted.
+* If there is only one queue defined at the top level, and it is not called
`root` a root queue is inserted.
The defined queue or queues will become a child queue of the inserted `root`
queue.
@@ -152,7 +156,7 @@ A queue cannot contain a dot "." character as that
character is used to separate
Valid Queue name:
-* must be 64 characters or less,
+* must be less than or equal to 64 characters,
* can contain alphanumeric character ([a-z0-9A-Z]), underscores (_), colons
(:), hashes (#), slashes (/), at signs (@), and dashes (-).
If the name is not unique for the queue in the configuration or contains a dot
or does not follow the above rules, a parsing error is generated and the
configuration is rejected.
@@ -175,7 +179,7 @@ The `maxapplications` property is an integer value, larger
than 1, which allows
The [properties](#properties) section contains simple key/value pairs. This is
used for further queue customization of features such as
[application sorting](sorting_policies.md#application-sorting) and priority
-scheduling. Future features will use the exisitng `properties` section as well
+scheduling. Future features will use the existing `properties` section as well
to avoid the need to define a new structure for queue configuration.
Access to a queue is set via the `adminacl` for administrative actions and for
submitting an application via the `submitacl` entry.
@@ -215,7 +219,7 @@ The recovery queue, identified by the name
`root.@recovery@`, is a dynamic queue
- The queue is created dynamically and will disappear when it is no longer in
use.
- The queue does not have quotas or Access Control Lists (ACLs).
- It cannot be submitted to directly by users. It is managed internally by
YuniKorn for specific recovery operations.
-- While the queue is unqueryable directly, its existence and activities can be
observed through the application RESTful API at
[/ws/v1/partition/:partition/applications/:state](../api/scheduler#partition-applications).
+- While the queue is un-queryable directly, its existence and activities can
be observed through the application RESTful API at
[/ws/v1/partition/:partition/applications/:state](../api/scheduler#partition-applications).
### Placement rules
@@ -278,7 +282,7 @@ Duplicate entries in the lists are ignored and do not cause
a parsing error.
Specifying a star beside other list elements is not allowed.
When a wildcard group is configured, a limit must be configured with at least
one named group.
-Valid User name:
+Valid username:
* must start with a letter (uppercase or lowercase) or an underscore (_),
* followed by alphanumeric character ([a-z0-9A-Z]), underscores (_), colons
(:), hashes (#), slashes (/), at signs (@), dots (.), and dashes (-).
@@ -382,7 +386,7 @@ Supported values: any positive or negative 32-bit integer
Default value: `0`
-Adjusts the priority of the queue relative to it's siblings. This can be useful
+Adjusts the priority of the queue relative to its siblings. This can be useful
to create high or low-priority queues without needing to set every task's
priority manually.
diff --git a/docs/user_guide/usergroup_resolution.md
b/docs/user_guide/usergroup_resolution.md
index 3cc358bedb..1f7d39b87a 100644
--- a/docs/user_guide/usergroup_resolution.md
+++ b/docs/user_guide/usergroup_resolution.md
@@ -24,32 +24,49 @@ under the License.
## User resolution
-User information is an important aspect of the scheduling cycle. It is one of
the key identifier that can be used to determine the queue to which a job
should be submitted. The Yunikorn Scheduler relies on the K8s Shim to provide
user information. In the world of Kubernetes, there is no object defined that
identfies the actual user. This is by design and more information can be found
[here](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#users-in-kubernetes).
-
-In Yunikorn, there are two ways of handling users and groups. The first is the
legacy way, which uses the label `yunikorn.apache.org/username`. If this label
is set on a pod, then the value is automatically extracted in the shim and will
be used accordingly. Group resolution is also done in the shim and is disabled
by default. The problem with this approach is twofold: user restrictions can be
easily bypassed because the submitter is free to set this label to any value,
therefore this on [...]
-
-A more reliable and robust mechanism is using the
`yunikorn.apache.org/user.info` annotation, where the user information can be
set externally by an allowed list of users or groups or the admission
controller can attach this automatically to every workload.
+User information is an important aspect of the scheduling cycle. It is one of
the key identifier that can be used to determine the queue to which a job
should be submitted.
+The YuniKorn Scheduler relies on the K8s Shim to provide user information. In
the world of Kubernetes, there is no object defined that identifies the actual
user.
+This is by design and more information can be found
[here](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#users-in-kubernetes).
+A more reliable and robust mechanism is using the
`yunikorn.apache.org/user.info` annotation.
+The user information can be set externally by an allowed list of users or
groups or the admission controller can attach this automatically to every
workload.
## Group resolution
-Group membership resolution is pluggables and is defined here. Groups do not
have to be part of provided user and group object. When the object is added to
the cache the groups are automatically resolved based on the resolution that is
configured.
+Groups do not have to be part of provided user and group object. If the user
information contains one or more groups no group resolution is triggered.
+The groups in the `user.info` annotation will be used regardless of the group
resolution configured.
+
+If groups are not provided the groups are resolved inside the `yunikorn-core`
code. Group membership resolution is pluggable.
+When the object is added to the cache the groups are automatically resolved
based on the resolution that is configured.
The resolver which is linked to the cache can be set per partition.
-The default group resolver is "no resolver".
-This resolver just echos the user name and a primary group with the same name
as the user.
+The default group resolver is "no resolver". This resolver just echos the
username and a primary group with the same name as the user.
+
+To specify a group resolver the following partition level configuration
setting in the queue config is available:
+```yaml
+partitions:
+ - name: <name of the partition>
+ usergroupresolver:
+ type: <resolver type>
+```
+
+The currently supported resolvers are:
+* Default resolver: ""
+* LDAP resolver: "ldap"
+* OS resolver: "os"
+* test resolver: "test"
-Other resolvers are:
-* OS resolver
-* test resolver
+The LDAP resolver is in technical preview. See
[YUNIKORN-656](https://issues.apache.org/jira/browse/YUNIKORN-656) for
implementation details.
+[YUNIKORN-3158](https://issues.apache.org/jira/browse/YUNIKORN-3158) is
required for the graduation of the LDAP resolver to become generally available.
+The OS resolver depends on the OS of the node to provide the group details.
There is no configuration options for the resolver in YuniKorn.
+The test resolver is not for production use cases and is only used during unit
testing to provide an implementation that is independent of the test
environment.
## User handling
-Since Yunikorn 1.2 a more sophisticated way of user/group resolution is
available.
-
-In this mode, Yunikorn no longer relies on the `yunikorn.apache.org/username`
label, instead, the annotation `yunikorn.apache.org/user.info` is attached to
the workload. The value is simple JSON, which defines the user name and groups:
+User details can be specified via an annotation on a workload. The annotation
`yunikorn.apache.org/user.info` is attached to the workload.
+The value is simple JSON, which defines the username and groups:
```yaml
metadata:
@@ -69,9 +86,9 @@ However, to enhance security, the following is enforced in
the admission control
* if the annotation is missing, the admission controller will add this
information automatically
* attempts to change this annotation will be rejected
-We also no longer do this on pods only, but also on Deployments, ReplicaSets,
DeamonSets, StatefulSets, Jobs and CronJobs.
+The workload considered is not just a pod, but also on Deployments,
ReplicaSets, DaemonSets, StatefulSets, Jobs and CronJobs.
-Group resolution is no longer necessary inside the shim.
+Group resolution is no longer required inside the shim.
### Configuring the admission controller
@@ -85,34 +102,25 @@ The admission controller can be configured with the
`yunikorn-configs` configmap
| `externalUsers` | "" | Regular
expression for the allowed external user list |
| `externalGroups` | "" | Regular
expression for the allowed external group list |
-If `bypassAuth` is set to true the admission controller will not add the
annotation to a pod if the annotation is not present and the deprecated user
label is set. If the annotation is not set and the user label is not set the
new annotation will be added. In the case that `bypassAuth` is false, the
default, the admission controller will always add the new annotation,
regardless of the existence of the deprecated label.
-
-In certain scenarios, users and groups must be provided to Yunikorn upon
submission because the user and group management is provided by external
systems and the lookup mechanism is not trivial. In these cases, the
`externalUsers` and `externalGroups` can be configured which are treated as
regular expressions. Matching users and groups are allowed to set the
`yunikorn.apache.org/user.info` annotation to any arbitrary value. Since this
has implications which affects scheduling inside Yuni [...]
-
+If `bypassAuth` is set to true the admission controller will not add the
annotation to a pod if the annotation is not present and the deprecated user
label is set.
+If the annotation is not set and the user label is not set the new annotation
will be added.
+In the case that `bypassAuth` is false, the default, the admission controller
will always add the new annotation, regardless of the existence of the
deprecated label.
-## Legacy user handling
-
-### Using the `yunikorn.apache.org/username` label
-
-Since, Kubernetes has no pre-defined field or resource for user information
and individual cluster deployments with unique user identification tools can
vary, we have defined a standard way of identifying the user. Yunikorn requires
a Kubernetes
[Label](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)
added. Using the
[recommendation](https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/)
provided here, the default label is defined as below:
-
-| Label | Value
|
-|--------------------------------|--------------------------------------------------------------------------------------------------------------|
-| yunikorn.apache.org/username | User name. It can have duplicate
entries but only the first value will be used. The default user is `nobody` |
-
-Example:
-```yaml
-metadata:
- labels:
- yunikorn.apache.org/username: "john"
-```
-:::tip
-In order to make this field uniquiely identifiable to the authorized user, the
suggestion is to add this label as an immutable field by the user
identification tool used by the cluster administrators. The cluster
administrators or users are free to use any method or tool to add this field
and value. This includes adding it manually at the time of submission.
-:::
+In certain scenarios, users and groups must be provided to YuniKorn upon
submission because the user and group management is provided by external
systems and the lookup mechanism is not trivial.
+In these cases, the `externalUsers` and `externalGroups` can be configured
which are treated as regular expressions.
+Matching users and groups are allowed to set the
`yunikorn.apache.org/user.info` annotation to any arbitrary value.
+Since this has implications which affects scheduling inside YuniKorn, these
properties must be set carefully.
:::note Assumption
-Assumption:
- Yunikorn assumes that all pods belonging to an application are owned by the
same user. We recommend that the user label is added to every pod of an app.
This is to ensure that there is no discrepency.
+YuniKorn assumes that all pods belonging to an application are owned by the
same user.
+We recommend that the annotation is added to every pod of an application.
+This is to ensure that there is no discrepancy between the users if pods get
processed in an unexpected order.
:::
+## Deprecated user handling
+
+### Username specification via label
+Using the `yunikorn.apache.org/username` label has been deprecated and is no
longer supported.
+In YuniKorn 1.9.0 the usage of the label will only be possible if the
annotation is not found.
+Usage will trigger a deprecation log message. The code to support the fallback
to the label will be removed in YuniKorn 1.10.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
