sunilgovind commented on a change in pull request #27:
URL:
https://github.com/apache/incubator-yunikorn-site/pull/27#discussion_r499965808
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to
simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to
the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on
submission but on actively consumed resources.
Review comment:
This is a real difference for YuniKorn vs Namespace. But many cannot
understand this from a one liner. So could we expand this a bit more.
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs,
you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the
k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue
with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart
template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted
based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration
set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find
more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation
by overwriting the configuration in the
+[helm chart
template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota
anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the
quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set
on the namespace.
+To allow specifying a quota on the namespace the following annotations should
be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the
queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit
specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical
queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64
CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods
in
-the corresponding queue. For example, if a job is submitted to namespace
`development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted
without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the
application to the queue.
+
+For example, if an application is submitted to namespace `development`, then
the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
Review comment:
Is this more like Parent Queue mapping with namespace?
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -121,80 +142,104 @@ partitions:
submitacl: '*'
properties:
application.sort.policy: stateaware
-
```
-Note, the property `application.sort.policy` in this configuration is set to
-`stateaware`. This is a simple app sorting policy applicable for batch jobs,
you
-can find more document [here](sorting_policies.md#StateAwarePolicy).
+This configuration places an application based on the `tag` rule.
+The tag selected is the `namespace` tag which is automatically added by the
k8s shim to all applications that get created.
+The `create` flag is set to true which will trigger the creation of the queue
with the same name as the namespace if it does not exist.
-You can do this during the installation by overwriting the configuration in the
-[helm chart
template](https://github.com/apache/incubator-yunikorn-release/blob/724ec82d0d548598e170cc6d5ca6aaae00f8286c/helm-charts/yunikorn/values.yaml#L71-L81).
+Applications within the automatically created child queues will be sorted
based sorting policy set on the parent queue.
+In this case the property `application.sort.policy` is in this configuration
set to `stateaware`.
+This is a simple app sorting policy applicable for batch jobs, you can find
more document [here](sorting_policies.md#StateAwarePolicy).
-### Set up namespaces
+You can change the configuration using the helm charts during the installation
by overwriting the configuration in the
+[helm chart
template](https://github.com/apache/incubator-yunikorn-release/blob/master/helm-charts/yunikorn/values.yaml#L71-L81).
-Continue to create namespaces like before, do not create namespace quota
anymore.
-Instead, set the following annotation in the namespace object:
+### Namespace quota
+Namespaces in Kubernetes contain the quota information.
+If a quota is set on a namespace Kubernetes will automatically enforce the
quota.
+In the case that YuniKorn is used for quota enforcement no quota must be set
on the namespace.
+To allow specifying a quota on the namespace the following annotations should
be set in the namespace object:
```yaml
yunikorn.apache.org/namespace.max.cpu: "64"
yunikorn.apache.org/namespace.max.memory: "100Gi"
```
+YuniKorn will parse these annotations and set the maximum capacity of the
queue mapped to this namespace.
+The values specified follow the standard Kubernetes formatting and unit
specification.
+Currently, we only support mapping memory and cpu not other resource types.
-YuniKorn will parse the annotation and set the max capacity of the dynamical
queue
-that mapped to this namespace to 64 CPU and 100GB memory.
+The example above will limit the queue mapped to the annotated namespace to 64
CPUs and 100GB memory.
-### Run workloads
+### Run a workload
-Jobs continue to be submitted to namespaces, based on the `Placementrule` used
-in the configuration. YuniKorn will automatically run the job and all its pods
in
-the corresponding queue. For example, if a job is submitted to namespace
`development`,
-then you will see the job is running in `root.development` queue.
+Applications, and the pods that are part of the application, can be submitted
without specific labels.
+YuniKorn will automatically add the required tags.
+The configured placement rule will create the queue, if required, and add the
application to the queue.
+
+For example, if an application is submitted to namespace `development`, then
the application will run in the `root.development` queue.
-## Option 3) Hierarchy queues with dynamical leaves that mapped to namespaces
+## Parent queue mapping
### Goal
-Though the tag placement rule using the `namespace` tag is capable of putting
applications based on the name of the namespace, sometimes more dynamic
placement is required.
+Though the tag placement rule using the `namespace` tag is capable of placing
an application in a queue this might not be enough in all setups.
+In some cases, multi tenancy for example, namespaces need to be grouped
together.
+Administrators could annotate namespaces which allows dynamic placement of
applications based on multiple annotations if placement rules were setup.
+YuniKorn cannot and does not just add all annotations from a namespace to an
application.
-Users can annotate namespaces which allows dynamic placement of applications
based on the annotation value if proper placement rules are present.
+To help support this grouping case a parent queue can be tagged on a
namespace.
### Configuration
Review comment:
Below section looks very complex to understand.
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to
simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to
the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on
submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue
hierarchy.
+The base of the queue structure, the `root` queue, does not allow setting a
quota as it reflects the current size of the cluster.
+Node additions and removals update the `root` queue quota automatically.
-Pre-setup a hierarchy of queues with min/max capacity, users can only submit
-jobs to the leaf queues. This approach fully manages the resource capacity for
-each of the queues, which is suitable to the scenarios that queues do not
change
-too often.
+Beside the `root` queue the quotas can be set, and is enforced, at any point
in the hierarchy.
+Every queue can have a quota set. The quota is enforced recursively throughout
the hierarchy.
+This means that a child queue can never use more resources than the
**configured** quota of the parent queue.
+Setting a quota on a child queue larger than its parent queue's quota would
thus not have any effect and is handled as a configuration error.
-### Configuration
+In the hierarchy there are some further rules that need to be considered.
+If a parent queue has multiple children the sum of the **usage** of all
children combined can never exceed the quota **configured** on the parent.
+However, from a configuration perspective this does not mean that the sum of
the **configured** quotas for all children must be smaller than the parent
quota.
+
+As an example the `root.parent` queue has a quota of 900.
Review comment:
Is there a possibility to insert some diagram or use some arrows to show
this queue?
##########
File path: docs/user_guide/resource_quota_mgmt.md
##########
@@ -22,57 +22,81 @@ specific language governing permissions and limitations
under the License.
-->
-YuniKorn can offer more fine-grained resource quota management comparing to
simply
-using namespace resource quota. Here are some how-to documents about setting up
-resource quota management with YuniKorn queues.
+## Quota configuration and rules
+YuniKorn can offer a finer grained resource quota management setup compared to
the simple
+namespace resource quota provided by Kubernetes.
-## Option 1) Static queues
+Contrary to quotas in Kubernetes YuniKorn does not enforce quotas on
submission but on actively consumed resources.
-### Goal
+Resource quotas in YuniKorn are linked to the queue and its place in the queue
hierarchy.
Review comment:
*its place -> it's placed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
