[
https://issues.apache.org/jira/browse/YUNIKORN-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17890774#comment-17890774
]
Xiaobao Wu commented on YUNIKORN-2933:
--------------------------------------
[~wilfreds] For the different specifications of the same name task-group, we
have taken into account, so we think it is better not to have the same name
task-group.
Because the current YK does not have a good way to deal with task-group with
different names and specifications. For example, in the tryPlaceholderAllocate
method, the ph alloc under the task-group associated with the current ask is
traversed, and if the resource does not satisfy the currently processed ask, it
is released. Then even if there are different specifications of the same name
task-group, when encountering this situation, will only retain the largest
number of resources taskgroup.
So, in that case, I think we should avoid the same name task-group situation,
which can save us the time to create ph Pod and improve the efficiency of the
App. Always take the first definition is only a way to deal with, of course,
there are other ways to deal with may be better.
We can explain our behavior both in the documentation and in the log, so that
submitter can understand what their intentions will lead to, rather than
getting confused ( before we have thought about how to deal with different
specifications of the task-group ).
What do you think, looking forward to your reply.
> Don't add duplicated taskGroup to app
> -------------------------------------
>
> Key: YUNIKORN-2933
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2933
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Affects Versions: 1.3.0, 1.5.2
> Reporter: Xiaobao Wu
> Priority: Minor
> Labels: pull-request-available
>
> When the app processes the _yunikorn.apache.org/task-groups_ annotation
> on Pod, it did not consider the case of taskgroup with the same name (i.e.
> taskgroup with the same {*}name{*}).For example, the following task-groups
> information is defined:
>
> {code:java}
> yunikorn.apache.org/task-groups='
> [{
> "name": "spark-executor",
> "minMember": 2,
> "minResource": {
> "cpu": "1",
> "memory": "1Gi"
> }
> }, {
> "name": "spark-driver",
> "minMember": 1,
> "minResource": {
> "cpu": "1",
> "memory": "1Gi"
> }
> },
> {
> "name": "spark-executor",
> "minMember": 1,
> "minResource": {
> "cpu": "2",
> "memory": "2Gi"
> }
> }
> ]
> '{code}
> From the above example, it can be seen that there are two task-group with
> *the same name* in the task-groups. So, after these ph Tasks are created, if
> an executor task (2C 2G)
> try to allocate placeholder‘s resource, it may erroneously release the ph
> task in the *spark-executor* group with a resource specification of 1C 1G
> (because the resource specification cannot meet 2C 2G). In this regard, I
> think that before properly handling ph tasks with different resource
> specifications within the same task group, it is necessary to avoid having
> two task-group with the same name but different resource specifications in
> the app.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
