[
https://issues.apache.org/jira/browse/YUNIKORN-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaobao Wu updated YUNIKORN-2933:
---------------------------------
Description:
When the app processes the _yunikorn.apache.org/task-groups_ annotation on
Pod, it did not consider the case of taskgroup with the same name (i.e.
taskgroup with the same {*}name{*}).For example, the following task-groups
information is defined:
{code:java}
yunikorn.apache.org/task-groups='
[{
"name": "spark-executor",
"minMember": 2,
"minResource": {
"cpu": "1",
"memory": "1Gi"
}
}, {
"name": "spark-driver",
"minMember": 1,
"minResource": {
"cpu": "1",
"memory": "1Gi"
}
},
{
"name": "spark-executor",
"minMember": 1,
"minResource": {
"cpu": "2",
"memory": "2Gi"
}
}
]
'{code}
From the above example, it can be seen that there are two task-group with
*the same name* in the task-groups. So, after these ph Tasks are created, if an
executor task (2C 2G)
try to allocate placeholder‘s resource, it may erroneously release the ph task
in the *spark-executor* group with a resource specification of 1C 1G (because
the resource specification cannot meet 2C 2G). In this regard, I think that
before properly handling ph tasks with different resource specifications within
the same task group, it is necessary to avoid having two task-group with the
same name but different resource specifications in the app.
was:
When the app processes the _yunikorn.apache.org/task-groups_ annotation on
Pod, it did not consider the case of taskgroup with the same name (i.e.
taskgroup with the same {*}name{*}).For example, the following task-groups
information is defined:
{code:java}
yunikorn.apache.org/task-groups='
[{
"name": "spark-executor",
"minMember": 2,
"minResource": {
"cpu": "2",
"memory": "2Gi"
}
}, {
"name": "spark-driver",
"minMember": 1,
"minResource": {
"cpu": "1",
"memory": "1Gi"
}
},
{
"name": "spark-executor",
"minMember": 1,
"minResource": {
"cpu": "1",
"memory": "1Gi"
}
}
]
'{code}
From the above example, it can be seen that there are two task-group with
*the same name* in the task-groups. So, after these ph Tasks are created, if an
executor task (2C 2G)
try to allocate placeholder‘s resource, it may erroneously release the ph task
in the *spark-executor* group with a resource specification of 1C 1G (because
the resource specification cannot meet 2C 2G). In this regard, I think that
before properly handling ph tasks with different resource specifications within
the same task group, it is necessary to avoid having two task-group with the
same name but different resource specifications in the app.
> Don't add duplicated taskGroup to app
> -------------------------------------
>
> Key: YUNIKORN-2933
> URL: https://issues.apache.org/jira/browse/YUNIKORN-2933
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Reporter: Xiaobao Wu
> Priority: Minor
>
> When the app processes the _yunikorn.apache.org/task-groups_ annotation
> on Pod, it did not consider the case of taskgroup with the same name (i.e.
> taskgroup with the same {*}name{*}).For example, the following task-groups
> information is defined:
>
> {code:java}
> yunikorn.apache.org/task-groups='
> [{
> "name": "spark-executor",
> "minMember": 2,
> "minResource": {
> "cpu": "1",
> "memory": "1Gi"
> }
> }, {
> "name": "spark-driver",
> "minMember": 1,
> "minResource": {
> "cpu": "1",
> "memory": "1Gi"
> }
> },
> {
> "name": "spark-executor",
> "minMember": 1,
> "minResource": {
> "cpu": "2",
> "memory": "2Gi"
> }
> }
> ]
> '{code}
> From the above example, it can be seen that there are two task-group with
> *the same name* in the task-groups. So, after these ph Tasks are created, if
> an executor task (2C 2G)
> try to allocate placeholder‘s resource, it may erroneously release the ph
> task in the *spark-executor* group with a resource specification of 1C 1G
> (because the resource specification cannot meet 2C 2G). In this regard, I
> think that before properly handling ph tasks with different resource
> specifications within the same task group, it is necessary to avoid having
> two task-group with the same name but different resource specifications in
> the app.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
