[
https://issues.apache.org/jira/browse/YUNIKORN-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17685233#comment-17685233
]
Wilfred Spiegelenburg commented on YUNIKORN-1546:
-------------------------------------------------
After some offline discussion:
The good specific change case for parent to leaf might work. The fact that all
the failure cases are failing silently is not acceptable.
We need to define what is more important: config or running applications and
children in memory. Config seems to be the one that should overrule the current
in running setup. That means we need to define what we do with the current
running applications if we change a leaf to a parent. What do we do with the
leafs under a parent and all the applications that run in those leafs? Do we
move the structure out of the way to allow the configuration change to be
applied...
The other points that need to be checked are the placement rules. There is no
check in the validator for the placement rules. A configuration change could
easily break them. This is an existing missing piece of functionality. As an
example I could create a rule that sets a parent without create flag. If that
queue does not exist the rule will not work. More examples can be created. Not
all can be caught but currently we have nothing and we should add some basic
checks for as part of the validator.
Failures cases that eat the failure are a problem. The scheduler runs with a
different configuration than what is there in the configmap. We cannot allow
that to happen. That is even a problem without the change in queue type. The
update either needs to be applied fully or we need to reject the config change
in the validation step.
For point 3:there is a lock in the partition context. The lock is not taken
before we call any of the scheduling steps. This might need to be added, a
combination of read and write locking while doing things might be needed.. That
would make sure that the configuration update could also take the lock. If a
config change uses a lock it can thus be applied in between the scheduling
steps which each could take a lock on the partition.
Point 4: the fact that it is stateless means it most likely misses things. The
existence of applications in a leaf queue is one point that should be checked.
The whole type change of the queue requires a stateful check.
This Jira might need to be changed into a task with subtask as this is not all
solvable in one "small" change.
> Allow changing queue type: parent to leaf and back
> --------------------------------------------------
>
> Key: YUNIKORN-1546
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1546
> Project: Apache YuniKorn
> Issue Type: New Feature
> Components: core - common
> Reporter: Wilfred Spiegelenburg
> Assignee: Peter Bacsko
> Priority: Major
> Attachments: TestUpdateQueues2.patch
>
>
> In certain cases we would like to allow changing the queue type from a parent
> to a leaf or the other way around in one update.
> Currently there is no way to change the type and it will throw an error when
> tried. That means two updates need to be made to first remove and then add
> the same queue with a different type. This is only important for managed
> queues, i.e. queues in the config. Dynamic queues should not allow type
> changes as they are removed when empty.
> * parent to leaf: only if no children are registered for the parent
> * leaf to parent: only of no applications are running in the queue
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
