[
https://issues.apache.org/jira/browse/YUNIKORN-3075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mit Desai updated YUNIKORN-3075:
--------------------------------
Description:
We have noticed that when the Kubernetes cluster is upgraded to version 1.30,
the installed version of YuniKorn continues to run without any issues. However,
after the upgrade, if the admission controller has to be redeployed, the
webhooks will be recreated, which could potentially cause problems.
At Visa, we use a modified version of the webhook that includes a namespaced
scope for scheduling YuniKorn pods in specific namespaces. When the admission
controller recreates the webhooks after redeployment, the code built on Go 1.29
fails to successfully create the webhooks with the namespaced scope. This issue
is silently ignored, and the webhooks are created with a cluster-level scope
instead. As a result, all applications in our clusters started being scheduled
by YuniKorn, even in namespaces where they should not be scheduled.
{*}Steps to Reproduce{*}:
# YuniKorn v1.5.2 running on k8s 1.29 (Webhooks code modified to add
Namespaced scope)
# Upgrade a Kubernetes cluster to version 1.30.
# Redeploy the admission controller.
# Observe the scope of the created webhooks.
{*}Expected Behavior{*}: The webhooks should be created with a namespaced scope
as configured in our modified version of the webhook.
{*}Actual Behavior{*}: The webhooks are created with a cluster-level scope,
causing all applications to be scheduled by YuniKorn across all namespaces.
{*}Impact{*}: This issue causes all applications in our clusters to be
scheduled by YuniKorn, even in namespaces where they should not be scheduled,
potentially leading to resource management issues and unintended behavior.
{*}Environment Details{*}:
* Kubernetes version: 1.30
* YuniKorn version: 1.5.2
* Go version: 1.29
* Modified webhook configuration with namespaced scope
was:
We have noticed that when the kubernetes cluster is upgraded to 1.30, Installed
version of YuniKorn does not get affected. It runs without any issues.
However, after the upgrade, if the admission controller has to be redeployed,
the webhooks will be recreated and this could potentially cause problems.
At Visa, we use a modified version of the webhook where we also have a
namespaed scope to schedule yunikorn pods in specific namespaces. When the
admission controller created the webhooks after redeployment, the code built on
go 1.29 was not able to successfully create the webhooks with namespaced scope.
This issue was silently ignored and the webhooks were created with the cluster
level scope. So all the applications in our clusters started to get scheduled
by YuniKorn where they should not have.
> YuniKorn (1.5.2) Webhooks fail after admission controller redeployment
> k8s-1.30
> -------------------------------------------------------------------------------
>
> Key: YUNIKORN-3075
> URL: https://issues.apache.org/jira/browse/YUNIKORN-3075
> Project: Apache YuniKorn
> Issue Type: Bug
> Components: shim - kubernetes
> Affects Versions: 1.5.2
> Reporter: Mit Desai
> Priority: Major
>
> We have noticed that when the Kubernetes cluster is upgraded to version 1.30,
> the installed version of YuniKorn continues to run without any issues.
> However, after the upgrade, if the admission controller has to be redeployed,
> the webhooks will be recreated, which could potentially cause problems.
> At Visa, we use a modified version of the webhook that includes a namespaced
> scope for scheduling YuniKorn pods in specific namespaces. When the admission
> controller recreates the webhooks after redeployment, the code built on Go
> 1.29 fails to successfully create the webhooks with the namespaced scope.
> This issue is silently ignored, and the webhooks are created with a
> cluster-level scope instead. As a result, all applications in our clusters
> started being scheduled by YuniKorn, even in namespaces where they should not
> be scheduled.
> {*}Steps to Reproduce{*}:
> # YuniKorn v1.5.2 running on k8s 1.29 (Webhooks code modified to add
> Namespaced scope)
> # Upgrade a Kubernetes cluster to version 1.30.
> # Redeploy the admission controller.
> # Observe the scope of the created webhooks.
> {*}Expected Behavior{*}: The webhooks should be created with a namespaced
> scope as configured in our modified version of the webhook.
> {*}Actual Behavior{*}: The webhooks are created with a cluster-level scope,
> causing all applications to be scheduled by YuniKorn across all namespaces.
> {*}Impact{*}: This issue causes all applications in our clusters to be
> scheduled by YuniKorn, even in namespaces where they should not be scheduled,
> potentially leading to resource management issues and unintended behavior.
> {*}Environment Details{*}:
> * Kubernetes version: 1.30
> * YuniKorn version: 1.5.2
> * Go version: 1.29
> * Modified webhook configuration with namespaced scope
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
