[
https://issues.apache.org/jira/browse/YUNIKORN-1196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17531921#comment-17531921
]
Craig Condit edited comment on YUNIKORN-1196 at 9/27/22 12:09 AM:
------------------------------------------------------------------
I've now successfully built against both 1.23.6 and 1.24.0. However, this is
going to have implications for minimum supported versions, especially in plugin
mode.
We currently officially support running on 1.20 -> 1.24. 1.19 is also known to
be functional but not tested regularly.
The main issue of compatibility has to do with the various API server versions
of key resources that are supported. Here is a summary of the relevant
resources that have been updated in each version:
* 1.20 -> 1.21
** PodDisruptionBudget: v1beta1 -> v1
** CSIStorageCapacity: v1alpha1 -> v1beta1 (v1alpha1 still supported)
* 1.21 -> 1.22
** CSIStorageCapacity: v1alpha1 support dropped
* 1.22 -> 1.23
** no major changes
* 1.23 -> 1.24
** CSIStorageCapacity: v1beta1 -> v1 (v1beta1 still supported)
* 1.24 -> 1.25
** PodDisruptionBudget: v1beta1 support dropped
PodSecurityPolicies: all versions dropped
CSIStorageCapacity is the biggest issue. The informers initialized as part of
the default scheduler framework use the latest supported version as of the K8s
version they are compiled against. Additionally, the framework waits for all
informers to finish registering before starting pod scheduling. The consequence
of this is that if we compile against 1.23, the scheduler hangs on startup with
K8s API server < 1.21 (as v1beta1 is not available). Compiling against 1.24 is
even worse, as that uses v1, which is only available on 1.24 and later.
Long story short: Compiling against 1.23 is going to force us to drop support
for 1.20 and earlier. 1.20 is officially end-of-life, however it is still in
wide use.
Compiling against 1.24 is going to drop support for 1.23 and earlier, which we
cannot even think about doing until next year (1.23 is supported upstream
through 28 Feb 2023).
Update (September 2022): Now that 1.25 is out and PodDisruptionBudget v1beta1
is no more, we have disabled the *PodDisruptionBudget* feature flag, which
cannot be done when compiling with > 1.20, does work, at least in the current
1.20 running against 1.25.
Given 1.20 is EOL, we should probably re-target 1.23 as a baseline. This gives
some benefits in the plugin framework, while still supporting 1.21 through 1.25
without ugly hacks.
was (Author: ccondit):
I've now successfully built against both 1.23.6 and 1.24.0. However, this is
going to have implications for minimum supported versions, especially in plugin
mode.
We currently officially support running on 1.20 -> 1.24. 1.19 is also known to
be functional but not tested regularly.
The main issue of compatibility has to do with the various API server versions
of key resources that are supported. Here is a summary of the relevant
resources that have been updated in each version:
* 1.20 -> 1.21
** PodDisruptionBudget: v1beta1 -> v1
** CSIStorageCapacity: v1alpha1 -> v1beta1 (v1alpha1 still supported)
* 1.21 -> 1.22
** CSIStorageCapacity: v1alpha1 support dropped
* 1.22 -> 1.23
** no major changes
* 1.23 -> 1.24
** CSIStorageCapacity: v1beta1 -> v1 (v1beta1 still supported)
* 1.24 -> 1.25
** PodDisruptionBudget: REMOVED
CSIStorageCapacity is the biggest issue. The informers initialized as part of
the default scheduler framework use the latest supported version as of the K8s
version they are compiled against. Additionally, the framework waits for all
informers to finish registering before starting pod scheduling. The consequence
of this is that if we compile against 1.23, the scheduler hangs on startup with
K8s API server < 1.21 (as v1beta1 is not available). Compiling against 1.24 is
even worse, as that uses v1, which is only available on 1.24 and later.
Long story short: Compiling against 1.23 is going to force us to drop support
for 1.20 and earlier. 1.20 is officially end-of-life, however it is still in
wide use.
Compiling against 1.24 is going to drop support for 1.23 and earlier, which we
cannot even think about doing until next year (1.23 is supported upstream
through 28 Feb 2023).
Update (September 2022): Now that 1.25 is out and PodDisruptionBudget is no
more, supporting 1.25 in plugin mode *requires* compiling against Kubernetes
1.25 or later. As before, doing this will drop support for Kubernetes 1.24 or
earlier. So, we are now in a scenario where we can support K8s 1.20-1.24, or
1.24-1.25, but not both with the same codebase.
Alternatively, disabling the *PodDisruptionBudget* feature flag, which cannot
be done when compiled against 1.23 or 1.24, does work (at least in the current
1.20 running against 1.25).
It seems for the moment, that we're stuck compiling against 1.20 for a while
yet.
> [UMBRELLA] Upgrade K8s build dependency
> ---------------------------------------
>
> Key: YUNIKORN-1196
> URL: https://issues.apache.org/jira/browse/YUNIKORN-1196
> Project: Apache YuniKorn
> Issue Type: Task
> Components: shim - kubernetes
> Reporter: Wilfred Spiegelenburg
> Assignee: Craig Condit
> Priority: Major
>
> The current build for YuniKorn uses 1.20.11. Even a small change to K8s 1.21
> causes compilation issues.
> Some issues can be solved by a simple one or two code line change.
> There is at least one that requires a major change in the predicate code to
> accomodate fundamental changes made to the plugin framework.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
