[
https://issues.apache.org/jira/browse/SPARK-26973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16775394#comment-16775394
]
shane knapp commented on SPARK-26973:
-------------------------------------
i was chatting over email w/[~eje] about this yesterday, and the TL;DR is:
only one version to test against, please!
here are some bullet points, in no particular order, to summarize what ~[~eje]
and i discussed:
* we can easily test against any version of k8s via the
{{--kubernetes-version}} flag passed to {{minikube start}}, so testing against
N versions shouldn't be hard.
* there is a moving range of k8s versions that a specific minikube release can
support (ie: minikube v.0.23.0 only supports up to k8s 1.13.1).
* we are limited to *one* k8s/minikube build per node at any time, so adding
tests for more than one k8s version to the suite will definitely increase
resource contention. currently spark is the only minikube consumer, but some
upcoming lab projects will need their own k8s integration tests.
* the operational overhead of managing minikube, k8s and all of the VM-layer
drivers is highly non-trivial.
> Kubernetes version support strategy on test nodes / backend
> -----------------------------------------------------------
>
> Key: SPARK-26973
> URL: https://issues.apache.org/jira/browse/SPARK-26973
> Project: Spark
> Issue Type: Test
> Components: Kubernetes
> Affects Versions: 3.0.0
> Reporter: Stavros Kontopoulos
> Priority: Major
>
> Kubernetes has a policy for supporting three minor releases and the current
> ones are defined here:
> [https://github.com/kubernetes/sig-release/blob/master/releases/patch-releases.md.|https://github.com/kubernetes/sig-release/blob/master/releases/patch-releases.md]
> Moving from release 1.x to 1.(x+1) happens roughly every 100
> days:[https://gravitational.com/blog/kubernetes-release-cycle.]
> This has an effect on dependencies upgrade at the Spark on K8s backend and
> the version of Minikube required to be supported for testing. One other issue
> is what the users actually want at the given time of a release. Some popular
> vendors like EKS([https://aws.amazon.com/eks/faqs/]) have their own roadmap
> for releases and may not catch up fast (what is our view on this).
> Follow the comments for a recent discussion on the topic:
> [https://github.com/apache/spark/pull/23814.]
> Clearly we need a strategy for this.
> A couple of options for the current state of things:
> a) Support only the last two versions, but that leaves out a version that
> still receives patches.
> b) Support only the latest, which makes testing easier, but leaves out other
> currently maintained version.
> A good strategy will optimize at least the following:
> 1) percentage of users satisfied at release time.
> 2) how long it takes to support the latest K8s version
> 3) testing requirements eg. minikube versions used
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
