Hello Thimme,
Your issue is related to
https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122
Deprecated API Migration Guide |
Kubernetes<https://kubernetes.io/docs/reference/using-api/deprecation-guide/#ingress-v122>
As the Kubernetes API evolves, APIs are periodically reorganized or upgraded.
When APIs evolve, the old API is deprecated and eventually removed. This page
contains information you need to know when migrating from deprecated API
versions to newer and more stable API versions. Removed APIs by release v1.29
The v1.29 release will stop serving the following deprecated API versions: Flow
control ...
kubernetes.io
You will need to upgrade your endpoint to use the new general available
endpoint.
Regards
Saurabh Gulati
________________________________
From: Thimme Gowda TP (Nokia) <thimme.gowda...@nokia.com>
Sent: 23 December 2022 11:31
To: user@spark.apache.org <user@spark.apache.org>
Subject: [EXTERNAL] spark-submit fails in kubernetes 1.24.x cluster
Caution! This email originated outside of FedEx. Please do not open attachments
or click links from an unknown or suspicious origin.
Hello,
We are facing issue with ingress during spark-submit with kubernetes cluster
1.24.x .
We are using spark 3.3.0 to do spark-submit.
# kubectl version
WARNING: This version information is deprecated and will be replaced with the
output from kubectl version --short. Use --output=yaml|json to get the full
version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9",
GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean",
BuildDate:"2022-12-08T10:15:09Z", GoVersion:"go1.18.9", Compiler:"gc",
Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.9",
GitCommit:"9710807c82740b9799453677c977758becf0acbb", GitTreeState:"clean",
BuildDate:"2022-12-08T10:08:06Z", GoVersion:"go1.18.9", Compiler:"gc",
Platform:"linux/amd64"}
Error:
{"type":"log", "level":"WARN", "time":"2022-12-23T10:04:57.536Z",
"timezone":"UTC", "log":"The client is using resource type 'ingresses' with
unstable version 'v1beta1'"}
Exception in thread "main"
io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST
at:
https://xx.xx.xx.xx:6443/apis/extensions/v1beta1/namespaces/test/ingresses<https://urldefense.com/v3/__https://xx.xx.xx.xx:6443/apis/extensions/v1beta1/namespaces/test/ingresses__;!!BL9GA0TyTA!YkATz9TvT8G7FBX1Vox2KJJu9Tz3DV3_InVLgRMsjiNjd88IseiBHRpDzSplA9w5xhtRlNsvUGIaqU57r_hj9bhBaExnLA$>.
Message: Not Found.
>From spark code we see that spark 3.3.0 is using
>“<kubernetes-client.version>5.12.2</kubernetes-client.version>”
We also tried changing this to 6.x as in PR :
https://github.com/apache/spark/pull/37990/files<https://urldefense.com/v3/__https://github.com/apache/spark/pull/37990/files__;!!BL9GA0TyTA!YkATz9TvT8G7FBX1Vox2KJJu9Tz3DV3_InVLgRMsjiNjd88IseiBHRpDzSplA9w5xhtRlNsvUGIaqU57r_hj9bi-yg9xnQ$>
Still we facing same issue.
Let us know if spark is tested/working with k8s 1.24.x cluster and what could
be the solution to resolve this.
Thanks.
Regards
Thimme Gowda
