ahmadamirahmadi1401 commented on issue #10948:
URL: https://github.com/apache/cloudstack/issues/10948#issuecomment-2936776869
Yes, but I did not make any changes to the file and this is the default
behavior when upgrading.
This happens for other versions as well. For example, the following error
was obtained for upgrading a Kubernetes cluster from version 1.30.5 to version
1.31.8. And in the end, it failed to unmount the ISO and got an alert.
2025-06-03 18:40:17,001 ERROR [c.c.u.s.SshHelper]
(API-Job-Executor-34:ctx-a226f7b8 job-32868 ctx-5a7f130c) (logid:f244ee01) SSH
execution of command sudo ./upgrade-kubernetes.sh13249113815295454536.sh 1.31.8
true false false has an error status code in return. Result output: Installing
binaries from /mnt/k8sdisk/
unpacking quay.io/apalia/cloudstack-csi-driver:0.0.2
(sha256:6f38051b27964da06af1cbcf1c759b4eb5266bdd02033f07c3d2e04630e2893f)...done
unpacking ghcr.io/leaseweb/cloudstack-csi-driver:0.8.1
(sha256:7a6cf3ba95be182ee8c991dced373f56faacbfac9a719056586d98af3aa932c5)...done
unpacking docker.io/apache/cloudstack-kubernetes-autoscaler:latest
(sha256:1c8a22c342daa5884f622f078be70ff913599aada0c4f859fe42ed28413afe98)...done
unpacking docker.io/apache/cloudstack-kubernetes-provider:v1.1.0
(sha256:10c058968e7d2f8e55da5976336a42d88b057e75880f000d3120aa2d75649e97)...done
unpacking registry.k8s.io/coredns/coredns:v1.11.3
(sha256:9caabbf6238b189a65d0d6e6ac138de60d6a1c419e5a341fbbb7c78382559c6e)...done
unpacking k8s.gcr.io/sig-storage/csi-attacher:v3.0.2
(sha256:6f80b12657a7e0a5c683b24e806c4bbbe33a43e39b041fe9b7514d665d478ea4)...done
unpacking registry.k8s.io/sig-storage/csi-attacher:v4.6.1
(sha256:b4d611100ece2f9bc980d1cb19c2285b8868da261e3b1ee8f45448ab5512ab94)...done
unpacking k8s.gcr.io/sig-storage/csi-node-driver-registrar:v2.0.1
(sha256:e07f914c32f0505e4c470a62a40ee43f84cbf8dc46ff861f31b14457ccbad108)...done
unpacking registry.k8s.io/sig-storage/csi-node-driver-registrar:v2.10.1
(sha256:f25af73ee708ff9c82595ae99493cdef9295bd96953366cddf36305f82555dac)...done
unpacking k8s.gcr.io/sig-storage/csi-provisioner:v2.0.4
(sha256:bec571992d40203edcd056ac0b0d97003887ee5e4be144c41932d18639673b03)...done
unpacking registry.k8s.io/sig-storage/csi-provisioner:v5.0.1
(sha256:405a14e1aa702f7ea133cea459e8395fe40a6125c088c55569e696d48e1bd385)...done
unpacking registry.k8s.io/sig-storage/csi-resizer:v1.11.1
(sha256:a541e6cc2d8b011bb21b1d4ffec6b090e85270cce6276ee302d86153eec0af43)...done
unpacking docker.io/kubernetesui/dashboard:v2.7.0
(sha256:2e500d29e9d5f4a086b908eb8dfe7ecac57d2ab09d65b24f588b1d449841ef93)...done
unpacking registry.k8s.io/etcd:3.5.15-0
(sha256:a6dc63e6e8cfa0307d7851762fa6b629afb18f28d8aa3fab5a6e91b4af60026a)...done
unpacking registry.k8s.io/kube-apiserver:v1.31.8
(sha256:30090db6a7d53799163ce82dae9e8ddb645fd47db93f2ec9da0cc787fd825625)...done
unpacking registry.k8s.io/kube-controller-manager:v1.31.8
(sha256:29eaddc64792a689df48506e78bbc641d063ac8bb92d2e66ae2ad05977420747)...done
unpacking registry.k8s.io/kube-proxy:v1.31.8
(sha256:dd0c9a37670f209947b1ed880f06a2e93e1d41da78c037f52f94b13858769838)...done
unpacking registry.k8s.io/kube-scheduler:v1.31.8
(sha256:22994a2632e81059720480b9f6bdeb133b08d58492d0b36dfd6e9768b159b22a)...done
unpacking registry.k8s.io/sig-storage/livenessprobe:v2.12.0
(sha256:5baeb4a6d7d517434292758928bb33efc6397368cbb48c8a4cf29496abf4e987)...done
unpacking docker.io/kubernetesui/metrics-scraper:v1.0.8
(sha256:76049887f07a0476dc93efc2d3569b9529bf982b22d29f356092ce206e98765c)...done
unpacking registry.k8s.io/pause:3.10
(sha256:ee6521f290b2168b6e0935a181d4cff9be1ac3f505666ef0e3c98fae8199917a)...done
unpacking registry.mainx.com/weaveworks/weave-kube:latest
(sha256:35827a9c549c095f0e9d1cf8b35d8f27ae2c76e31bc6f7f3c0bc95911d5accea)...done
unpacking registry.mainx.com/weaveworks/weave-npc:latest
(sha256:062832fd25b5e9e16650e618f26bba1409a7b3bf2c3903e1b369d788abc63aef)...done
registry.k8s.io/pause:3.9
[preflight] Running pre-flight checks.
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n
kube-system get cm kubeadm-config -o yaml'
[upgrade] Running cluster health checks
[preflight] Running pre-flight checks.
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n
kube-system get cm kubeadm-config -o yaml'
[upgrade] Running cluster health checks
====================
2025-06-03 18:43:04,106 ERROR [c.c.k.c.a.KubernetesClusterActionWorker]
(API-Job-Executor-34:ctx-a226f7b8 job-32868 ctx-5a7f130c) (logid:f244ee01)
Failed to upgrade Kubernetes cluster :
e1faffb3e614e6c2fba74296962386b7-01jwvgydkaj5jket7z5qmqnsfe, unable to upgrade
Kubernetes node on VM :
e1faffb3e614e6c2fba74296962386b7-01jwvgy-control-197370ff089
com.cloud.utils.exception.CloudRuntimeException: Failed to upgrade
Kubernetes cluster :
e1faffb3e614e6c2fba74296962386b7-01jwvgydkaj5jket7z5qmqnsfe, unable to upgrade
Kubernetes node on VM :
e1faffb3e614e6c2fba74296962386b7-01jwvgy-control-197370ff089
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterActionWorker.logTransitStateDetachIsoAndThrow(KubernetesClusterActionWorker.java:266)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterUpgradeWorker.upgradeKubernetesClusterNodes(KubernetesClusterUpgradeWorker.java:127)
at
com.cloud.kubernetes.cluster.actionworkers.KubernetesClusterUpgradeWorker.upgradeCluster(KubernetesClusterUpgradeWorker.java:177)
at
com.cloud.kubernetes.cluster.KubernetesClusterManagerImpl.upgradeKubernetesCluster(KubernetesClusterManagerImpl.java:1622)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at
org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
at com.sun.proxy.$Proxy466.upgradeKubernetesCluster(Unknown Source)
at
org.apache.cloudstack.api.command.user.kubernetes.cluster.UpgradeKubernetesClusterCmd.execute(UpgradeKubernetesClusterCmd.java:114)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:172)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:112)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:654)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:52)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:45)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:602)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
=============================
2025-06-03 18:43:04,685 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-34:ctx-a226f7b8 job-32868) (logid:f244ee01) Complete async
job-32868, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed
to upgrade Kubernetes cluster :
e1faffb3e614e6c2fba74296962386b7-01jwvgydkaj5jket7z5qmqnsfe, unable to upgrade
Kubernetes node on VM :
e1faffb3e614e6c2fba74296962386b7-01jwvgy-control-197370ff089"}
@weizhouapache
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
