i succeded to install systemvm template 4.19 as serving template for control and worker nodes with cks community iso 1.28.4 when cluster saying "starting", i connect to node controller via p2222, saw that docker was not installed, but containerd.io yes
I've done the following : apt install docker-ce cp /etc/containerd/config.toml /etc/containerd/config.toml.bck containerd config default | tee /etc/containerd/config.toml /opt/bin/setup-kube-system /opt/bin/deploy-kube-system on to control node and same after on worker node CS UI show kubernetes yaml a "kubectl.exe --kubeconfig kube.conf.conf get nodes" NAME STATUS ROLES AGE VERSION k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4 k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4 However CS says the cluster is in alert state And dashboad is not working Any advice ? when executing this on my laptop : kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy and opening this : http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ I have this result : { "kind": "Status", "apiVersion": "v1", "metadata": {}, "status": "Failure", "message": "no endpoints available for service \"kubernetes-dashboard\"", "reason": "ServiceUnavailable", "code": 503 } Somebody got this problem with dashboard ? Did i miss something installing the cluster manually with acs stuff ? Le jeu. 11 avr. 2024 à 17:40, benoit lair <kurushi4...@gmail.com> a écrit : > Hi Wei, > > Thanks for the sharing, i also tried to install the systemvm 4.19 > I have control node and worker node under template systemvm 4.19 ( > http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2 > ) > I tried with community cks iso 1.25.0 and 1.28.9 > On systemvm 4.19 docker was not present by default, just containerd in > 1.6.x version > I installed docker-ce and docker-ce-cli : > > apt install -y apt-transport-https ca-certificates curl > software-properties-common > curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg > --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg > echo "deb [arch=$(dpkg --print-architecture) > signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] > https://download.docker.com/linux/debian $(lsb_release -cs) stable" | > sudo tee /etc/apt/sources.list.d/docker.list > /dev/null > remove line about docker in double in /etc/apt/sources.list > apt update > apt install -y docker-ce > apt install -y docker-cli > i mounted the 1.28.4 cks community iso in control vm > > and /opt/bin/setup-kube-system > i have these errors : > > Is it the good way to manually install kube system > > root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core# > /opt/bin/setup-kube-system > Installing binaries from /mnt/k8sdisk/ > 5b1fa8e3e100: Loading layer > [==================================================>] 803.8kB/803.8kB > 39c831b1aa26: Loading layer > [==================================================>] 26.25MB/26.25MB > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest > 417cb9b79ade: Loading layer > [==================================================>] 657.7kB/657.7kB > 8d323b160d65: Loading layer > [==================================================>] 24.95MB/24.95MB > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0 > 6a4a177e62f3: Loading layer > [==================================================>] 115.2kB/115.2kB > 398c9baff0ce: Loading layer > [==================================================>] 16.07MB/16.07MB > Loaded image: registry.k8s.io/coredns/coredns:v1.10.1 > bd8a70623766: Loading layer > [==================================================>] 75.78MB/75.78MB > c88361932af5: Loading layer > [==================================================>] 508B/508B > Loaded image: kubernetesui/dashboard:v2.7.0 > e023e0e48e6e: Loading layer > [==================================================>] 103.7kB/103.7kB > 6fbdf253bbc2: Loading layer > [==================================================>] 21.2kB/21.2kB > 7bea6b893187: Loading layer > [==================================================>] 716.5kB/716.5kB > ff5700ec5418: Loading layer > [==================================================>] 317B/317B > d52f02c6501c: Loading layer > [==================================================>] 198B/198B > e624a5370eca: Loading layer > [==================================================>] 113B/113B > 1a73b54f556b: Loading layer > [==================================================>] 385B/385B > d2d7ec0f6756: Loading layer > [==================================================>] 355B/355B > 4cb10dd2545b: Loading layer > [==================================================>] 130.6kB/130.6kB > ba9afb2b3e0c: Loading layer > [==================================================>] 1.076MB/1.076MB > 22bba3da6b0d: Loading layer > [==================================================>] 91.29MB/91.29MB > a4563151d59b: Loading layer > [==================================================>] 9.544MB/9.544MB > Loaded image: registry.k8s.io/etcd:3.5.9-0 > 54ad2ec71039: Loading layer > [==================================================>] 103.7kB/103.7kB > 6fbdf253bbc2: Loading layer > [==================================================>] 21.2kB/21.2kB > 7bea6b893187: Loading layer > [==================================================>] 716.5kB/716.5kB > ff5700ec5418: Loading layer > [==================================================>] 317B/317B > d52f02c6501c: Loading layer > [==================================================>] 198B/198B > e624a5370eca: Loading layer > [==================================================>] 113B/113B > 1a73b54f556b: Loading layer > [==================================================>] 385B/385B > d2d7ec0f6756: Loading layer > [==================================================>] 355B/355B > 4cb10dd2545b: Loading layer > [==================================================>] 130.6kB/130.6kB > 3f3c83f9a2fa: Loading layer > [==================================================>] 724.8kB/724.8kB > 6a2c4bedb8de: Loading layer > [==================================================>] 32.98MB/32.98MB > Loaded image: registry.k8s.io/kube-apiserver:v1.28.4 > e0b89915180d: Loading layer > [==================================================>] 31.72MB/31.72MB > Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4 > d2a4b045f3d0: Loading layer > [==================================================>] 7.73MB/7.73MB > 3905b20a2df2: Loading layer > [==================================================>] 16.85MB/16.85MB > Loaded image: registry.k8s.io/kube-proxy:v1.28.4 > f35f4c1ae44f: Loading layer > [==================================================>] 17.13MB/17.13MB > Loaded image: registry.k8s.io/kube-scheduler:v1.28.4 > d01384fea991: Loading layer > [==================================================>] 19.74MB/19.74MB > bcec7eb9e567: Loading layer > [==================================================>] 530B/530B > Loaded image: kubernetesui/metrics-scraper:v1.0.8 > e3e5579ddd43: Loading layer > [==================================================>] 317.6kB/317.6kB > Loaded image: registry.k8s.io/pause:3.9 > 1b3ee35aacca: Loading layer > [==================================================>] 2.796MB/2.796MB > 910ce076f504: Loading layer > [==================================================>] 7.425MB/7.425MB > a8e8b7b8e08a: Loading layer > [==================================================>] 8.101MB/8.101MB > 084e56a9c24b: Loading layer > [==================================================>] 4.376MB/4.376MB > 334d70dc85ec: Loading layer > [==================================================>] 159B/159B > a13197d8bda5: Loading layer > [==================================================>] 8.216MB/8.216MB > Loaded image: weaveworks/weave-kube:2.8.1 > 998efb010df6: Loading layer > [==================================================>] 728.2kB/728.2kB > ed37391def99: Loading layer > [==================================================>] 9.28MB/9.28MB > 175a472c5f77: Loading layer > [==================================================>] 299B/299B > a8764e32e9fe: Loading layer > [==================================================>] 613B/613B > Loaded image: weaveworks/weave-npc:2.8.1 > net.bridge.bridge-nf-call-iptables = 1 > [init] Using Kubernetes version: v1.28.4 > [preflight] Running pre-flight checks > error execution phase preflight: [preflight] Some fatal errors occurred: > [ERROR CRI]: container runtime is not running: output: > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection: > validate CRI v1 runtime API for endpoint > \"unix:///var/run/containerd/containerd.sock\": rpc error: code = > Unimplemented desc = unknown service runtime.v1.RuntimeService" > , error: exit status 1 > [preflight] If you know what you are doing, you can make a check non-fatal > with `--ignore-preflight-errors=...` > To see the stack trace of this error execute with --v=5 or higher > [init] Using Kubernetes version: v1.28.4 > [preflight] Running pre-flight checks > error execution phase preflight: [preflight] Some fatal errors occurred: > [ERROR CRI]: container runtime is not running: output: > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection: > validate CRI v1 runtime API for endpoint > \"unix:///var/run/containerd/containerd.sock\": rpc error: code = > Unimplemented desc = unknown service runtime.v1.RuntimeService" > , error: exit status 1 > [preflight] If you know what you are doing, you can make a check non-fatal > with `--ignore-preflight-errors=...` > To see the stack trace of this error execute with --v=5 or higher > [init] Using Kubernetes version: v1.28.4 > [preflight] Running pre-flight checks > error execution phase preflight: [preflight] Some fatal errors occurred: > [ERROR CRI]: container runtime is not running: output: > time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection: > validate CRI v1 runtime API for endpoint > \"unix:///var/run/containerd/containerd.sock\": rpc error: code = > Unimplemented desc = unknown service runtime.v1.RuntimeService" > , error: exit status 1 > [preflight] If you know what you are doing, you can make a check non-fatal > with `--ignore-preflight-errors=...` > To see the stack trace of this error execute with --v=5 or higher > Error: kubeadm init failed! > > > > > > Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <ustcweiz...@gmail.com> a écrit : > >> Hi, >> >> Please refer to >> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241 >> >> The containerd in 4.16/4.17 sysyemvm template is too old. >> >> -Wei >> >> >> On Thursday, April 11, 2024, benoit lair <kurushi4...@gmail.com> wrote: >> >> > Hi Rohit, >> > >> > I already tested with different iso from cloudstack from CKS repos, >> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from >> 1.24.0 >> > and following others versions >> > >> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails, >> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3 >> > and 1.28.4 >> > >> > Last try, i tested k8s community iso 1.25.0 >> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in >> progress" >> > and it fails >> > If i try to ssh on control node with VR_Pub_IP port 2222 >> > i have nothing in execution >> > >> > If i try manual install : >> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo >> /opt/bin/setup-kube-system >> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk. >> > Installing binaries from /mnt/k8sdisk/ >> > 5b1fa8e3e100: Loading layer >> > [==================================================>] 803.8kB/803.8kB >> > 39c831b1aa26: Loading layer >> > [==================================================>] 26.25MB/26.25MB >> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest >> > 417cb9b79ade: Loading layer >> > [==================================================>] 657.7kB/657.7kB >> > 8d323b160d65: Loading layer >> > [==================================================>] 24.95MB/24.95MB >> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0 >> > 256bc5c338a6: Loading layer >> > [==================================================>] 122.2kB/122.2kB >> > df1818f16337: Loading layer >> > [==================================================>] 14.71MB/14.71MB >> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3 >> > bd8a70623766: Loading layer >> > [==================================================>] 75.78MB/75.78MB >> > c88361932af5: Loading layer >> > [==================================================>] 508B/508B >> > Loaded image: kubernetesui/dashboard:v2.7.0 >> > 0b031aac6569: Loading layer >> > [==================================================>] 804kB/804kB >> > cc0597072b83: Loading layer >> > [==================================================>] 1.076MB/1.076MB >> > 5a8045f731fc: Loading layer >> > [==================================================>] 90.58MB/90.58MB >> > f5a5ef1a005f: Loading layer >> > [==================================================>] 1.188MB/1.188MB >> > 5cf3bfe5417d: Loading layer >> > [==================================================>] 8.506MB/8.506MB >> > Loaded image: registry.k8s.io/etcd:3.5.4-0 >> > 8d7366c22fd8: Loading layer >> > [==================================================>] 804.1kB/804.1kB >> > 7d5f5bba7c45: Loading layer >> > [==================================================>] 694.9kB/694.9kB >> > 8154c9483a6c: Loading layer >> > [==================================================>] 32.72MB/32.72MB >> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0 >> > c5f1a47b47bc: Loading layer >> > [==================================================>] 29.76MB/29.76MB >> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0 >> > e4d19dd12df2: Loading layer >> > [==================================================>] 7.631MB/7.631MB >> > ba3637ccd78a: Loading layer >> > [==================================================>] 12.63MB/12.63MB >> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0 >> > 026969d859f7: Loading layer >> > [==================================================>] 14.29MB/14.29MB >> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0 >> > d01384fea991: Loading layer >> > [==================================================>] 19.74MB/19.74MB >> > bcec7eb9e567: Loading layer >> > [==================================================>] 530B/530B >> > Loaded image: kubernetesui/metrics-scraper:v1.0.8 >> > 961e93cda9dd: Loading layer >> > [==================================================>] 307kB/307kB >> > Loaded image: registry.k8s.io/pause:3.8 >> > 1b3ee35aacca: Loading layer >> > [==================================================>] 2.796MB/2.796MB >> > 17d89e2a5199: Loading layer >> > [==================================================>] 7.425MB/7.425MB >> > 148b94859892: Loading layer >> > [==================================================>] 8.1MB/8.1MB >> > dbea472c1ced: Loading layer >> > [==================================================>] 4.376MB/4.376MB >> > e81f1a367c17: Loading layer >> > [==================================================>] 131B/131B >> > 351848f6b295: Loading layer >> > [==================================================>] 8.216MB/8.216MB >> > Loaded image: weaveworks/weave-kube:latest >> > b20f7ff12b24: Loading layer >> > [==================================================>] 727.9kB/727.9kB >> > 1d49e386162e: Loading layer >> > [==================================================>] 9.281MB/9.281MB >> > 20145eeac146: Loading layer >> > [==================================================>] 297B/297B >> > d4cc72748587: Loading layer >> > [==================================================>] 600B/600B >> > Loaded image: weaveworks/weave-npc:latest >> > net.bridge.bridge-nf-call-iptables = 1 >> > [init] Using Kubernetes version: v1.25.0 >> > [preflight] Running pre-flight checks >> > [WARNING SystemVerification]: missing optional cgroups: blkio >> > error execution phase preflight: [preflight] Some fatal errors occurred: >> > [ERROR CRI]: container runtime is not running: output: E0411 >> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime >> service >> > failed" err="rpc error: code = Unimplemented desc = unknown service >> > runtime.v1alpha2.RuntimeService" >> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: >> rpc >> > error: code = Unimplemented desc = unknown service >> > runtime.v1alpha2.RuntimeService" >> > , error: exit status 1 >> > [preflight] If you know what you are doing, you can make a check >> non-fatal >> > with `--ignore-preflight-errors=...` >> > To see the stack trace of this error execute with --v=5 or higher >> > [init] Using Kubernetes version: v1.25.0 >> > [preflight] Running pre-flight checks >> > [WARNING SystemVerification]: missing optional cgroups: blkio >> > error execution phase preflight: [preflight] Some fatal errors occurred: >> > [ERROR CRI]: container runtime is not running: output: E0411 >> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime >> service >> > failed" err="rpc error: code = Unimplemented desc = unknown service >> > runtime.v1alpha2.RuntimeService" >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: >> rpc >> > error: code = Unimplemented desc = unknown service >> > runtime.v1alpha2.RuntimeService" >> > , error: exit status 1 >> > [preflight] If you know what you are doing, you can make a check >> non-fatal >> > with `--ignore-preflight-errors=...` >> > To see the stack trace of this error execute with --v=5 or higher >> > [init] Using Kubernetes version: v1.25.0 >> > [preflight] Running pre-flight checks >> > [WARNING SystemVerification]: missing optional cgroups: blkio >> > error execution phase preflight: [preflight] Some fatal errors occurred: >> > [ERROR CRI]: container runtime is not running: output: E0411 >> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime >> service >> > failed" err="rpc error: code = Unimplemented desc = unknown service >> > runtime.v1alpha2.RuntimeService" >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: >> rpc >> > error: code = Unimplemented desc = unknown service >> > runtime.v1alpha2.RuntimeService" >> > , error: exit status 1 >> > [preflight] If you know what you are doing, you can make a check >> non-fatal >> > with `--ignore-preflight-errors=...` >> > To see the stack trace of this error execute with --v=5 or higher >> > Error: kubeadm init failed! >> > >> > >> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <rohit.ya...@shapeblue.com> a >> > écrit : >> > >> > > Hi Benoit, >> > > >> > > The CKS feature has been improving over versions, I don't know if what >> > > you're trying to achieve is possible with it. Maybe try a different >> > version >> > > of the data iso: >> > > http://download.cloudstack.org/cks/ >> > > >> > > Alternatively, you can also have a look at the CAPC project: >> > > https://cluster-api-cloudstack.sigs.k8s.io >> > > >> > > >> > > Regards. >> > > >> > > >> > > >> > > >> > > ________________________________ >> > > From: benoit lair <kurushi4...@gmail.com> >> > > Sent: Thursday, April 11, 2024 14:16 >> > > To: us...@cloudstack.apache.org <us...@cloudstack.apache.org>; dev < >> > > dev@cloudstack.apache.org> >> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS >> > > >> > > Any advices ? >> > > >> > > Best regards >> > > >> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <kurushi4...@gmail.com> a >> > écrit : >> > > >> > > > I am opened to every alternatives of changing system vm templates >> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :) >> > > > >> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <kurushi4...@gmail.com> a >> > > écrit : >> > > > >> > > >> Hello Folks, >> > > >> >> > > >> I am trying to install K8s cluster with community iso 1.28.4 >> > > >> I am with a ACS 4.16.0 environment >> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to >> > > >> containerd.io version >> > > >> >> > > >> I would like to release a template who will work with K8s > 1.23.3 >> on >> > > acs >> > > >> 4.16 >> > > >> How can i tell CS to take another system vm template, avoiding to >> mess >> > > >> normal features with VR and VPC VR keeping the systemvm template >> > issued >> > > >> with CS 4.16 ? >> > > >> >> > > >> I am with XCP-NG 8.2.1 in production >> > > >> >> > > >> Thanks for your help or advises >> > > >> Best regards, >> > > >> Benoit >> > > >> >> > > > >> > > >> > >> >
