Hi, I would suggest you not to spend time on this... 4.16 is EOL
Please upgrade to cloudstack 4.18/4.19, or use old kubernetes versions .. -Wei On Fri, Apr 12, 2024 at 2:25 PM benoit lair <kurushi4...@gmail.com> wrote: > > i succeded to install systemvm template 4.19 as serving template for > control and worker nodes with cks community iso 1.28.4 > when cluster saying "starting", i connect to node controller via p2222, saw > that docker was not installed, but containerd.io yes > > I've done the following : > > apt install docker-ce > cp /etc/containerd/config.toml /etc/containerd/config.toml.bck > containerd config default | tee /etc/containerd/config.toml > /opt/bin/setup-kube-system > /opt/bin/deploy-kube-system > > on to control node and same after on worker node > CS UI show kubernetes yaml > a "kubectl.exe --kubeconfig kube.conf.conf get nodes" > > NAME STATUS ROLES AGE VERSION > k8s-cks-cl19-control-18ed18311c9 Ready control-plane 3h1m v1.28.4 > k8s-cks-cl19-node-18ed1850433 Ready <none> 170m v1.28.4 > > However CS says the cluster is in alert state > And dashboad is not working > > Any advice ? > > when executing this on my laptop : > kubectl.exe --kubeconfig cl19_k8s_1.28.4.conf proxy > > and opening this : > http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/ > > I have this result : > { > "kind": "Status", > "apiVersion": "v1", > "metadata": {}, > "status": "Failure", > "message": "no endpoints available for service \"kubernetes-dashboard\"", > "reason": "ServiceUnavailable", > "code": 503 > } > > Somebody got this problem with dashboard ? > > Did i miss something installing the cluster manually with acs stuff ? > > Le jeu. 11 avr. 2024 à 17:40, benoit lair <kurushi4...@gmail.com> a écrit : > > > Hi Wei, > > > > Thanks for the sharing, i also tried to install the systemvm 4.19 > > I have control node and worker node under template systemvm 4.19 ( > > http://download.cloudstack.org/systemvm/4.19/systemvmtemplate-4.19.0-xen.vhd.bz2 > > ) > > I tried with community cks iso 1.25.0 and 1.28.9 > > On systemvm 4.19 docker was not present by default, just containerd in > > 1.6.x version > > I installed docker-ce and docker-ce-cli : > > > > apt install -y apt-transport-https ca-certificates curl > > software-properties-common > > curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg > > --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg > > echo "deb [arch=$(dpkg --print-architecture) > > signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] > > https://download.docker.com/linux/debian $(lsb_release -cs) stable" | > > sudo tee /etc/apt/sources.list.d/docker.list > /dev/null > > remove line about docker in double in /etc/apt/sources.list > > apt update > > apt install -y docker-ce > > apt install -y docker-cli > > i mounted the 1.28.4 cks community iso in control vm > > > > and /opt/bin/setup-kube-system > > i have these errors : > > > > Is it the good way to manually install kube system > > > > root@k8s-cks-cl17-control-18ecdc1e5bd:/home/core# > > /opt/bin/setup-kube-system > > Installing binaries from /mnt/k8sdisk/ > > 5b1fa8e3e100: Loading layer > > [==================================================>] 803.8kB/803.8kB > > 39c831b1aa26: Loading layer > > [==================================================>] 26.25MB/26.25MB > > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest > > 417cb9b79ade: Loading layer > > [==================================================>] 657.7kB/657.7kB > > 8d323b160d65: Loading layer > > [==================================================>] 24.95MB/24.95MB > > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0 > > 6a4a177e62f3: Loading layer > > [==================================================>] 115.2kB/115.2kB > > 398c9baff0ce: Loading layer > > [==================================================>] 16.07MB/16.07MB > > Loaded image: registry.k8s.io/coredns/coredns:v1.10.1 > > bd8a70623766: Loading layer > > [==================================================>] 75.78MB/75.78MB > > c88361932af5: Loading layer > > [==================================================>] 508B/508B > > Loaded image: kubernetesui/dashboard:v2.7.0 > > e023e0e48e6e: Loading layer > > [==================================================>] 103.7kB/103.7kB > > 6fbdf253bbc2: Loading layer > > [==================================================>] 21.2kB/21.2kB > > 7bea6b893187: Loading layer > > [==================================================>] 716.5kB/716.5kB > > ff5700ec5418: Loading layer > > [==================================================>] 317B/317B > > d52f02c6501c: Loading layer > > [==================================================>] 198B/198B > > e624a5370eca: Loading layer > > [==================================================>] 113B/113B > > 1a73b54f556b: Loading layer > > [==================================================>] 385B/385B > > d2d7ec0f6756: Loading layer > > [==================================================>] 355B/355B > > 4cb10dd2545b: Loading layer > > [==================================================>] 130.6kB/130.6kB > > ba9afb2b3e0c: Loading layer > > [==================================================>] 1.076MB/1.076MB > > 22bba3da6b0d: Loading layer > > [==================================================>] 91.29MB/91.29MB > > a4563151d59b: Loading layer > > [==================================================>] 9.544MB/9.544MB > > Loaded image: registry.k8s.io/etcd:3.5.9-0 > > 54ad2ec71039: Loading layer > > [==================================================>] 103.7kB/103.7kB > > 6fbdf253bbc2: Loading layer > > [==================================================>] 21.2kB/21.2kB > > 7bea6b893187: Loading layer > > [==================================================>] 716.5kB/716.5kB > > ff5700ec5418: Loading layer > > [==================================================>] 317B/317B > > d52f02c6501c: Loading layer > > [==================================================>] 198B/198B > > e624a5370eca: Loading layer > > [==================================================>] 113B/113B > > 1a73b54f556b: Loading layer > > [==================================================>] 385B/385B > > d2d7ec0f6756: Loading layer > > [==================================================>] 355B/355B > > 4cb10dd2545b: Loading layer > > [==================================================>] 130.6kB/130.6kB > > 3f3c83f9a2fa: Loading layer > > [==================================================>] 724.8kB/724.8kB > > 6a2c4bedb8de: Loading layer > > [==================================================>] 32.98MB/32.98MB > > Loaded image: registry.k8s.io/kube-apiserver:v1.28.4 > > e0b89915180d: Loading layer > > [==================================================>] 31.72MB/31.72MB > > Loaded image: registry.k8s.io/kube-controller-manager:v1.28.4 > > d2a4b045f3d0: Loading layer > > [==================================================>] 7.73MB/7.73MB > > 3905b20a2df2: Loading layer > > [==================================================>] 16.85MB/16.85MB > > Loaded image: registry.k8s.io/kube-proxy:v1.28.4 > > f35f4c1ae44f: Loading layer > > [==================================================>] 17.13MB/17.13MB > > Loaded image: registry.k8s.io/kube-scheduler:v1.28.4 > > d01384fea991: Loading layer > > [==================================================>] 19.74MB/19.74MB > > bcec7eb9e567: Loading layer > > [==================================================>] 530B/530B > > Loaded image: kubernetesui/metrics-scraper:v1.0.8 > > e3e5579ddd43: Loading layer > > [==================================================>] 317.6kB/317.6kB > > Loaded image: registry.k8s.io/pause:3.9 > > 1b3ee35aacca: Loading layer > > [==================================================>] 2.796MB/2.796MB > > 910ce076f504: Loading layer > > [==================================================>] 7.425MB/7.425MB > > a8e8b7b8e08a: Loading layer > > [==================================================>] 8.101MB/8.101MB > > 084e56a9c24b: Loading layer > > [==================================================>] 4.376MB/4.376MB > > 334d70dc85ec: Loading layer > > [==================================================>] 159B/159B > > a13197d8bda5: Loading layer > > [==================================================>] 8.216MB/8.216MB > > Loaded image: weaveworks/weave-kube:2.8.1 > > 998efb010df6: Loading layer > > [==================================================>] 728.2kB/728.2kB > > ed37391def99: Loading layer > > [==================================================>] 9.28MB/9.28MB > > 175a472c5f77: Loading layer > > [==================================================>] 299B/299B > > a8764e32e9fe: Loading layer > > [==================================================>] 613B/613B > > Loaded image: weaveworks/weave-npc:2.8.1 > > net.bridge.bridge-nf-call-iptables = 1 > > [init] Using Kubernetes version: v1.28.4 > > [preflight] Running pre-flight checks > > error execution phase preflight: [preflight] Some fatal errors occurred: > > [ERROR CRI]: container runtime is not running: output: > > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection: > > validate CRI v1 runtime API for endpoint > > \"unix:///var/run/containerd/containerd.sock\": rpc error: code = > > Unimplemented desc = unknown service runtime.v1.RuntimeService" > > , error: exit status 1 > > [preflight] If you know what you are doing, you can make a check non-fatal > > with `--ignore-preflight-errors=...` > > To see the stack trace of this error execute with --v=5 or higher > > [init] Using Kubernetes version: v1.28.4 > > [preflight] Running pre-flight checks > > error execution phase preflight: [preflight] Some fatal errors occurred: > > [ERROR CRI]: container runtime is not running: output: > > time="2024-04-11T15:33:26Z" level=fatal msg="validate service connection: > > validate CRI v1 runtime API for endpoint > > \"unix:///var/run/containerd/containerd.sock\": rpc error: code = > > Unimplemented desc = unknown service runtime.v1.RuntimeService" > > , error: exit status 1 > > [preflight] If you know what you are doing, you can make a check non-fatal > > with `--ignore-preflight-errors=...` > > To see the stack trace of this error execute with --v=5 or higher > > [init] Using Kubernetes version: v1.28.4 > > [preflight] Running pre-flight checks > > error execution phase preflight: [preflight] Some fatal errors occurred: > > [ERROR CRI]: container runtime is not running: output: > > time="2024-04-11T15:33:27Z" level=fatal msg="validate service connection: > > validate CRI v1 runtime API for endpoint > > \"unix:///var/run/containerd/containerd.sock\": rpc error: code = > > Unimplemented desc = unknown service runtime.v1.RuntimeService" > > , error: exit status 1 > > [preflight] If you know what you are doing, you can make a check non-fatal > > with `--ignore-preflight-errors=...` > > To see the stack trace of this error execute with --v=5 or higher > > Error: kubeadm init failed! > > > > > > > > > > > > Le jeu. 11 avr. 2024 à 17:19, Wei ZHOU <ustcweiz...@gmail.com> a écrit : > > > >> Hi, > >> > >> Please refer to > >> https://github.com/apache/cloudstack/issues/8681#issuecomment-1999083241 > >> > >> The containerd in 4.16/4.17 sysyemvm template is too old. > >> > >> -Wei > >> > >> > >> On Thursday, April 11, 2024, benoit lair <kurushi4...@gmail.com> wrote: > >> > >> > Hi Rohit, > >> > > >> > I already tested with different iso from cloudstack from CKS repos, > >> > It is ok with ACS 4.16 with community iso 1.23.3 but it fails from > >> 1.24.0 > >> > and following others versions > >> > > >> > I tried to do an upgrade from 1.23.3 with 1.24.0, it fails, > >> > I tried to bootstrap a k8s cluster with 1.24.0, also with 1.25 or 1.27.3 > >> > and 1.28.4 > >> > > >> > Last try, i tested k8s community iso 1.25.0 > >> > in CS UI it saying "Create Kubernetes Cluster k8s-cks-cl16 in > >> progress" > >> > and it fails > >> > If i try to ssh on control node with VR_Pub_IP port 2222 > >> > i have nothing in execution > >> > > >> > If i try manual install : > >> > core@k8s-cks-cl16-control-18eccd0c12c:~$ sudo > >> /opt/bin/setup-kube-system > >> > mount: /mnt/k8sdisk: /dev/sr0 already mounted on /mnt/k8sdisk. > >> > Installing binaries from /mnt/k8sdisk/ > >> > 5b1fa8e3e100: Loading layer > >> > [==================================================>] 803.8kB/803.8kB > >> > 39c831b1aa26: Loading layer > >> > [==================================================>] 26.25MB/26.25MB > >> > Loaded image: apache/cloudstack-kubernetes-autoscaler:latest > >> > 417cb9b79ade: Loading layer > >> > [==================================================>] 657.7kB/657.7kB > >> > 8d323b160d65: Loading layer > >> > [==================================================>] 24.95MB/24.95MB > >> > Loaded image: apache/cloudstack-kubernetes-provider:v1.0.0 > >> > 256bc5c338a6: Loading layer > >> > [==================================================>] 122.2kB/122.2kB > >> > df1818f16337: Loading layer > >> > [==================================================>] 14.71MB/14.71MB > >> > Loaded image: registry.k8s.io/coredns/coredns:v1.9.3 > >> > bd8a70623766: Loading layer > >> > [==================================================>] 75.78MB/75.78MB > >> > c88361932af5: Loading layer > >> > [==================================================>] 508B/508B > >> > Loaded image: kubernetesui/dashboard:v2.7.0 > >> > 0b031aac6569: Loading layer > >> > [==================================================>] 804kB/804kB > >> > cc0597072b83: Loading layer > >> > [==================================================>] 1.076MB/1.076MB > >> > 5a8045f731fc: Loading layer > >> > [==================================================>] 90.58MB/90.58MB > >> > f5a5ef1a005f: Loading layer > >> > [==================================================>] 1.188MB/1.188MB > >> > 5cf3bfe5417d: Loading layer > >> > [==================================================>] 8.506MB/8.506MB > >> > Loaded image: registry.k8s.io/etcd:3.5.4-0 > >> > 8d7366c22fd8: Loading layer > >> > [==================================================>] 804.1kB/804.1kB > >> > 7d5f5bba7c45: Loading layer > >> > [==================================================>] 694.9kB/694.9kB > >> > 8154c9483a6c: Loading layer > >> > [==================================================>] 32.72MB/32.72MB > >> > Loaded image: registry.k8s.io/kube-apiserver:v1.25.0 > >> > c5f1a47b47bc: Loading layer > >> > [==================================================>] 29.76MB/29.76MB > >> > Loaded image: registry.k8s.io/kube-controller-manager:v1.25.0 > >> > e4d19dd12df2: Loading layer > >> > [==================================================>] 7.631MB/7.631MB > >> > ba3637ccd78a: Loading layer > >> > [==================================================>] 12.63MB/12.63MB > >> > Loaded image: registry.k8s.io/kube-proxy:v1.25.0 > >> > 026969d859f7: Loading layer > >> > [==================================================>] 14.29MB/14.29MB > >> > Loaded image: registry.k8s.io/kube-scheduler:v1.25.0 > >> > d01384fea991: Loading layer > >> > [==================================================>] 19.74MB/19.74MB > >> > bcec7eb9e567: Loading layer > >> > [==================================================>] 530B/530B > >> > Loaded image: kubernetesui/metrics-scraper:v1.0.8 > >> > 961e93cda9dd: Loading layer > >> > [==================================================>] 307kB/307kB > >> > Loaded image: registry.k8s.io/pause:3.8 > >> > 1b3ee35aacca: Loading layer > >> > [==================================================>] 2.796MB/2.796MB > >> > 17d89e2a5199: Loading layer > >> > [==================================================>] 7.425MB/7.425MB > >> > 148b94859892: Loading layer > >> > [==================================================>] 8.1MB/8.1MB > >> > dbea472c1ced: Loading layer > >> > [==================================================>] 4.376MB/4.376MB > >> > e81f1a367c17: Loading layer > >> > [==================================================>] 131B/131B > >> > 351848f6b295: Loading layer > >> > [==================================================>] 8.216MB/8.216MB > >> > Loaded image: weaveworks/weave-kube:latest > >> > b20f7ff12b24: Loading layer > >> > [==================================================>] 727.9kB/727.9kB > >> > 1d49e386162e: Loading layer > >> > [==================================================>] 9.281MB/9.281MB > >> > 20145eeac146: Loading layer > >> > [==================================================>] 297B/297B > >> > d4cc72748587: Loading layer > >> > [==================================================>] 600B/600B > >> > Loaded image: weaveworks/weave-npc:latest > >> > net.bridge.bridge-nf-call-iptables = 1 > >> > [init] Using Kubernetes version: v1.25.0 > >> > [preflight] Running pre-flight checks > >> > [WARNING SystemVerification]: missing optional cgroups: blkio > >> > error execution phase preflight: [preflight] Some fatal errors occurred: > >> > [ERROR CRI]: container runtime is not running: output: E0411 > >> > 11:38:11.935458 12988 remote_runtime.go:948] "Status from runtime > >> service > >> > failed" err="rpc error: code = Unimplemented desc = unknown service > >> > runtime.v1alpha2.RuntimeService" > >> > time="2024-04-11T11:38:11Z" level=fatal msg="getting status of runtime: > >> rpc > >> > error: code = Unimplemented desc = unknown service > >> > runtime.v1alpha2.RuntimeService" > >> > , error: exit status 1 > >> > [preflight] If you know what you are doing, you can make a check > >> non-fatal > >> > with `--ignore-preflight-errors=...` > >> > To see the stack trace of this error execute with --v=5 or higher > >> > [init] Using Kubernetes version: v1.25.0 > >> > [preflight] Running pre-flight checks > >> > [WARNING SystemVerification]: missing optional cgroups: blkio > >> > error execution phase preflight: [preflight] Some fatal errors occurred: > >> > [ERROR CRI]: container runtime is not running: output: E0411 > >> > 11:38:12.079422 13015 remote_runtime.go:948] "Status from runtime > >> service > >> > failed" err="rpc error: code = Unimplemented desc = unknown service > >> > runtime.v1alpha2.RuntimeService" > >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: > >> rpc > >> > error: code = Unimplemented desc = unknown service > >> > runtime.v1alpha2.RuntimeService" > >> > , error: exit status 1 > >> > [preflight] If you know what you are doing, you can make a check > >> non-fatal > >> > with `--ignore-preflight-errors=...` > >> > To see the stack trace of this error execute with --v=5 or higher > >> > [init] Using Kubernetes version: v1.25.0 > >> > [preflight] Running pre-flight checks > >> > [WARNING SystemVerification]: missing optional cgroups: blkio > >> > error execution phase preflight: [preflight] Some fatal errors occurred: > >> > [ERROR CRI]: container runtime is not running: output: E0411 > >> > 11:38:12.228669 13043 remote_runtime.go:948] "Status from runtime > >> service > >> > failed" err="rpc error: code = Unimplemented desc = unknown service > >> > runtime.v1alpha2.RuntimeService" > >> > time="2024-04-11T11:38:12Z" level=fatal msg="getting status of runtime: > >> rpc > >> > error: code = Unimplemented desc = unknown service > >> > runtime.v1alpha2.RuntimeService" > >> > , error: exit status 1 > >> > [preflight] If you know what you are doing, you can make a check > >> non-fatal > >> > with `--ignore-preflight-errors=...` > >> > To see the stack trace of this error execute with --v=5 or higher > >> > Error: kubeadm init failed! > >> > > >> > > >> > Le jeu. 11 avr. 2024 à 12:03, Rohit Yadav <rohit.ya...@shapeblue.com> a > >> > écrit : > >> > > >> > > Hi Benoit, > >> > > > >> > > The CKS feature has been improving over versions, I don't know if what > >> > > you're trying to achieve is possible with it. Maybe try a different > >> > version > >> > > of the data iso: > >> > > http://download.cloudstack.org/cks/ > >> > > > >> > > Alternatively, you can also have a look at the CAPC project: > >> > > https://cluster-api-cloudstack.sigs.k8s.io > >> > > > >> > > > >> > > Regards. > >> > > > >> > > > >> > > > >> > > > >> > > ________________________________ > >> > > From: benoit lair <kurushi4...@gmail.com> > >> > > Sent: Thursday, April 11, 2024 14:16 > >> > > To: us...@cloudstack.apache.org <us...@cloudstack.apache.org>; dev < > >> > > dev@cloudstack.apache.org> > >> > > Subject: Re: ACS 4.16 - Change SystemVM template for CKS > >> > > > >> > > Any advices ? > >> > > > >> > > Best regards > >> > > > >> > > Le lun. 8 avr. 2024 à 16:53, benoit lair <kurushi4...@gmail.com> a > >> > écrit : > >> > > > >> > > > I am opened to every alternatives of changing system vm templates > >> > > > I just need to run K8s clyusters 1.28 with my CS 4.16 :) > >> > > > > >> > > > Le lun. 8 avr. 2024 à 16:52, benoit lair <kurushi4...@gmail.com> a > >> > > écrit : > >> > > > > >> > > >> Hello Folks, > >> > > >> > >> > > >> I am trying to install K8s cluster with community iso 1.28.4 > >> > > >> I am with a ACS 4.16.0 environment > >> > > >> It seems K8S is not working out of the box with v > 1.23.3 due to > >> > > >> containerd.io version > >> > > >> > >> > > >> I would like to release a template who will work with K8s > 1.23.3 > >> on > >> > > acs > >> > > >> 4.16 > >> > > >> How can i tell CS to take another system vm template, avoiding to > >> mess > >> > > >> normal features with VR and VPC VR keeping the systemvm template > >> > issued > >> > > >> with CS 4.16 ? > >> > > >> > >> > > >> I am with XCP-NG 8.2.1 in production > >> > > >> > >> > > >> Thanks for your help or advises > >> > > >> Best regards, > >> > > >> Benoit > >> > > >> > >> > > > > >> > > > >> > > >> > >
