Here is a workaround that seems to work that allows us to proceed further. It looks like the cluster is fine. So instead of spending more time trying to find out what is going with the rancher server, we decided to directly talk to the kubernetes API server.
During the initial few minutes when rancher server is still available. Get the node VM IP addresses if you don’t know them already. kubectl get nodes -o=wide Find out which VM has api server running. ssh ubuntu@<VM-IP> docker ps | grep kubernetes-kubernetes confirm that it is indeed running apiserver: docker exec <container ID> ps -aux Use that VM for running kubectl commands after copying the .kube/config file and kubectl file to that VM. Modify the .kube/config server line with the container’s IP address. Thanks, -- K i r a n From: Ukponmwan, Itohan Sent: Friday, October 26, 2018 9:06 AM To: [email protected]; Ukponmwan, Itohan <[email protected]>; Kamineni, Kiran K <[email protected]>; [email protected] Subject: RE: [onap-discuss] Issues with Tiller (Cannot Connect to Tiller) Hi All, We haven’t been able to solve this problem. The problem seems to be with the rancher server which crashes and is only fixed by restarting dockerd. The thing is after restarting, it comes back up and then it dies again very shortly after and then I can’t connect to tiller. I have seen this problem on multiple ONAP installs. I am running v1.6.18 of the rancher server, does anyone know how to fix this and is anyone else facing this issue? Thanks, Itohan From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Ukponmwan, Itohan Sent: Thursday, October 25, 2018 2:12 PM To: Kamineni, Kiran K <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]> Subject: Re: [onap-discuss] Issues with Tiller (Cannot Connect to Tiller) Thanks Kiran. -Itohan From: Kamineni, Kiran K Sent: Thursday, October 25, 2018 1:56 PM To: [email protected]<mailto:[email protected]>; Ukponmwan, Itohan <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Subject: RE: [onap-discuss] Issues with Tiller (Cannot Connect to Tiller) It looked like dockerd was messed up. I was not able to restart the rancher server. In fact it said that no such container existed. I had no choice but to restart docker itself. Everything seems to be working now. I tried the following commands: kubectl version helm ls kubectl -n onap get pods and they all are working fine. Thanks, -- K i r a n From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Ukponmwan, Itohan Sent: Thursday, October 25, 2018 10:54 AM To: [email protected]<mailto:[email protected]>; [email protected]<mailto:[email protected]> Subject: Re: [onap-discuss] Issues with Tiller (Cannot Connect to Tiller) Thanks Steve. Is it possible to run these commands without doing a new install or does this have to be integrated into the ONAP install. I ask because when I try to run the first command, I see the same kubectl error (The connection to the server 10.12.5.195:8080 was refused - did you specify the right host or port?) I have seen with other kubectl commands. Thanks, Itohan From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Steve Siani Sent: Thursday, October 25, 2018 10:44 AM To: [email protected]<mailto:[email protected]> Subject: Re: [onap-discuss] Issues with Tiller (Cannot Connect to Tiller) Hi Itohan, I have also faced this issue when installing ONAP recently. The problem comes from the fact that there is no service account for the Tiller service. To solve this issue, you just need to create a Kubernetes service account for Tiller and init helm. See the command bellow $kubectl --namespace kube-system create serviceaccount tiller $kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller $helm init --service-account tiller --upgrade Regards, Steve -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13341): https://lists.onap.org/g/onap-discuss/message/13341 Mute This Topic: https://lists.onap.org/mt/27623611/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
