Thanks Dan, I will redeploy dev-mariadb-galera and check the status again.
Gulsum From: TIMONEY, DAN <[email protected]> Sent: Wednesday, June 24, 2020 4:54 PM To: [email protected]; Gülsüm Atıcı <[email protected]>; [email protected] Subject: Re: [onap-discuss] Frankfurt SDNC modul deploy fails That seems to be the issue then – one of the mariadb-galera pods is down, which is preventing the mariadb-galera service from coming online. In my test environment, I can usually clear that by bringing down all the ONAP pods, removing everything under /dockerdata-nfs and restarting. As long as you don’t need to preserve any data, you can try that. Usually when mariadb-galera gets in that state – at least in my experience – it’s because somehow the pods can no longer replicate to each other. In a non-kubernetes environment, I’d normally be able to work around that by bringing down all but the lead node, then bouncing the lead and restarting the others so that I force the ordering. In kubernetes, it’s not quite so simple, since kubernetes will restart pods for you as you kill them – so it’s harder to force the ordering. Or perhaps there is a way to force it, but I’m afraid that’s beyond my knowledge. That’s why I go with the more brute force method of stopping everything and removing the backing store (/dockerdata-nfs) to force mariadb-galera to come up clean. Dan From: onap-discuss <[email protected]<mailto:[email protected]>> on behalf of Gülsüm Atıcı <[email protected]<mailto:[email protected]>> Reply-To: onap-discuss <[email protected]<mailto:[email protected]>>, "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, June 24, 2020 at 9:44 AM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>>, "TIMONEY, DAN" <[email protected]<mailto:[email protected]>>, onap-discuss <[email protected]<mailto:[email protected]>> Subject: Re: [onap-discuss] Frankfurt SDNC modul deploy fails Dear Dan, Here are the MariaDB pods status. ubuntu@onap-nfs:~/scripts$ kubectl -n onap get pods | grep -i maria dev-mariadb-594dbb55b5-wbqs4 1/1 Running 0 163m dev-mariadb-galera-0 1/1 Running 0 129m dev-mariadb-galera-1 0/1 CrashLoopBackOff 24 128m dev-modeling-mariadb-0 1/1 Running 0 129m dev-modeling-mariadb-1 1/1 Running 6 128m dev-modeling-mariadb-2 0/1 Running 30 108m dev-policy-mariadb-0 1/1 Running 0 128m dev-so-mariadb-config-job-42g9r 0/1 Init:Error 0 23m dev-so-mariadb-config-job-cl8tp 0/1 Init:Error 0 63m dev-so-mariadb-config-job-d4rx7 0/1 Init:0/1 0 7m2s dev-so-mariadb-config-job-frvws 0/1 Init:Error 0 73m dev-so-mariadb-config-job-m2mqt 0/1 Init:Error 0 38m dev-so-mariadb-config-job-m5mvf 0/1 Init:Error 0 84m dev-so-mariadb-config-job-ncgbj 0/1 Init:Error 0 51m dev-so-mariadb-config-job-qdw99 0/1 Init:Error 0 94m dev-vfc-mariadb-0 1/1 Running 0 61m dev-vfc-mariadb-1 1/1 Running 4 50m dev-vfc-mariadb-2 1/1 Running 0 42m ubuntu@onap-nfs:~/scripts$ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#21488): https://lists.onap.org/g/onap-discuss/message/21488 Mute This Topic: https://lists.onap.org/mt/75080232/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
