Hi, Team,
I am trying to bring up ONAP Dublin cluster in Kubernetes. But the common
MariaDB-galera cannot come up and ready with the following error dump from
'logs' and 'describe'. Please help!!! Thanks!!!
(How am I going to resolve this mysql Error 2002 in a container? Is there some
old configuration data need to be cleaned up?)
ubuntu@onap-rancher-vm-1804:~$ kubectl logs -n onap
dev-mariadb-galera-mariadb-galera-0
Galera: Finding peers
+ CONTAINER_SCRIPTS_DIR=/usr/share/container-scripts/mysql
+ EXTRA_DEFAULTS_FILE=/etc/my.cnf.d/galera.cnf
+ '[' -z onap ']'
+ echo 'Galera: Finding peers'
++ hostname -f
++ cut -d. -f2
+ K8S_SVC_NAME=mariadb-galera
+ echo 'Using service name: mariadb-galera'
+ cp /usr/share/container-scripts/mysql/galera.cnf /etc/my.cnf.d/galera.cnf
Using service name: mariadb-galera
+ /usr/bin/peer-finder
-on-start=/usr/share/container-scripts/mysql/configure-galera.sh
-service=mariadb-galera
2019/08/13 23:16:03 lookup mariadb-galera on 10.43.0.10:53: read udp
10.42.2.225:53756->10.43.0.10:53: i/o timeout
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
ubuntu@onap-rancher-vm-1804:~$ kubectl describe pod -n onap
dev-mariadb-galera-mariadb-galera-0
Name: dev-mariadb-galera-mariadb-galera-0
Namespace: onap
Priority: 0
PriorityClassName: <none>
Node: onap-k8s-vm-2/10.3.3.7
Start Time: Tue, 13 Aug 2019 22:28:04 +0000
Labels: app=dev-mariadb-galera-mariadb-galera
chart=mariadb-galera-4.0.0
controller-revision-hash=dev-mariadb-galera-mariadb-galera-7bc879f597
heritage=Tiller
release=dev-mariadb-galera
statefulset.kubernetes.io/pod-name=dev-mariadb-galera-mariadb-galera-0
Annotations: cni.projectcalico.org/podIP: 10.42.2.225/32
pod.alpha.kubernetes.io/initialized: true
Status: Running
IP: 10.42.2.225
Controlled By: StatefulSet/dev-mariadb-galera-mariadb-galera
Init Containers:
mariadb-galera-prepare:
Container ID:
docker://9575dac6d343a9aa284f4e34d6ef1af16adf66f9e0193087445806696cdbf97f
Image: nexus3.onap.org:10001/busybox
Image ID:
docker-pullable://busybox@sha256:4b6ad3a68d34da29bf7c8ccb5d355ba8b4babcad1f99798204e7abb43e54ee3d
Port: <none>
Host Port: <none>
Command:
sh
-c
chown -R 27:27 /var/lib/mysql
State: Terminated
Reason: Completed
Exit Code: 0
Started: Tue, 13 Aug 2019 22:28:07 +0000
Finished: Tue, 13 Aug 2019 22:28:07 +0000
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/lib/mysql from dev-mariadb-galera-mariadb-galera-data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-7q65g
(ro)
Containers:
mariadb-galera:
Container ID:
docker://7e93c364c93d5fa68586ba9cb6d585435b2058c4a0c0e5a5a829a832ba88c5aa
Image:
nexus3.onap.org:10001/adfinissygroup/k8s-mariadb-galera-centos:v002
Image ID:
docker-pullable://nexus3.onap.org:10001/adfinissygroup/k8s-mariadb-galera-centos@sha256:fbcb842f30065ae94532cb1af9bb03cc6e2acaaf896d87d0ec38da7dd09a3dde
Ports: 3306/TCP, 4444/TCP, 4567/TCP, 4568/TCP
Host Ports: 0/TCP, 0/TCP, 0/TCP, 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Tue, 13 Aug 2019 23:15:13 +0000
Finished: Tue, 13 Aug 2019 23:16:42 +0000
Ready: False
Restart Count: 16
Liveness: exec [mysqladmin ping] delay=30s timeout=5s period=10s
#success=1 #failure=3
Readiness: exec
[/usr/share/container-scripts/mysql/readiness-probe.sh] delay=15s timeout=5s
period=10s #success=1 #failure=3
Environment:
POD_NAMESPACE: onap (v1:metadata.namespace)
MYSQL_USER: my-user
MYSQL_PASSWORD: <set to the key 'user-password' in secret
'dev-mariadb-galera-mariadb-galera'> Optional: false
MYSQL_DATABASE: my-database
MYSQL_ROOT_PASSWORD: <set to the key 'db-root-password' in secret
'dev-mariadb-galera-mariadb-galera'> Optional: false
Mounts:
/etc/localtime from localtime (ro)
/var/lib/mysql from dev-mariadb-galera-mariadb-galera-data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from default-token-7q65g
(ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
dev-mariadb-galera-mariadb-galera-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim
in the same namespace)
ClaimName:
dev-mariadb-galera-mariadb-galera-data-dev-mariadb-galera-mariadb-galera-0
ReadOnly: false
localtime:
Type: HostPath (bare host directory volume)
Path: /etc/localtime
HostPathType:
default-token-7q65g:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-7q65g
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 51m (x2 over 51m) default-scheduler pod
has unbound immediate PersistentVolumeClaims (repeated 2 times)
Normal Scheduled 51m default-scheduler
Successfully assigned onap/dev-mariadb-galera-mariadb-galera-0 to onap-k8s-vm-2
Normal Pulled 51m kubelet, onap-k8s-vm-2
Container image "nexus3.onap.org:10001/busybox" already present on machine
Normal Created 51m kubelet, onap-k8s-vm-2
Created container
Normal Started 51m kubelet, onap-k8s-vm-2
Started container
Normal Pulled 49m (x2 over 51m) kubelet, onap-k8s-vm-2
Container image
"nexus3.onap.org:10001/adfinissygroup/k8s-mariadb-galera-centos:v002" already
present on machine
Normal Killing 49m kubelet, onap-k8s-vm-2
Killing container with id docker://mariadb-galera:Container failed liveness
probe.. Container will be killed and recreated.
Normal Created 49m (x2 over 51m) kubelet, onap-k8s-vm-2
Created container
Normal Started 49m (x2 over 51m) kubelet, onap-k8s-vm-2
Started container
Warning Unhealthy 35m (x58 over 50m) kubelet, onap-k8s-vm-2
Readiness probe failed: ERROR 2002 (HY000): Can't connect to local MySQL server
through socket '/var/lib/mysql/mysql.sock' (2 "No such file or directory")
Warning Unhealthy 11m (x45 over 50m) kubelet, onap-k8s-vm-2
Liveness probe failed: mysqladmin: connect to server at 'localhost' failed
error: 'Can't connect to local MySQL server through socket
'/var/lib/mysql/mysql.sock' (2 "No such file or directory")'
Check that mysqld is running and that the socket: '/var/lib/mysql/mysql.sock'
exists!
Warning BackOff 64s (x102 over 39m) kubelet, onap-k8s-vm-2 Back-off
restarting failed container
ubuntu@onap-rancher-vm-1804:~$
Thanks a lot!
Xin Miao
Solution Engineering
Fujitsu Network Communication
(W)972-479-2263 (M)469-268-5226
2811 Telecom Drive
Richardson, TX 75081, USA
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18551): https://lists.onap.org/g/onap-discuss/message/18551
Mute This Topic: https://lists.onap.org/mt/32858722/21656
Group Owner: onap-discuss+ow...@lists.onap.org
Unsubscribe: https://lists.onap.org/g/onap-discuss/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
