zhangyihong opened a new issue, #290:
URL: https://github.com/apache/apisix-helm-chart/issues/290
k8s版本: aws eks
kubectl version --short
Client Version: v1.19.6-eks-49a6c0
Server Version: v1.20.15-eks-0d102a7
apisix 版本:
```
$ helm list -n ingress-apisix
NAME NAMESPACE REVISION UPDATED
STATUS CHART APP VERSION
apisix ingress-apisix 1 2022-05-18 16:17:51.725532
+0800 CST deployed apisix-0.9.3 2.13.1
```
apache/apisix:2.13.1-alpine
apache/apisix-ingress-controller:1.4.1
docker.io/bitnami/etcd:3.4.18-debian-10-r14
pod:
```
$ kubectl get po -o wide -n ingress-apisix
NAME READY STATUS RESTARTS AGE
IP NODE
NOMINATED NODE READINESS GATES
apisix-78f5cb9b84-82czl 1/1 Running 0 16h
172.19.115.235 ip-172-19-113-187.cn-northwest-1.compute.internal <none>
<none>
apisix-78f5cb9b84-qczbv 1/1 Running 0 16h
172.19.117.215 ip-172-19-112-190.cn-northwest-1.compute.internal <none>
<none>
apisix-dashboard-5465fb4df7-9z8xg 1/1 Running 4 16h
172.19.116.190 ip-172-19-124-247.cn-northwest-1.compute.internal <none>
<none>
apisix-etcd-0 1/1 Running 0 16h
172.19.125.197 ip-172-19-119-174.cn-northwest-1.compute.internal <none>
<none>
apisix-etcd-1 1/1 Running 0 16h
172.19.98.160 ip-172-19-124-249.cn-northwest-1.compute.internal <none>
<none>
apisix-etcd-2 1/1 Running 0 16h
172.19.119.222 ip-172-19-117-72.cn-northwest-1.compute.internal <none>
<none>
apisix-ingress-controller-87d78d98d-dftls 1/1 Running 0 16h
172.19.97.227 ip-172-19-124-247.cn-northwest-1.compute.internal <none>
<none>
```
我们把
apisix-etcd-2所在的k8s节点(ip-172-19-117-72.cn-northwest-1.compute.internal)设置为维护模式,并驱逐该节点的apisix-etcd-2
到其他节点上:
```
$ kubectl drain ip-172-19-117-72.cn-northwest-1.compute.internal
--ignore-daemonsets --delete-local-data
node/ip-172-19-117-72.cn-northwest-1.compute.internal cordoned
WARNING: ignoring DaemonSet-managed Pods:
default/prometheus-node-exporter-55rbb, kube-system/aws-node-m5xjb,
kube-system/efs-csi-node-dlln4, kube-system/kube-proxy-xhjz7
evicting pod ingress-apisix/apisix-etcd-2
pod/apisix-etcd-2 evicted
node/ip-172-19-117-72.cn-northwest-1.compute.internal evicted
```
检查apisix-etcd-2迁移到新k8s节点后的运行状况,apisix-etcd-2在新的k8s节点(ip-172-19-97-147.cn-northwest-1.compute.internal)
一直处在CrashLoopBackOff状态:
```
$ kubectl get po -o wide -n ingress-apisix
NAME READY STATUS
RESTARTS AGE IP NODE
NOMINATED NODE READINESS GATES
apisix-78f5cb9b84-82czl 1/1 Running 0
17h 172.19.115.235 ip-172-19-113-187.cn-northwest-1.compute.internal
<none> <none>
apisix-78f5cb9b84-qczbv 1/1 Running 0
17h 172.19.117.215 ip-172-19-112-190.cn-northwest-1.compute.internal
<none> <none>
apisix-dashboard-5465fb4df7-9z8xg 1/1 Running 4
17h 172.19.116.190 ip-172-19-124-247.cn-northwest-1.compute.internal
<none> <none>
apisix-etcd-0 1/1 Running 0
17h 172.19.125.197 ip-172-19-119-174.cn-northwest-1.compute.internal
<none> <none>
apisix-etcd-1 1/1 Running 0
17h 172.19.98.160 ip-172-19-124-249.cn-northwest-1.compute.internal
<none> <none>
apisix-etcd-2 0/1 CrashLoopBackOff 8
16m 172.19.100.35 ip-172-19-97-147.cn-northwest-1.compute.internal
<none> <none>
apisix-ingress-controller-87d78d98d-dftls 1/1 Running 0
17h 172.19.97.227 ip-172-19-124-247.cn-northwest-1.compute.internal
<none> <none>
```
apisix-etcd-2 的日志:
```
$ kubectl logs apisix-etcd-2 -n ingress-apisix -f
etcd 01:28:08.43
etcd 01:28:08.43 Welcome to the Bitnami etcd container
etcd 01:28:08.43 Subscribe to project updates by watching
https://github.com/bitnami/bitnami-docker-etcd
etcd 01:28:08.43 Submit issues and feature requests at
https://github.com/bitnami/bitnami-docker-etcd/issues
etcd 01:28:08.44
etcd 01:28:08.44 INFO ==> ** Starting etcd setup **
etcd 01:28:08.46 INFO ==> Validating settings in ETCD_* env vars..
etcd 01:28:08.46 WARN ==> You set the environment variable
ALLOW_NONE_AUTHENTICATION=yes. For safety reasons, do not use this flag in a
production environment.
etcd 01:28:08.47 INFO ==> Initializing etcd
etcd 01:28:08.47 INFO ==> Generating etcd config file using env variables
etcd 01:28:08.48 INFO ==> Detected data from previous deployments
etcd 01:28:08.61 INFO ==> Updating member in existing cluster
Error: bad member ID arg (strconv.ParseUint: parsing "": invalid syntax),
expecting ID in Hex
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]