ziyou434 commented on issue #2695:
URL: https://github.com/apache/apisix/issues/2695#issuecomment-725829212
> > > @ziyou434 Could you provides the options that used for etcd start.
> >
> >
> > I use bitnami/etcd chart ,and --set auth.rbac.enabled=false.
> > The chart use setup.sh to start etcd
> > setup.sh
> > ```
> > #!/bin/bash
> >
> > set -o errexit
> > set -o pipefail
> > set -o nounset
> >
> > # Debug section
> > exec 3>&1
> > exec 4>&2
> >
> > if [[ "${BITNAMI_DEBUG:-false}" = true ]]; then
> > echo "==> Bash debug is on"
> > else
> > echo "==> Bash debug is off"
> > exec 1>/dev/null
> > exec 2>/dev/null
> > fi
> >
> > # Constants
> > HOSTNAME="$(hostname -s)"
> > AUTH_OPTIONS=""
> > export
ETCDCTL_ENDPOINTS="etcd-0.etcd-headless.api-gateway.svc.cluster.local:2380"
> > export ROOT_PASSWORD="${ETCD_ROOT_PASSWORD:-}"
> > if [[ -n "${ETCD_ROOT_PASSWORD:-}" ]]; then
> > unset ETCD_ROOT_PASSWORD
> > fi
> > # Functions
> > ## Store member id for later member replacement
> > store_member_id() {
> > while ! etcdctl $AUTH_OPTIONS member list; do sleep 1; done
> > etcdctl $AUTH_OPTIONS member list | grep -w "$HOSTNAME" | awk '{
print $1}' | awk -F "," '{ print $1}' > "$ETCD_DATA_DIR/member_id"
> > echo "==> Stored member id: $(cat ${ETCD_DATA_DIR}/member_id)" 1>&3
2>&4
> > exit 0
> > }
> > ## Configure RBAC
> > configure_rbac() {
> > # When there's more than one replica, we can assume the 1st member
> > # to be created is "etcd-0" since a statefulset is used
> > if [[ -n "${ROOT_PASSWORD:-}" ]] && [[ "$HOSTNAME" == "etcd-0" ]];
then
> > echo "==> Configuring RBAC authentication!" 1>&3 2>&4
> > etcd &
> > ETCD_PID=$!
> > while ! etcdctl $AUTH_OPTIONS member list; do sleep 1; done
> > echo "$ROOT_PASSWORD" | etcdctl $AUTH_OPTIONS user add root
--interactive=false
> > etcdctl $AUTH_OPTIONS auth enable
> > kill "$ETCD_PID"
> > sleep 5
> > fi
> > }
> > ## Checks whether there was a disaster or not
> > is_disastrous_failure() {
> > local endpoints_array=(${ETCDCTL_ENDPOINTS//,/ })
> > local active_endpoints=0
> > local -r min_endpoints=$(((1 + 1)/2))
> >
> > for e in "${endpoints_array[@]}"; do
> > if [[ "$e" != "$ETCD_ADVERTISE_CLIENT_URLS" ]] && (unset -v
ETCDCTL_ENDPOINTS; etcdctl $AUTH_OPTIONS endpoint health --endpoints="$e");
then
> > active_endpoints=$((active_endpoints + 1))
> > fi
> > done
> > if [[ $active_endpoints -lt $min_endpoints ]]; then
> > true
> > else
> > false
> > fi
> > }
> >
> > ## Check wether the member was succesfully removed from the cluster
> > should_add_new_member() {
> > return_value=0
> > if (grep -E
"^Member[[:space:]]+[a-z0-9]+\s+removed\s+from\s+cluster\s+[a-z0-9]+$"
"$(dirname "$ETCD_DATA_DIR")/member_removal.log") || \
> > ! ([[ -d "$ETCD_DATA_DIR/member/snap" ]] && [[ -f
"$ETCD_DATA_DIR/member_id" ]]); then
> > rm -rf $ETCD_DATA_DIR/* 1>&3 2>&4
> > else
> > return_value=1
> > fi
> > rm -f "$(dirname "$ETCD_DATA_DIR")/member_removal.log" 1>&3 2>&4
> > return $return_value
> > }
> >
> > if [[ ! -d "$ETCD_DATA_DIR" ]]; then
> > echo "==> Creating data dir..." 1>&3 2>&4
> > echo "==> There is no data at all. Initializing a new member of the
cluster..." 1>&3 2>&4
> > store_member_id & 1>&3 2>&4
> > configure_rbac
> > else
> > echo "==> Detected data from previous deployments..." 1>&3 2>&4
> > if [[ $(stat -c "%a" "$ETCD_DATA_DIR") != *700 ]]; then
> > echo "==> Setting data directory permissions to 700 in a
recursive way (required in etcd >=3.4.10)" 1>&3 2>&4
> > chmod -R 700 $ETCD_DATA_DIR
> > else
> > echo "==> The data directory is already configured with the
proper permissions" 1>&3 2>&4
> > fi
> > if [[ 1 -eq 1 ]]; then
> > echo "==> Single node cluster detected!!" 1>&3 2>&4
> > elif is_disastrous_failure; then
> > echo "==> Cluster not responding!!" 1>&3 2>&4
> > echo "==> Disaster recovery is disabled, the cluster will try to
recover on it's own..." 1>&3 2>&4
> > elif should_add_new_member; then
> > echo "==> Adding new member to existing cluster..." 1>&3 2>&4
> > etcdctl $AUTH_OPTIONS member add "$HOSTNAME"
--peer-urls="http://${HOSTNAME}.etcd-headless.api-gateway.svc.cluster.local:2380";
| grep "^ETCD_" > "$ETCD_DATA_DIR/new_member_envs"
> > sed -ie "s/^/export /" "$ETCD_DATA_DIR/new_member_envs"
> > echo "==> Loading env vars of existing cluster..." 1>&3 2>&4
> > source "$ETCD_DATA_DIR/new_member_envs" 1>&3 2>&4
> > store_member_id & 1>&3 2>&4
> > else
> > echo "==> Updating member in existing cluster..." 1>&3 2>&4
> > etcdctl $AUTH_OPTIONS member update "$(cat
"$ETCD_DATA_DIR/member_id")"
--peer-urls="http://${HOSTNAME}.etcd-headless.api-gateway.svc.cluster.local:2380";
1>&3 2>&4
> > fi
> > fi
> > exec etcd 1>&3 2>&4
> > ```
>
> The start up script seems normal, could you paste some etcd logs?
sure.
2020-11-12 03:02:10.270824 I | pkg/flags: recognized and used environment
variable
ETCD_ADVERTISE_CLIENT_URLS=http://etcd-0.etcd-headless.api-gateway.svc.cluster.local:2379
2020-11-12 03:02:10.270943 I | pkg/flags: recognized and used environment
variable ETCD_DATA_DIR=/bitnami/etcd/data
2020-11-12 03:02:10.271003 I | pkg/flags: recognized and used environment
variable
ETCD_INITIAL_ADVERTISE_PEER_URLS=http://etcd-0.etcd-headless.api-gateway.svc.cluster.local:2380
2020-11-12 03:02:10.271030 I | pkg/flags: recognized and used environment
variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
2020-11-12 03:02:10.271049 I | pkg/flags: recognized and used environment
variable ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
2020-11-12 03:02:10.271073 I | pkg/flags: recognized and used environment
variable ETCD_NAME=etcd-0
2020-11-12 03:02:10.271194 W | pkg/flags: unrecognized environment variable
ETCD_SERVICE_HOST=172.20.199.58
2020-11-12 03:02:10.271210 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2380_TCP_ADDR=172.20.199.58
2020-11-12 03:02:10.271222 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2379_TCP=tcp://172.20.199.58:2379
2020-11-12 03:02:10.271241 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2380_TCP_PROTO=tcp
2020-11-12 03:02:10.271254 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2379_TCP_PORT=2379
2020-11-12 03:02:10.271262 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2379_TCP_ADDR=172.20.199.58
2020-11-12 03:02:10.271273 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2380_TCP_PORT=2380
2020-11-12 03:02:10.271287 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2380_TCP=tcp://172.20.199.58:2380
2020-11-12 03:02:10.271309 W | pkg/flags: unrecognized environment variable
ETCD_SERVICE_PORT_CLIENT=2379
2020-11-12 03:02:10.271323 W | pkg/flags: unrecognized environment variable
ETCD_SERVICE_PORT_PEER=2380
2020-11-12 03:02:10.271335 W | pkg/flags: unrecognized environment variable
ETCD_PORT_2379_TCP_PROTO=tcp
2020-11-12 03:02:10.271350 W | pkg/flags: unrecognized environment variable
ETCD_PORT=tcp://172.20.199.58:2379
2020-11-12 03:02:10.271361 W | pkg/flags: unrecognized environment variable
ETCD_SERVICE_PORT=2379
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap'
flag instead
2020-11-12 03:02:10.271400 I | etcdmain: etcd Version: 3.4.9
2020-11-12 03:02:10.271413 I | etcdmain: Git SHA: 54ba95891
2020-11-12 03:02:10.271423 I | etcdmain: Go Version: go1.12.17
2020-11-12 03:02:10.271429 I | etcdmain: Go OS/Arch: linux/amd64
2020-11-12 03:02:10.271439 I | etcdmain: setting maximum number of CPUs to
2, total number of available CPUs is 2
2020-11-12 03:02:10.271522 W | etcdmain: found invalid file/dir member_id
under data dir /bitnami/etcd/data (Ignore this if you are upgrading etcd)
2020-11-12 03:02:10.271540 N | etcdmain: the server is already initialized
as member before, starting as etcd member...
[WARNING] Deprecated '--logger=capnslog' flag is set; use '--logger=zap'
flag instead
2020-11-12 03:02:10.271813 I | embed: name = etcd-0
2020-11-12 03:02:10.271836 I | embed: data dir = /bitnami/etcd/data
2020-11-12 03:02:10.271847 I | embed: member dir = /bitnami/etcd/data/member
2020-11-12 03:02:10.271859 I | embed: heartbeat = 100ms
2020-11-12 03:02:10.271865 I | embed: election = 1000ms
2020-11-12 03:02:10.271881 I | embed: snapshot count = 100000
2020-11-12 03:02:10.271895 I | embed: advertise client URLs =
http://etcd-0.etcd-headless.api-gateway.svc.cluster.local:2379
2020-11-12 03:02:10.271908 I | embed: initial advertise peer URLs =
http://etcd-0.etcd-headless.api-gateway.svc.cluster.local:2380
2020-11-12 03:02:10.271920 I | embed: initial cluster =
2020-11-12 03:02:10.276156 I | etcdserver: restarting member
8ecb0b7cde5e4235 in cluster 2b0eb2956f410bc1 at commit index 107
raft2020/11/12 03:02:10 INFO: 8ecb0b7cde5e4235 switched to configuration
voters=()
raft2020/11/12 03:02:10 INFO: 8ecb0b7cde5e4235 became follower at term 4
raft2020/11/12 03:02:10 INFO: newRaft 8ecb0b7cde5e4235 [peers: [], term: 4,
commit: 107, applied: 0, lastindex: 107, lastterm: 4]
2020-11-12 03:02:10.279984 W | auth: simple token is not cryptographically
signed
2020-11-12 03:02:10.283770 I | etcdserver: starting server... [version:
3.4.9, cluster version: to_be_decided]
raft2020/11/12 03:02:10 INFO: 8ecb0b7cde5e4235 switched to configuration
voters=(10289330404592599605)
2020-11-12 03:02:10.284749 I | etcdserver/membership: added member
8ecb0b7cde5e4235
[http://etcd-0.etcd-headless.api-gateway.svc.cluster.local:2380] to cluster
2b0eb2956f410bc1
2020-11-12 03:02:10.285151 N | etcdserver/membership: set the initial
cluster version to 3.4
2020-11-12 03:02:10.285318 I | etcdserver/api: enabled capabilities for
version 3.4
2020-11-12 03:02:10.287927 I | embed: listening for peers on [::]:2380
raft2020/11/12 03:02:11 INFO: 8ecb0b7cde5e4235 is starting a new election at
term 4
raft2020/11/12 03:02:11 INFO: 8ecb0b7cde5e4235 became candidate at term 5
raft2020/11/12 03:02:11 INFO: 8ecb0b7cde5e4235 received MsgVoteResp from
8ecb0b7cde5e4235 at term 5
raft2020/11/12 03:02:11 INFO: 8ecb0b7cde5e4235 became leader at term 5
raft2020/11/12 03:02:11 INFO: raft.node: 8ecb0b7cde5e4235 elected leader
8ecb0b7cde5e4235 at term 5
2020-11-12 03:02:11.477491 I | etcdserver: published {Name:etcd-0
ClientURLs:[http://etcd-0.etcd-headless.api-gateway.svc.cluster.local:2379]} to
cluster 2b0eb2956f410bc1
2020-11-12 03:02:11.477626 I | embed: ready to serve client requests
2020-11-12 03:02:11.478714 N | embed: serving insecure client requests on
[::]:2379, this is strongly discouraged!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
