unsw-luopan commented on issue #12580: URL: https://github.com/apache/apisix/issues/12580#issuecomment-3685215542
> Hi [@unsw-luopan](https://github.com/unsw-luopan), could you share the specific steps and configuration for reproducing this? I will try to reproduce it. Sure. Etcd is a 3 nodes cluster and the version is 3.5.4, which is installed from offcial binary file. And the systemd configuration is: [Unit] Description=etcd After=network.target [Service] Type=notify ExecStart=/usr/local/bin/etcd \ --name etcd03 \ --data-dir /var/lib/etcd \ --listen-client-urls https://127.0.0.1:2379,https://10.150.9.179:2379 \ --advertise-client-urls https://10.150.9.179:2379 \ --listen-peer-urls https://10.150.9.179:2380 \ --initial-advertise-peer-urls https://10.150.9.179:2380 \ --cert-file=/etc/etcd/certs2/etcd03.crt \ --key-file=/etc/etcd/certs2/etcd03.key \ --trusted-ca-file=/etc/etcd/certs2/ca.crt \ --client-cert-auth \ --peer-cert-file=/etc/etcd/certs2/etcd03.crt \ --peer-key-file=/etc/etcd/certs2/etcd03.key \ --peer-trusted-ca-file=/etc/etcd/certs2/ca.crt \ --peer-client-cert-auth \ --initial-cluster-token etcd-prod-v1 \ --initial-cluster-state new \ --initial-cluster etcd01=https://10.179.208.136:2380,etcd02=https://10.179.208.144:2380,etcd03=https://10.150.9.179:2380 Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target The remaining two nodes' configuration files just modify IP and name. The cert generation script is: #!/bin/bash # generate-etcd-mtls.sh set -e echo ">>> Generating CA..." # Step 1: Generate CA key and CSR openssl genrsa -out ca.key 2048 openssl req -new -sha256 -key ca.key -out ca.csr -subj "/CN=ROOTCA" # Step 2: Create CA extension config (for self-signed CA cert) cat > v3_ca.ext <<EOF basicConstraints = critical,CA:TRUE keyUsage = critical,keyCertSign,cRLSign subjectKeyIdentifier = hash EOF # Step 3: Generate self-signed CA certificate using extfile openssl x509 -req -days 36500 -sha256 \ -extfile v3_ca.ext \ -signkey ca.key -in ca.csr -out ca.crt # Step 4: Prepare server/client cert extensions (serverAuth + clientAuth) cat > v3_req.ext <<EOF extendedKeyUsage = serverAuth,clientAuth subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer EOF # Step 5: Define etcd nodes declare -A NODES=( [etcd01]="10.179.208.136" [etcd02]="10.179.208.144" [etcd03]="10.150.9.179" ) # Step 6: Generate cert for each node for name in "${!NODES[@]}"; do ip="${NODES[$name]}" echo ">>> Generating cert for $name ($ip)..." # SAN file cat > "${name}.san.ext" <<EOF subjectAltName = IP:$ip,IP:127.0.0.1 EOF # Key openssl genrsa -out "${name}.key" 2048 # CSR with CN=ETCD (as per official guide) openssl req -new -sha256 -key "${name}.key" -out "${name}.csr" -subj "/CN=ETCD" # Combine EKU + SAN into one temp file cat v3_req.ext "${name}.san.ext" > "${name}.ext" # Sign with CA openssl x509 -req -days 36500 -sha256 \ -extfile "${name}.ext" \ -CA ca.crt -CAkey ca.key -CAserial ca.srl -CAcreateserial \ -in "${name}.csr" -out "${name}.crt" done # Step 7: Generate CLIENT cert for APISIX echo ">>> Generating CLIENT cert for APISIX..." openssl genrsa -out client.key 2048 openssl req -new -sha256 -key client.key -out client.csr -subj "/CN=CLIENT" # Optional SAN for client (not required, but harmless) cat > client.san.ext <<EOF subjectAltName = IP:127.0.0.1 EOF cat v3_req.ext client.san.ext > client.ext openssl x509 -req -days 36500 -sha256 \ -extfile client.ext \ -CA ca.crt -CAkey ca.key -CAserial ca.srl \ -in client.csr -out client.crt # Step 8: Cleanup temp files and set permissions rm -f *.csr *.ext v3_ca.ext v3_req.ext ca.srl chmod 600 *.key chmod 644 *.crt ca.crt echo "✅ All certificates generated successfully!" ls -l *.crt *.key After installing etcd and preparing cert file, I start etcd cluster and do cluster health test using: ETCDCTL_API=3 etcdctl \ --endpoints=https://10.179.208.136:2379,https://10.179.208.144:2379,https://10.150.9.179:2379 \ --cacert=/etc/etcd/certs/ca.crt \ --cert=/etc/etcd/certs/etcd01.crt \ --key=/etc/etcd/certs/etcd01.key \ endpoint health The result is health. Then I deploy apisix using apisix-docker repository with tag 3.13. The docker-compose about apisix is: version: "3" services: apisix: image: apache/apisix:${APISIX_IMAGE_TAG:-3.13.0-debian} restart: always privileged: true volumes: - ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro - ./certs2:/etc/etcd/certs:ro - ./logs:/usr/local/apisix/logs # network_mode: host ports: - "9180:9180/tcp" - "9080:9080/tcp" - "9093:9091/tcp" - "9443:9443/tcp" - "9092:9092/tcp" networks: apisix: And the config file of apisix is: apisix: ssl: ssl_trusted_certificate: /etc/etcd/certs/ca.crt deployment: role: traditional role_traditional: config_provider: etcd admin: admin_key_required: true admin_key: - name: admin key: Sup3rs3cretWr1teK3y role: admin etcd: host: - "https://10.179.208.136:2379"; - "https://10.179.208.144:2379"; - "https://10.150.9.179:2379"; prefix: "/apisix" timeout: 30 tls: cert: /etc/etcd/certs/client.crt key: /etc/etcd/certs/client.key The cert file of apisix is generated before, then I run docker-compose and apisix's container hangs on. The log of container is: Attaching to apisix-1 apisix-1 | /usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init apisix-1 | /usr/local/openresty//luajit/bin/luajit ./apisix/cli/apisix.lua init_etcd apisix-1 | trying to initialize the data of etcd I checked network, file permission, docker volumns and other configurations. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
