AlinsRan commented on issue #847:
URL:
https://github.com/apache/apisix-helm-chart/issues/847#issuecomment-6618219264
Hi @SRELETI,
The controller doesn't send the config through the admin Service's
ClusterIP, so it doesn't matter whether the Service has ready endpoints.
When the provider is `apisix-standalone` and the GatewayProxy uses
`controlPlane.service` (the chart's default GatewayProxy does), the controller
reads the EndpointSlices of that Service. It then pushes the config to every
Pod IP directly. The only endpoints it skips are the ones marked `terminating`:
```go
// internal/adc/translator/gatewayproxy.go
func(endpoint *discoveryv1.Endpoint) bool {
if endpoint.Conditions.Terminating != nil &&
*endpoint.Conditions.Terminating {
return false
}
return true
}
```
EndpointSlices still list Pods that fail the readiness probe, with `ready:
false`. kube-proxy doesn't send traffic to them, but the controller can still
reach them. So the startup goes like this:
1. An APISIX Pod starts, and `/status/ready` returns 503 because it has no
config yet.
2. The Pod shows up in the EndpointSlice with `ready: false`.
3. The controller sends the config straight to `podIP:9180`.
4. The workers get the config, the readiness probe passes, and the Pod
becomes Ready.
On chart 2.11.1, the `adminService` setting had no effect, and
`controlPlane.endpoints` always took priority over `controlPlane.service`
(apache/apisix-ingress-controller#2442). The controller ended up calling the
Service address, which only routes to Ready Pods, and that caused the deadlock.
#2442 fixed both problems.
If you still see it on 2.12.3, please check:
- `kubectl get gatewayproxy -n <ns> -o yaml`: `spec.provider.controlPlane`
should use `service`, not `endpoints`. If you set
`gatewayProxy.provider.controlPlane.endpoints` to the Service DNS name, you get
the old behavior back.
- The provider type is `apisix-standalone`. The per-Pod push only happens in
that mode.
- `kubectl get endpointslices -n <ns> -l
kubernetes.io/service-name=<admin-svc>`: it should list the APISIX Pods.
- The controller logs: look for connection errors to the Pod IPs, such as a
wrong port or a wrong admin key.
If it still happens, please share your values, the GatewayProxy YAML, and
the controller logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
