avantgardnerio opened a new pull request, #2245:
URL: https://github.com/apache/datafusion-ballista/pull/2245
## Summary
The example `cluster.yaml` in
`docs/source/user-guide/deployment/kubernetes.md` pairs a single ClusterIP
Service with the scheduler's default `--min-ready-executors=1` /readyz gate. On
a fresh deploy this is chicken-and-egg:
- ClusterIP only routes to Ready pods.
- The scheduler is not Ready until 1 executor registers.
- Executors resolve the scheduler through the ClusterIP.
- → executors can't reach the scheduler → scheduler never flips green →
cluster never converges.
Discovered bringing the example up verbatim on a real k8s cluster.
## Fix
Add a second Service, `ballista-scheduler-registration`, with
`publishNotReadyAddresses: true`, alongside the existing one:
- `ballista-scheduler` (unchanged) — **clients** dial this. The /readyz gate
still protects clients during rolling scheduler upgrades: a new scheduler with
0 executors won't receive query traffic until it's Ready.
- `ballista-scheduler-registration` — **executors** dial this. Bootstrap
registration works before /readyz flips green.
Both Services target the same pod via `selector: { app: ballista-scheduler
}`; they only differ in the readiness-gating behavior.
Pure yaml/docs change — no code, no proto, no config.
## Test plan
- [x] `cargo doc` unaffected (docs source is under `docs/`, not `///`).
- [x] Verified on a real k8s cluster: with the two-Service pattern, fresh
deploys converge without operator intervention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
