L. C. Hsieh created SPARK-59196:
-----------------------------------
Summary: Run the graceful-drain and multi-replica HA smoke
examples in CI
Key: SPARK-59196
URL: https://issues.apache.org/jira/browse/SPARK-59196
Project: Spark
Issue Type: Sub-task
Components: Connect
Affects Versions: connect-gateway-0.1.0
Reporter: L. C. Hsieh
Assignee: L. C. Hsieh
Two binaries under crates/proxy/examples/ assert end-to-end invariants but were
only ever compiled by `cargo clippy --all-targets`, never executed, so the
behaviour they cover had no CI verification at all:
- drain_smoke — exercises the two-step drain the gateway performs on SIGTERM
(in-process, triggered through the same channel shutdown_signal() uses).
Asserts that /readyz flips to not-ready when the drain starts, that
in-flight
streams keep receiving messages through the drain, and that the gateway
shuts
down cleanly afterwards. No external dependencies.
- ha_smoke — spawns two real SparkConnectProxy gateways sharing one Redis
affinity store and one static pool of two fake backends, then verifies three
failover preserves an existing session binding, and that the op-id reverse
index resolves across replicas. Needs a Redis (REDIS_URL, default
redis://127.0.0.1:6399).
Both signal failure by panicking on an assert, so a plain `cargo run` is a
sufficient CI check. This adds them as two steps in the `integration` job, plus
a
Redis service container on :6399 for ha_smoke. The Redis *tests* in that job are
unaffected — they start their own container through testcontainers and do not
use
the service.
Verified locally with a Redis on :6399, running all four steps of the job in
sequence: the testcontainers tests still pass (4 + 5), tracing passes, and both
smoke examples exit 0 ("all drain invariants passed" / "all HA invariants
passed").
This covers graceful drain and multi-replica HA, both of which are production
behaviours that previously had no automated coverage. The kind/Helm end-to-end
walkthroughs under deploy/examples/ remain out of scope; they need a much
heavier
harness and are better suited to a separate scheduled workflow.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
