mengw15 opened a new pull request, #6719: URL: https://github.com/apache/texera/pull/6719
### What changes were proposed in this PR? Nothing in CI verifies that an amber worker node can **join a running cluster**. The `computing-unit-master` boot forms only a 1-node self-seeded cluster, `texera-web` doesn't touch clustering, and the packaged `computing-unit-worker` launcher (`bin/computing-unit-worker`) is never started at all. A regression in the worker's join path — its Pekko remoting/serialization, seed-node config, or `startActorWorker` itself — would break the deployed worker while build, unit tests, and the existing boot checks all stay green. The worker can't reuse `smoke-boot.sh`: `ComputingUnitWorker.main` only calls `AmberRuntime.startActorWorker`, which runs no HTTP server and binds `pekko.remote.artery.canonical.port = 0` (a random ephemeral port), so there is no fixed port to probe. Booted standalone it just retries its seed and stays up, so a "did not crash" check proves nothing — the worker's real health signal is **cluster membership**. - **Add `.github/scripts/smoke-cluster.sh`** — a self-contained harness that boots a `computing-unit-master` (non-cluster mode self-seeds `pekko://Amber@localhost:2552` and serves Dropwizard on `:8085`) and a no-arg `computing-unit-worker` (defaults its seed to `localhost:2552` and skips the `getNodeIpAddress` → `checkip.amazonaws.com` lookup that only fires with `--serverAddr`, so **no external network**), then asserts the worker actually **JOINS** — waiting for `ClusterListener`'s `Now we have 2 nodes in the cluster` readiness line on the master (a deliberate signal the app emits, **not** a crash-scan; contrast #6332). It fails fast if either JVM dies and reaps both on exit. - **Wire it into `amber-integration`** right after the `texera-web` boot, reusing the amber dist already built and unzipped there. `ubuntu-latest` only, mirroring the existing boot steps. It boots its own master, so it is independent of #6377. - **Add `.github/scripts/test_smoke_cluster.sh`** — drives the harness end-to-end with fake launchers (6 cases), asserting the **specific** verdict message for each outcome so that deleting a fast-fail branch (and letting a case pass only via the slow timeout) turns the test red. The happy path ties the join signal to the worker actually running, so it can't pass if the worker is never launched. Discovered and run by the `infra` job. ### Any related issues, documentation, discussions? Closes #6523, raised from the #6377 review (suggested by @Yicong-Huang). Complements the `texera-web` (#6319) and `computing-unit-master` (#6377) boot tests — it is the only check that exercises a real cluster join. Smoke rationale from #6220. ### How was this PR tested? - `amber-integration` (ubuntu-latest) runs the new step in this PR: the master and worker boot, the worker joins, and the master logs `2 nodes in the cluster`. - `test_smoke_cluster.sh` passes 6/6 locally on bash 3.2.57 (macOS) and bash 5; **mutation-tested** — deleting the worker-alive fast-fail, the master-alive fast-fail, or the worker launch each turns the test red; `shellcheck` clean; deterministic across repeated runs with no leftover processes. - Verified single-host + no external egress by reading `AmberRuntime.startActorMaster/startActorWorker`: the non-cluster master self-seeds `localhost:2552`, and a no-arg worker joins it while skipping the `checkip.amazonaws.com` lookup. ### Was this PR authored or co-authored using generative AI tooling? Generated-by: Claude Code (claude-opus-4-8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
