ump45nose opened a new pull request, #2740: URL: https://github.com/apache/datafusion-ballista/pull/2740
# Which issue does this PR close? Closes #2720. # Rationale for this change `free_ports` only keeps the ports of a single call distinct. It drops its listeners before returning, and the executor child binds them a few milliseconds later, so the next `spawn_executor` call can be handed a port the previous executor has not bound yet. That is what happened in the CI run in the issue: executor-1's flight port was executor-0's gRPC port, it failed with `AddrInUse`, and `await_executors(2)` timed out. I went with the first option from the issue, but made it cluster-wide instead of reserving everything up front. That way a restarted executor (`restart_executor`) is covered too, without an executor-side hook. # What changes are included in this PR? - Replace `free_ports` with `reserve_ports(&mut HashSet<u16>)`. It still holds every listener until all `N` ports are read, but it also skips (and keeps bound) any port already in the set, and adds the ports it returns to the set. - `TestCluster` keeps an `assigned_ports` set. The scheduler, the result service and every executor spawn, including respawns, reserve their ports through it, so no two processes of one cluster share a port. Ports of dead executors stay in the set, so they are never reused. - New unit test `reserved_ports_skip_ports_already_taken`: it reserves 8 ports, lets their listeners go, then reserves 64 more pairs and checks none of the first 8 come back. The existing `ports_reserved_together_are_distinct` now uses `reserve_ports`. Testing: `cargo test -p ballista-chaos --lib` (22 passed, including `cluster_starts_with_the_requested_executors_registered` and `killed_executor_is_reaped_and_can_be_restarted`), `cargo test -p ballista-chaos --test ha`, `cargo clippy -p ballista-chaos --all-targets -- -D warnings`, `cargo fmt --all -- --check`. # Are there any user-facing changes? No, this only changes the chaos test harness. --- This PR was prepared with AI assistance. I reviewed the change and ran the checks above. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
