damccorm opened a new pull request, #41119: URL: https://github.com/apache/beam/pull/41119
### Summary In `PerformanceTests ParquetIOIT HDFS` (https://github.com/apache/beam/actions/runs/42422557252, https://github.com/apache/beam/actions/runs/42365700589), `ParquetIOIT` failed with `ParquetCloseResourceException: There are 0 datanode(s) running and 0 node(s) are excluded in this operation` because `Pod/namenode-0` started concurrently with `StatefulSet/datanode` (`replicas: 3`) and ran `/opt/bootstrap.sh` (, `ssh`, `start-dfs.sh`) before `datanode-0..2` had started `sshd`. This change: 1. Sets `podManagementPolicy: Parallel` on `StatefulSet/datanode` so all 3 datanode pods start concurrently. 2. Waits in `namenode-0` for `datanode-0..2` to accept SSH connections before invoking `/opt/bootstrap.sh -d`. 3. Adds a TCP `readinessProbe` on port 9000 for `namenode-0`. ### Verification - Triggered workflow run on branch: https://github.com/apache/beam/actions/runs/42653534228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
