damccorm opened a new pull request, #41034: URL: https://github.com/apache/beam/pull/41034
### Summary Fixes flaky/startup failures in `PostCommit XVR JavaUsingPython Dataflow` (e.g. [run 42212921122](https://github.com/apache/beam/actions/runs/42212921122)) where the first wave of `ValidateRunnerXlangTest` test cases fail with `UNAVAILABLE: io exception / Connection refused: localhost/127.0.0.1:<port>` while the remaining 15 tests succeed once the Python expansion service finishes importing `apache_beam` and binds its port. ### Root Cause 1. `sdks/python/scripts/run_expansion_services.sh` starts the Java and Python expansion services in the background (`&`) and exits immediately without waiting for them to bind to `$JAVA_PORT` and `$PYTHON_PORT` (and also leaks the `flock` file descriptor `200` to the child processes). 2. `BaseExternalTest.waitForReady()` built its `ManagedChannel` without `.usePlaintext()` (unlike `External.java`) and without calling `channel.resetConnectBackoff()` on `TRANSIENT_FAILURE`, and only polled for 15 seconds (`30 * 500ms`). Under CPU contention (8 parallel Gradle test workers starting simultaneously on Python 3.14), the Python expansion service can take ~35-40 seconds to finish importing `apache_beam` and bind the port, causing the first 5 test classes to fail right at the 15s mark. ### Changes - In `sdks/python/scripts/run_expansion_services.sh`, close FD `200` in backgrounded child processes and wait up to 60s for both expansion service ports to accept connections (or dump logs and fail fast if the process exits prematurely). - In `BaseExternalTest.waitForReady()`, use `.usePlaintext()`, call `channel.resetConnectBackoff()` on `TRANSIENT_FAILURE`, and increase the readiness timeout from 15s to 60s. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
