damccorm opened a new pull request, #41046: URL: https://github.com/apache/beam/pull/41046
## Summary Fixes `PostCommit PortableJar Flink` (`testUberJarFlinkRunner` and `testUberJarFlinkRunnerSaveMainSession`) failure observed on `master` in [run 42258986361](https://github.com/apache/beam/actions/runs/42258986361). ### Root Cause In `runners/portability/test_flink_uber_jar.sh`, `FlinkMiniClusterEntryPoint` (`java ... -jar "$FLINK_MINI_CLUSTER_JAR" --rest-port "$FLINK_PORT" --rest-bind-address localhost &`) is launched in the background and the script immediately runs `python -c "$PIPELINE_PY" ... --flink_master "localhost:$FLINK_PORT" --flink_submit_uber_jar`. Because `--flink_job_server_jar` is passed locally, `FlinkUberJarJobServer` skips querying `/v1/config`, packages the uber jar in ~5s, and immediately `POST`s to `http://localhost:$FLINK_PORT/v1/jars/upload`. When Flink 2.2's `MiniCluster.start()` takes ~6-8s to initialize Pekko RPC, BlobServer, TaskExecutor, and the Netty REST server, the upload fails with `ConnectionRefusedError: [Errno 111] Connection refused` ~0.8s before the REST server finishes binding. ### Changes - Poll `http://localhost:$FLINK_PORT/v1/config` (up to 60s, checking that `FLINK_PID` is still alive) in `runners/portability/test_flink_uber_jar.sh` before submitting the pipeline. ## Verification - Triggered `PostCommit PortableJar Flink` workflow on branch `fix/flink-uber-jar-minicluster-readiness`: https://github.com/apache/beam/actions/runs/42269072775 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
