damccorm opened a new pull request, #41255: URL: https://github.com/apache/beam/pull/41255
### Summary Increases `--testTimeoutSeconds` from the 900s (15m) default to `1800` (30m) for the Dataflow streaming ValidatesRunner suites (`validatesRunnerStreamingConfig` and `validatesRunnerV2Streaming`). ### Root Cause In consecutive `PostCommit Java ValidatesRunner Dataflow V2 Streaming` runs ([43169620883](https://github.com/apache/beam/actions/runs/43169620883) and [43281087492](https://github.com/apache/beam/actions/runs/43281087492)), `ParDoSchemaTest > testReadAndWriteMultiOutput` failed with: ``` java.lang.RuntimeException: Dataflow job ... terminated in state RUNNING but did not return a failure reason. ``` Unlike batch ValidatesRunner suites (which use `DataflowTestBatchCoordinator` with `DEFAULT_MERGED_JOB_TIMEOUT_SECONDS = 1800` or wait without a 900s bound), streaming ValidatesRunner tests run via `TestDataflowRunner.waitForStreamingJobTermination`, which calls `job.waitUntilFinish(Duration.standardSeconds(options.getTestTimeoutSeconds()), messageHandler)` starting immediately after `submit()`. Because `validatesRunnerV2Streaming` runs with `--max-workers=12` (`12 forks * 2 parallelThreads = 24` concurrent streaming jobs) alongside other Dataflow postcommits in `apache-beam-testing`, individual streaming jobs can spend 10–13 minutes in `JOB_STATE_QUEUED` or worker pool startup/teardown (e.g., job `2026-11-12_11_56_13-3714750566918859942` in run `43281087492` was queued from `19:56:13Z` to `20:08:53Z` and started workers at `20:10:55Z`, leaving only 18s before the 900s timeout cancelled the job at `20:11:13Z`). Passing `--testTimeoutSeconds=1800` aligns streaming ValidatesRunner timeouts with the 30-minute batch timeout without slowing down passing tests (which complete in ~6–7m once workers start) or failing tests (which `CancelOnError` cancels immediately upon seeing an error log). ### Verification - Triggered `PostCommit Java ValidatesRunner Dataflow V2 Streaming` on branch `fix/dataflow-streaming-vr-test-timeout`: https://github.com/apache/beam/actions/runs/43319211140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
