RyujiTamaki opened a new pull request, #77112: URL: https://github.com/apache/airflow/pull/77112
When `CloudBatchSubmitJobOperator` runs with `deferrable=True`, a transient error from the Cloud Batch API while the trigger polls the job status (for example `504 Deadline Exceeded`) fails the task, even though the Batch job itself keeps running. The SDK's default retry for `get_job` only covers `ServiceUnavailable`, so other transient gRPC errors reach `CloudBatchJobFinishedTrigger.run()`. Its broad `except Exception` turns them into an `error` event. When task-level retries are configured, they then submit the Batch job a second time. This PR applies the same approach as the Dataflow (#66293) and Cloud Run (#67219) triggers: - Catch the same set of transient gRPC errors as the Cloud Run trigger (`ServiceUnavailable`, `InternalServerError`, `DeadlineExceeded`, `GatewayTimeout`, `ResourceExhausted`, `Aborted`) around `get_batch_job`, log a warning, wait `polling_period_seconds` and poll again. - Count those retries against `timeout` like regular polls, so a long outage still ends with a `timed out` event. - Any other exception still yields an `error` event, as before. Tests: the Cloud Batch trigger, operator and hook tests plus the Cloud Run trigger tests pass in Breeze (58 passed), and the new tests fail without the change. `mypy-providers` passes. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
