GitHub user vitstransky created a discussion: DataprocCreateBatchOperator:
Retry on 5xx errors for deferrable
Hello,
we are using Airflow 2.11, apache-airflow-providers-google 15.1.0.
We run Dataproc Serverless batches via DataprocCreateBatchOperator with
defferable = True. We are getting randomly error:
`grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "Getting metadata from plugin failed with error: ("Error code
{'code': 503, 'message': 'The service is currently unavailable.', 'status':
'UNAVAILABLE'}", '{\n "error": {\n "code": 503,\n "message": "The
service is currently unavailable.",\n "status": "UNAVAILABLE"\n }\n}\n')"
debug_error_string = "UNKNOWN:Error received from peer
{grpc_message:"Getting metadata from plugin failed with error: (\"Error code
{\'code\': 503, \'message\': \'The service is currently unavailable.\',
\'status\': \'UNAVAILABLE\'}\", \'{\\n \"error\": {\\n \"code\": 503,\\n
\"message\": \"The service is currently unavailable.\",\\n \"status\":
\"UNAVAILABLE\"\\n }\\n}\\n\')", grpc_status:14,
created_time:"2026-09-24T10:12:11.505328013+02:00"}"`
When I look at params of call DataprocBatchTrigger, there are no retry options
passed there.
According to GCP docs:
`Any client interacting with Google Cloud APIs must implement retry logic with
exponential backoff for 5xx errors (specifically 500, 503 and 409")"`
Could somebody please help us how to solve this? Would update to latest version
of providers help?
Thank you for help.
GitHub link: https://github.com/apache/airflow/discussions/73675
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
