amirmor1 commented on code in PR #69855:
URL: https://github.com/apache/airflow/pull/69855#discussion_r4044444320
##########
providers/google/src/airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -2536,13 +2545,15 @@ def execute(self, context: Context):
region=self.region,
project_id=self.project_id,
batch=self.batch,
- batch_id=self.batch_id,
+ batch_id=requested_batch_id,
request_id=self.request_id,
retry=self.retry,
timeout=self.timeout,
metadata=self.metadata,
)
except AlreadyExists:
+ if not batch_id:
+ raise ValueError("Dataproc reported an existing batch without
a requested batch_id.")
Review Comment:
Good catch, and sorry for the unclear message. That guard only triggered
when neither batch_id nor batch_id_prefix was set — in which case Dataproc
generates the ID server-side and ALREADY_EXISTS shouldn't be reachable. Rather
than document an error for an unreachable state, I've removed the guard here
and the duplicate in retry_batch_creation. This restores the original behaviour
exactly: the except AlreadyExists block in execute() is now unchanged from main.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
