shahar1 commented on code in PR #41153:
URL: https://github.com/apache/airflow/pull/41153#discussion_r1767535861
##########
airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -3021,18 +3021,26 @@ def __init__(
self.polling_interval_seconds = polling_interval_seconds
def execute(self, context: Context):
+ if self.region is None:
+ raise AirflowException("Region should be set here")
hook = DataprocHook(gcp_conn_id=self.gcp_conn_id,
impersonation_chain=self.impersonation_chain)
# batch_id might not be set and will be generated
if self.batch_id:
+ project_id = self.project_id or hook.project_id
link = DATAPROC_BATCH_LINK.format(
- region=self.region, project_id=self.project_id,
batch_id=self.batch_id
+ region=self.region, project_id=project_id,
batch_id=self.batch_id
)
self.log.info("Creating batch %s", self.batch_id)
self.log.info("Once started, the batch job will be available at
%s", link)
Review Comment:
```suggestion
self.log.info("Once created succesfully, the batch job will be
available at %s", link)
```
To somehow mititage the drawback you mentioned in the description
##########
airflow/providers/google/cloud/operators/dataproc.py:
##########
@@ -3021,18 +3021,26 @@ def __init__(
self.polling_interval_seconds = polling_interval_seconds
def execute(self, context: Context):
+ if self.region is None:
+ raise AirflowException("Region should be set here")
Review Comment:
It has become a mandatory attribute in #41527, so there's no need now for
this check (you'll figure it out while resolving the conflicts)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
