Lee-W commented on code in PR #39315:
URL: https://github.com/apache/airflow/pull/39315#discussion_r1585980016
##########
airflow/providers/google/cloud/operators/bigquery.py:
##########
@@ -1026,7 +1045,10 @@ def execute(self, context: Context):
if not self.table_project_id:
self.table_project_id = self.project_id
else:
- self.log.info("Ignoring project_id parameter, as
table_project_id is found.")
+ self.log.info("Ignoring 'project_id' parameter, as
'table_project_id' is found.")
+
+ if self.job_id and self.table_id:
Review Comment:
What would happen if both are None 🤔 If exactly one is needed, there's a
handle `exactly_one` in `airflow.utils.helpers`
##########
airflow/providers/google/cloud/triggers/bigquery.py:
##########
@@ -235,7 +237,11 @@ async def run(self) -> AsyncIterator[TriggerEvent]: #
type: ignore[override]
job_status = await hook.get_job_status(job_id=self.job_id,
project_id=self.project_id)
if job_status["status"] == "success":
query_results = await
hook.get_job_output(job_id=self.job_id, project_id=self.project_id)
- records = hook.get_records(query_results=query_results,
as_dict=self.as_dict)
+ records = hook.get_records(
Review Comment:
maybe we should add a TODO here to make it async in the future
##########
airflow/providers/google/cloud/operators/bigquery.py:
##########
@@ -1035,31 +1057,45 @@ def execute(self, context: Context):
)
if not self.deferrable:
- self.log.info(
- "Fetching Data from %s.%s.%s max results: %s",
- self.table_project_id or hook.project_id,
- self.dataset_id,
- self.table_id,
- int(self.max_results),
- )
- if not self.selected_fields:
- schema: dict[str, list] = hook.get_schema(
+ if not self.job_id:
+ self.log.info(
+ "Fetching Data from %s.%s.%s max results: %s",
+ self.table_project_id or hook.project_id,
+ self.dataset_id,
+ self.table_id,
+ int(self.max_results),
Review Comment:
May I know why `int` is required? When will this not be an int
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
