shahar1 commented on code in PR #69279:
URL: https://github.com/apache/airflow/pull/69279#discussion_r3523364876
##########
providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py:
##########
@@ -2437,8 +2438,7 @@ async def cancel_job(self, job_id: str, project_id: str |
None, location: str |
self.log.error("Failed to cancel BigQuery job %s: %s", job_id,
str(e))
raise
- # TODO: Convert get_records into an async method
- def get_records(
+ async def get_records(
Review Comment:
This is a breaking-change - please following the instructions in the
CHANGELOG file to add an appropriate warning.
##########
providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py:
##########
@@ -2460,6 +2460,7 @@ def get_records(
fields_names = [field["name"] for field in fields]
col_types = [field["type"] for field in fields]
for dict_row in rows:
+ await asyncio.sleep(0)
Review Comment:
Yielding on every row may create an overhead, do you find this pattern safe
comparing to doing it for every number (>1) of rows?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
