its-amann commented on code in PR #76883:
URL: https://github.com/apache/airflow/pull/76883#discussion_r4534295339
##########
providers/common/sql/src/airflow/providers/common/sql/hooks/sql.py:
##########
@@ -983,7 +983,7 @@ def insert_rows(
self.get_conn_id(),
)
- for chunked_rows in chunked(rows, commit_every):
+ for chunked_rows in chunked(rows, commit_every or None):
Review Comment:
same zero chunk bug is still in PostgresHook.insert_rows on the psycopg2
fast_executemany path, it calls chunked(rows, commit_every) and
execute_values(page_size=commit_every) itself, so commit_every=0 still loads 0
rows
```diff
--- a/providers/postgres/src/airflow/providers/postgres/hooks/postgres.py
+++ b/providers/postgres/src/airflow/providers/postgres/hooks/postgres.py
- for chunked_rows in chunked(rows, commit_every):
+ for chunked_rows in chunked(rows, commit_every or None):
@@
- execute_values(cur, sql, values,
page_size=commit_every)
+ execute_values(cur, sql, values,
page_size=commit_every or len(values))
```
fixed that plus a test in this commit, you can just cherry pick it instead
of redoing it:
https://github.com/its-amann/airflow/commit/9b6e9cfa455c0f290b77f340b488a4f40f8c6b0f
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
