aoelvp94 opened a new pull request, #76363:
URL: https://github.com/apache/airflow/pull/76363
## Why
`AdbcHook.insert_rows` built an Arrow `RecordBatch` per `commit_every` chunk
and bound it with `cursor.executemany`. Several ADBC drivers run a bound stream
**once per row**, so the previous `fast_executemany` path was a no-op that set
`cursor.fast_executemany` inside `suppress(AttributeError)`.
## What
- Route `fast_executemany` (**now the default**) to `cursor.adbc_ingest` in
append mode
- Keep `executemany` when `fast_executemany=False`
- Always bind when `replace=True` (bulk ingest cannot carry the dialect’s
conflict clause)
- Chunking, commits, `target_fields`, and `schema.table` splitting unchanged
- No `GenericTransfer` / operator changes; `insert_args={"fast_executemany":
False}` keeps the old path
## How to test
- `pytest providers/apache/arrow/tests/unit/apache/arrow/hooks/test_adbc.py`
- Manually vs PostgreSQL 16, 1M rows × 4 cols:
| `commit_every` | bind (`False`) | ingest (`True`) |
|---|---:|---:|
| 1000 | 17.99 s | 2.87 s |
| 10000 | 17.68 s | 1.11 s |
Discussed with [@dabla](https://github.com/dabla) on Slack
([thread](https://apache-airflow.slack.com/archives/C06K9Q5G2UA/p1793676405815659)).
**Follow-up (separate):** Arrow read/write protocols in `common.sql` for
end-to-end `RecordBatch`es in `GenericTransfer`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
