alberttwong opened a new issue, #76273:
URL: https://github.com/apache/airflow/issues/76273
### Under which category would you file this issue?
Providers
### Apache Airflow version
3.3.2; current main source also inspected
### What happened and how to reproduce it?
The generic Arrow-provider AdbcHook.get_records() fails with the Redis ADBC
driver because it attempts to disable autocommit. Redis ADBC explicitly
supports autocommit only. Direct native ADBC reads and the external
RedisAdbcHook work.
Tested versions: Airflow 3.3.2, apache-airflow-providers-apache-arrow 0.1.1,
common-sql 2.1.1, ADBC manager 1.12.0. Recorded live integration used macOS
arm64 and Redis Software 8.6.2 with Search and active verified 4x QPF.
Configure an adbc connection with driver=adbc_driver_redis and db_kwargs.uri
pointing to a disposable Redis endpoint. Resolve the native library using an
ADBC manifest as shown in the linked regression, to keep platform library
discovery separate from this failure.
Then:
```python
from airflow.providers.apache.arrow.hooks.adbc import AdbcHook,
fetch_all_handler
hook = AdbcHook(adbc_conn_id="redis_adbc")
# Works in the recorded integration; initial connection warns that
# autocommit cannot be disabled.
assert hook.run("SELECT 42 AS answer", autocommit=True,
handler=fetch_all_handler) == [(42,)]
# Fails: NotSupportedError: transactions are not supported (autocommit only)
hook.get_records("SELECT 42 AS answer")
```
The existing integration regression also checks successful CREATE, batched
INSERT and bound reads with autocommit=True, and explicitly asserts the default
get_records failure:
https://github.com/redis/redis-adbc-driver/blob/main/python/tests/test_airflow.py
Live validation record (October 29, 2026):
https://github.com/redis/redis-adbc-driver/blob/main/docs/sdlc/changes/airflow-smoke/validation.md
Source inspection on November 1, 2026 shows the same relevant behavior on
Airflow main:
https://github.com/apache/airflow/blob/main/providers/apache/arrow/src/airflow/providers/apache/arrow/hooks/adbc.py
- get_conn() hardcodes autocommit=False.
- get_records() calls run() without forwarding an autocommit choice, so
run's False default is used.
- set_autocommit() explicitly applies False at the native ADBC layer, where
an autocommit-only driver rejects it.
A fresh mock-based check in the installed versions above confirmed the first
two calls on November 1. No fresh live Redis integration or latest-main runtime
test is claimed by this report.
### What you think should happen instead?
Provide an explicit, supported way to use autocommit-only ADBC drivers
through connection creation and read helpers. Preserve current defaults for
transactional drivers and continue rejecting explicit unsupported transaction
requests. A configurable initial autocommit setting alone is insufficient if
get_records subsequently resets it to False. Regression coverage should
exercise both connection creation and get_records, plus a transactional driver
control.
### Operating System
macOS arm64
### Deployment
None
### Apache Airflow Provider(s)
apache-arrow
### Versions of Apache Airflow Providers
apache-airflow-providers-apache-arrow==0.1.1
apache-airflow-providers-common-sql==2.1.1
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
Local integration tracker:
https://github.com/redis/redis-adbc-driver/issues/272
Existing documentation proposal https://github.com/apache/airflow/pull/75988
explains the external RedisAdbcHook workaround; it does not fix the generic
provider behavior. This report concerns that separate runtime compatibility
gap. SQLExecuteQueryOperator hook selection/provider registration is separate
work.
Prepared with AI assistance (OpenAI Codex).
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
