meel-ayush opened a new pull request, #76563: URL: https://github.com/apache/airflow/pull/76563
Allow `Connection.get_connection_from_secrets` to accept an optional keyword-only `session: Session | None = None` parameter and forward it to `MetastoreBackend.get_connection` when supported. When internal engine components (such as the scheduler or DAG processor) query connections while holding an open transaction under `with prohibit_commit(session):`, `MetastoreBackend.get_connection` previously had no session passed to it. It invoked `create_session()`, which scoped to the ambient session and committed on exit. This tripped `CommitProhibitorGuard`, rolled back the transaction, and silently discarded uncommitted caller state. This is the sibling fix to PR #71968 (`Variable.get`), restoring consistency across secrets resolution: - Accepts `session` on `Connection.get_connection_from_secrets` and forwards it to `MetastoreBackend.get_connection` when supported. - Guards against session usage in Task SDK execution contexts by raising `ValueError` (guiding callers to `airflow.sdk.Connection.get()`). - Preserves caller transactions, uncommitted rows, and locks without tripping commit guards. closes: #76553 ### Tests - Unit and DB integration tests added in `airflow-core/tests/unit/models/test_connection.py` covering session propagation to `MetastoreBackend`, exclusion for non-metastore backends, execution context rejection, and behavior under `prohibit_commit`. --- ##### Was generative AI tooling used to co-author this PR? - [ ] Yes (please specify the tool below) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
