dheerajturaga opened a new pull request, #57459:
URL: https://github.com/apache/airflow/pull/57459
The dag_processor was unable to retrieve connections from the database,
causing GitHook (and other hooks) to fail with:
AirflowNotFoundException: The conn_id `<conn_id>` isn't defined
Root cause: dag_processor was running in FALLBACK context, which only
loads EnvironmentVariablesBackend, not MetastoreBackend. This meant
connections stored in the database were inaccessible.
The dag_processor is a server-side component that needs database access
for connection retrieval, similar to the scheduler and API server.
Fix: Set _AIRFLOW_PROCESS_CONTEXT=server in
DagProcessorJobRunner._execute()
to enable MetastoreBackend, matching the pattern already used in:
- SchedulerJobRunner (scheduler_job_runner.py:1064)
- API FastAPI server (api_fastapi/main.py:24)
This ensures the dag_processor uses the correct secrets backend chain:
EnvironmentVariablesBackend → MetastoreBackend (database access)
Affects: DAG bundle processing with GitHook and any other hooks that
rely on database-stored connections during DAG parsing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
