potiuk commented on PR #72206: URL: https://github.com/apache/airflow/pull/72206#issuecomment-6630077907
A second look at this turned up real problems, so the PR changes shape. Please review ea2398b0cd rather than the earlier state. Celery derives its result backend from `sql_alchemy_conn` when none is set, which is the chart default. Dropping that variable from workers made them write results to a pod-local SQLite file while the scheduler reads Postgres, with no error. What changed: - Celery workers keep `AIRFLOW__DATABASE__SQL_ALCHEMY_CONN` unless a result backend is explicitly configured (`data.resultBackendConnection`, `data.resultBackendSecretName` or `config.celery.result_backend`). Default installs therefore still give workers the connection; the production guide explains how to remove it. They no longer get `AIRFLOW_CONN_AIRFLOW_DB`, except with KEDA. - The API server no longer gets `AIRFLOW_CONN_AIRFLOW_DB`. The Execution API answers task connection lookups from its environment, so tasks could fetch it from there. - Kerberos sidecars keep `SQL_ALCHEMY_CONN` below Airflow 3.4.0, because `airflow kerberos` opens the metadata DB at startup there. From 3.4.0 they lose it. They don't run task code. - Containers without the connection fall back to an empty SQLite database, so the chart disables the migration check there and clears `sql_alchemy_engine_args` set through `config.database`. - Added a `significant` newsfragment, since this is a breaking chart change. The PR title now promises more than this delivers for default Celery workers, so it may need adjusting. --- Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
