potiuk commented on issue #44793: URL: https://github.com/apache/airflow/issues/44793#issuecomment-2529278535
I seriously doubt this is a problem Airflow uses the same resolution mechanism in consistent order. Most likely what was different is that you had DIFFERENT configuration in two places where things were running. For example in worker you did not have SECRET configured. This is impossible expectation to use a configuration that you have no access to. Generally speaking the expectation is that it's on yoy (deployment manager) to make sure your configuratoin is consistently set on all your nodes and in all processes of Airflow. That's very basic assumption of airflow environment documented in https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#configuration-reference > Use the same configuration across all the Airflow components. While each component does not require all, some configurations need to be same otherwise they would not work as expected. A good example for that is [secret_key](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#config-webserver-secret-key) which should be same on the Webserver and Worker to allow Webserver to fetch logs from Worker. Note that this is changing in Airflow 3, because the "workers" will no longer have database access and they wil not require most of configuration so the configuration for sql_alchemy_conn will only be needed in scheduler, webserver and triggerer (but it will still need to be consistent across all three components) Converting it into discussion in case more concrete evidences of the error and inconsistency is on airflow side (but it's practically impossible). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
