Hello everyone, I have a proposal (Draft PR here: https://github.com/apache/airflow/pull/28495) how we can change the approach to default values for Airflow configuration - namely to keep "defaults" internally in airflow application rather than with generated "airflow.cfg" file.
This has been popping up in casual conversations we were having at various occasions, and I wanted to implement another fix (with different default on MacOS) and decided to give it a try today - and I believe (with some more testing and likely some tweaks) - I pretty much achieved what I think makes sense (still a few small things around dumping values for diagnostics need to be updated) I believe what I propose is fully backwards compatible, but it has some nice future-looking properties. The gist of the proposal: * defaults for config values are kept internally as a generated Python dictionary rather than a generated .cfg file (generated from the same single source of truth - .yaml config file). * the "airflow.cfg" file is still generated when it is missing - but with commented out values. This way first-time users can still see available sections and options and can easily change them in the config file * if you have a configuration file already generated - nothing changes - the config values will still be used as they used to be and the internal defaults are not used. Doing it this way has some nice properties: * we can easily change defaults in the future and rename properties * we can easily know if the value comes from internal defaults or from a file (even if it has the same value as default) * we can have dynamic defaults depending on different factors (for example disable forking on MacOS dynamically) Let me know what you think. J.
