potiuk opened a new pull request, #32669:
URL: https://github.com/apache/airflow/pull/32669
We would like to use the config.yml approach as our main source of truth for
airflow configuration. So far it has been split amongst multiple files:
* confg.yml -> descroption
* default_airflow.cfg -> JINJA template to keep cofiguration, examples and
description (used to generate airflow.cfg when airflow starts)
* default_test.cfg -> storing test-only configuration used in some test
cases - used to generate unittest.cfg
* scripts/in_container/airflow_ci.cfg where dCI-specific configuration
overwrote the unittest.cfg
This change consolidates it all into unified appraoch, where all
configuration information is retrieved from .yml files stored in
config_templates. No more additional template files processed by JINJA, no more
special CI versions of it, no more unittestdb.cfg file where such configuration
would be generated, no more unittestdb to be used separately for tests.
* The default_*.cfg files were not real configuration files, becuase they
were really JINJA templates and it got peoeple confused when copying the files.
This change leaves the file empty with the comment that instructs the user how
they can get the default configuration.
* The default_airflow.cfg is gone and instead, we have a way to show
defaults via `airflow config list --defaults` command
* Unittest config is generated on-the-flight using defaults stored in
confg_templates. constituing a single place where they need to be changed for
the tests to use them
* internally, description of the configuration is stored in
AirflowConfigurationParser and used by the parser to generate the default
configuration when needed.
* we got rid of `{{{{` in templated config defaults by marking the templates
with ``is_template`` and getting rid of processing those entries with regular
formatting when generating the default values. This only concerns defaults from
config.yml. Reading those configuration entries from file is unaffected.
This change aims to be 100% backwards compatible with the previous
implementation when it comes to functionality, even if internals changed. It
also does not add provider-specific changes that are coming separately.
The only changes are that generated airflow.cfg is slightly more readable
and displays names of variables that can be used to override each configuration
(which is very useful for copy&pasting)
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
