The GitHub Actions job "Tests" on airflow.git has failed. Run started by GitHub user potiuk (triggered by potiuk).
Head commit for run: e88d618df223a15aa65784be101e177541b39db8 / Jarek Potiuk <[email protected]> Allow configuration to be contributed by providers The changes implemented: * provider.yaml files for providers can optionally contribute extra configuration, the configuration is exposed via "get_provider_info" entrypoint, thus allowing Airflow to discover the configuration from both - sources (in Breeze and local development) and from installed packages * Documentation for configuration contributed by providers is generated as part of Provider documentation. It is also discoverable by having a "core-extension" page displaying all community providers that contribute their own configuration. * Celery configuration (and in the future Kubernetes configuration) is linked directly from the airflow documentation - the providers are preinstalled, which means that celery (and Kubernetes in the future) configuration is considered as important to be directly mentioned and linked from the core. Similarly Celery and Kubernetes executor documentation remains in the core documentation (still configuration options are detailed only in the provider documentation and only linked from the core. * the `airflow config list` command becomes more powerful, with new flags showing optionally decriptions, examples, environment variables corresponding to the commands, showing sources where the configuration value comes from (including provider information and also allowing to show just default values - which nicely replaces the "default_airflow.cfg" In order to that, howver, some crucial changes had to be done to refactor the code for configuration retrieval. This change refactors configuration retrieval quite heavily in order to allow the default configuration to be contributed by providers. It simplifies and removes some of the historical internals of configuration reading and writing, removing unnecessary and unused code and finally replacing the mechanism where we kept configuration description in both yaml files and pre-rendered template files, thus removing the need of keeping the rendered configuration in the airflow package. * config.yml becomes the only source of truth of default configuration values and configuration description (including comments) for Airflow core * default_airflow.cfg and default_test.cfg templates are deleted and default_airflow.cfg content is replaced with comments explaining how to get default config (with airflow config defaults command). Those templates were confusing because they almost looked like airflow configuration but they weren't (they contained JINJA directives) * unittest.cfg is not used any more - test configuration is dynamically generated when tests are starting with pytest fixture. The source of truth for test configuration is config_unit_test.yml * removed airflow_ci.cfg which was yet another configuration file used only in CI - yet another source of confusion. All the test configuration needed is now part of config_unit_tests.yml. Thanks to that, unit tests for configuration work now in CI and local venv without having to set special environment variables or configuration * old methods of processing the default_*.cfg and writing the config have been removed as we do not have the JINJA templates any more for those. * the code that generates documentation from configuration has been refactored to avoid unnecessary repetitions * secret configuration tests using Hashicorp have been moved to Hashicorp provider, thanks to that we can run config tests without installing hvac. * there were test covering type coertion for configuration - they were using "kerberos" configuration but thanks to config_unit_tests.yml approach, converted them to "abstract" tests * moved default_celery.py configuration to provider (this is a remnant of the celery executor moving to provider. There is no need for backwards compatibility and PEP 562, it is only used by Celery* executors. Report URL: https://github.com/apache/airflow/actions/runs/5564354474 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
