The GitHub Actions job "Tests" on airflow.git has failed. Run started by GitHub user o-nikolas (triggered by o-nikolas).
Head commit for run: 27d73a6329319f7289ed4cb6c5ab8f411b5e99a2 / Niko Oliveira <[email protected]> Team Based Configuration (AIP-67) Context: In the context of multi team airflow, teams will have their own configuration files for the components that run on team-only hosts (for workers, dag parsers, etc running on those hosts). However some team components must live along side one another on the scheduler host, namely executors, and some airflow configuration affecting teams must also be accessible to the scheduler. This commit delivers the capability for airflow `conf` to load and allow access to multiple team configurations in addition to the main/global configuration we have today. A new config is added `core.multi_team_configurations` (the name is not set in stone) in which teams and their associated configuration files are specified. E.g.: ``` path/to/team_a/config:team_a,different/path/team_b/configuration:team_b ``` Airflow conf, during initialization, loads each of these configurations and makes them accessible by id, e.g.: `conf.get("core", "executor", team_id="team_a")` Within those team configurations, teams can specify the executors they would like to use and the associated configuration for those executors. Since each team configuration is loaded and stored separately, this allows multiple instances of the same executor to be configured. The Base executor has been updated with a config shim to allow easier access to team based executor configations and the AWS ECS executor has been updated to use it as a proof of concept. Other executors will need to be updated to be "multi team compliant" at a later time to minimize the size and scope of this commit. Report URL: https://github.com/apache/airflow/actions/runs/12383554856 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
