potiuk commented on issue #34015: URL: https://github.com/apache/airflow/issues/34015#issuecomment-1703751426
The `self.configuration_description = retrieve_configuration_description(include_providers=False)` is deliberately set - and mainly because our configuration is needed often when commands are displayed and we cannot load provider configuration then. Loading provider's configuration is somewhat costly, because we need to find all providers installed, retrieve their `get_provider_info` entrypoint and update configuration with them. Generally speaking loading configuration now is done in two stages: 1) config parser is loaded first without providers (as you rightfully noticed 2) config parser is UPDATED with provider configuration information after the configuration is discovered from provoders https://github.com/apache/airflow/blob/3ae6b4e86fe807c00bd736c59df58733df2b9bf9/airflow/providers_manager.py#L548 When `initialize_providers_configuration` is called on ProvidersManager, it retrieves all the provider information. and once this information is retrieved it runs ` conf.load_providers_configuration()` that updates information about provider's configuraiton. I believe all the places that airflow needs, the step of initializing provider's configuration should be done before. But maybe there is a place where it is not - but you need to tell me which place it is. If this is some of your code that you wrote - you need to update your code to also load provider configuration. Converting it into discussion - because we seem to be discussing about some internals of Airflow and you are talking about somethign rather custom. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
