jason810496 opened a new pull request, #64469:
URL: https://github.com/apache/airflow/pull/64469
* Fix provider config loading and fallback logic
Simplifies and centralizes provider configuration loading by removing
redundant methods and properties, ensuring consistent state management. Updates
fallback logic to use a sentinel value instead of None, improving accuracy in
config lookups. Improves test coverage for provider metadata fallbacks and
clarifies lazy initialization behavior. Streamlines provider manager interfaces
to reduce duplication and potential import cycles.
* Cleanup load_providers_configuration and respect with new provider conf
mechanism
* Fix configuration_description cached_property error to restore .write
compatibility
* Try fix unit tests error
* Fix provider config merging and section separation
Ensures provider configuration sections do not overlap with core sections
and gives priority to provider metadata when merging configuration options.
Refactors tests to verify correct toggling of provider configuration and proper
separation of provider and core config sections. Updates fallback lookup order
and cleans up unused properties for clarity and maintainability.
* Fix check-default-configuration check and harden the error case
Root cause: providers_discovery.py used structlog.getLogger() directly.
Before structlog.configure() is called (which happens later in
settings.py:726), structlog's default PrintLogger writes to stdout with no
level filtering. So debug logs during early provider
discovery pollute the stdout of airflow config list --default, corrupting
the generated config file.
Fix: Switched to logging.getLogger() (stdlib). stdlib logging defaults to
WARNING level and writes to stderr, so debug logs are suppressed and stdout
stays clean. This is also the correct pattern for shared library code —
structlog configuration is the application's
responsibility.
* Use conf.invalidate_cache() instead of __dict__.pop
* Harden the provider conf tests
* Address Copilot's review comments
* Use invalidate_cache instead of __dict__.pop
* Restore __dict__.pop back instead of invalidate_cache
* Fix lookup_sequence not respect _use_providers_configuration
* Address Wei's review comment for run_check_default_configuration
* Add dedicated _invalidate_provider_flag_caches method
* Create fresh conf instance for tests case that might mutate its internal
state for thread-safe concern
* Deprecate load_providers_configuration, restore_core_default_configuration
instead of directly remove
* Address Amogh's comment: Deprecate already_initialized_provider_configs
instead of directly removing it
Add deprecation warning to already_initialized_provider_configs property in
both ProvidersManager and ProvidersManagerTaskRuntime, guiding users to use
provider_configs instead.
* Address Amogh's comment: Use VALUE_NOT_FOUND_SENTINEL for provider cfg
fallback consistency
Change fallback=None to fallback=VALUE_NOT_FOUND_SENTINEL in
get_from_provider_cfg_config_fallback_defaults and update the corresponding
sentinel check for consistency with the metadata variant.
* Address Amogh's comment: Fix typo in _use_providers_configuration comment
---------
(cherry picked from commit 4c63306)
<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->
<!--
Thank you for contributing!
Please provide above a brief description of the changes made in this pull
request.
Write a good git commit message following this guide:
http://chris.beams.io/posts/git-commit/
Please make sure that your code changes are covered with tests.
And in case of new features or big changes remember to adjust the
documentation.
Feel free to ping (in general) for the review if you do not see reaction for
a few days
(72 Hours is the minimum reaction time you can expect from volunteers) - we
sometimes miss notifications.
In case of an existing issue, reference it using one of the following:
* closes: #ISSUE
* related: #ISSUE
-->
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR,
please
change below checkbox to `[X]` followed by the name of the tool, uncomment
the "Generated-by".
-->
- [ ] Yes (please specify the tool below)
<!--
Generated-by: [Tool Name] following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
-->
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
