neronsoda commented on PR #60074:
URL: https://github.com/apache/airflow/pull/60074#issuecomment-3707876034
@sunank200 @shahar1 @potiuk
I’d like to ask for guidance before opening a PR regarding a provider-side
change :)
In a previous commit, some provider code paths were modified to use SDK
configuration via:
```python
from airflow.providers.common.compat.sdk import conf
```
Unlike Core `conf`, SDK `conf` does not appear to consult
`provider_config_fallback_defaults.cfg`, so provider default values defined
there are not available when using SDK configuration.
Because of this, when provider modules access configuration values at import
time, e.g.:
```python
conf.get("kubernetes_executor", "namespace")
conf.get("local_kubernetes_executor", "kubernetes_queue")
```
I can reproduce `AirflowConfigException` being raised, causing unit test
collection or docs build failures.
At the moment, I see a few possible ways to address this and wanted to ask
which approach is preferred:
1) Explicitly add `fallback=...` to `conf.get()` calls, using defaults
already
defined in `provider_config_fallback_defaults.cfg`
```python
conf.get("kubernetes_executor", "namespace", fallback="default")
```
2) Wrap `conf.get()` calls in try/except `AirflowConfigException`
```python
try:
NAMESPACE = conf.get("kubernetes_executor", "namespace")
except AirflowConfigException:
NAMESPACE = "default"
```
3) (Longer term) Extend SDK `conf` to also include provider fallback defaults
in its lookup chain
From a minimal-change perspective, option 1 (explicit fallbacks) seems
safest,
but I’d appreciate guidance on the recommended pattern for provider code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
