rabsr opened a new issue #16214:
URL: https://github.com/apache/airflow/issues/16214
**Apache Airflow version**: 2.0.2
**Environment**:
- **OS** (e.g. from /etc/os-release): Red Hat Enterprise Linux Server
- **Kernel** (e.g. `uname -a`): Linux
**What happened**:
Using secret backend for sensitive config in airflow.cfg. Specifically when
using secret backend for sql_alchemy_conn option and loading airflow
configuration, it fails with following stacktrace
```
>>> from airflow.configuration import conf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/__init__.py", line
34, in <module>
from airflow import settings
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/settings.py", line
37, in <module>
from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf
# NOQA F401
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 1098, in <module>
conf = initialize_config()
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 860, in initialize_config
conf.validate()
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 199, in validate
self._validate_config_dependencies()
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 227, in _validate_config_dependencies
is_sqlite = "sqlite" in self.get('core', 'sql_alchemy_conn')
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 340, in get
option = self._get_option_from_secrets(deprecated_key,
deprecated_section, key, section)
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 358, in _get_option_from_secrets
option = self._get_secret_option(section, key)
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 317, in _get_secret_option
return _get_config_value_from_secret_backend(secrets_path)
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 83, in _get_config_value_from_secret_backend
secrets_client = get_custom_secret_backend()
File
"/home/user/airflow/lib64/python3.6/site-packages/airflow/configuration.py",
line 999, in get_custom_secret_backend
secrets_backend_cls = conf.getimport(section='secrets', key='backend')
NameError: name 'conf' is not defined
```
**What you expected to happen**:
Airflow configuration should be loaded
**How to reproduce it**:
1. Use secret backend
2. Configure sql_alchemy_conn option in core section to use secret backend
```
[core]
sql_alchemy_conn_secret = sql_alchemy_conn
[secrets]
backend = <any.secret.backend>
```
3. Load Airflow configuration
**Root Cause**:
While initialising Airflow config it validates for db and executor settings.
As sql_alchemy_conn has to be fetched from backend, in
get_custom_secret_backend() it tries to access conf object which is not
initialised yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
