Taragolis commented on PR #23560:
URL: https://github.com/apache/airflow/pull/23560#issuecomment-1125798842
@potiuk @dstandish
Nice idea! Just to clarify some moments
---
1. We want to use list of dictionary, or just dictionary (json-object)
because since Python 3.7 should keep initial order of dictionary keys (CPython
since 3.6)?
```python
secrets = {
"awesome.secrets.Backend": {
"key": "value",
"key2": "another value",
},
"super.secrets.Backend": {},
"airflow.secrets.environment_variables.EnvironmentVariablesBackend": {},
"airflow.secrets.metastore.MetastoreBackend": {},
}
```
---
2. If user doesn't set up every alternative secrets backends than use
default one?
```python
secrets = {
"airflow.secrets.environment_variables.EnvironmentVariablesBackend": {},
"airflow.secrets.metastore.MetastoreBackend": {},
}
```
or
```python
secrets = [
{
"backend":
"airflow.secrets.environment_variables.EnvironmentVariablesBackend",
},
{
"backend": "airflow.secrets.metastore.MetastoreBackend",
},
]
```
---
3. If user setup alternative secrets backend by current parameters
```ini
[secrets]
backend = awesome.secrets.Backend
backend_kwargs = '{"key": "value", "key2": "another value"}'
```
We need to convert to appropriate object with same current order than uses
now: Alternative backend, Env Var, Metastore.
But should we show some kind of depreciation warning?
---
4. If user uses configurations which required get values from secret
backend, e.g. `[database] sql_alchemy_conn_secret` we should try to [get
config](https://github.com/apache/airflow/blob/2111d73a9277c8e036279f2cc59f146270ef8e5b/airflow/secrets/base_secrets.py#L141)
in each secrets backend in same order that this backends defined until found
value or raise an error in case of config not exists.
Only one tricky things, probably we need to some how define is secrets
backend support store configurations. For example:
`airflow.secrets.environment_variables.EnvironmentVariablesBackend` and
`airflow.secrets.metastore.MetastoreBackend` doesn't support by it by design,
and we probably need to skip it.
---
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
