Taragolis commented on PR #23560: URL: https://github.com/apache/airflow/pull/23560#issuecomment-1174010308
@potiuk @dstandish tl;dr: After tried to change something in secrets backends I found that might be better to try make this mechanism more transparent first at least for developers and after that we could go back to backend order/configurations. I don't think that is possible completely solve chicken-egg unfortunately. --- In each case if we change order or add ability to configure multiple backends we got some false positive warnings. **Variables expected that Metastore always last when created Variable in UI** https://github.com/apache/airflow/blob/e2fd41f7b14adef2c3a88dde14d088b5ef93b460/airflow/models/variable.py#L226-L237 --- Some additional findings **Connections and Variables have methods for lookup values which might be better to move in some SecretsBackendsClass** https://github.com/apache/airflow/blob/e2fd41f7b14adef2c3a88dde14d088b5ef93b460/airflow/models/variable.py#L257-L275 https://github.com/apache/airflow/blob/e2fd41f7b14adef2c3a88dde14d088b5ef93b460/airflow/models/connection.py#L410-L430 **List variables/connections use different approach rather than get** Get variables/connections stored in metastore - use `airflow.secrets.metastore.MetastoreBackend` List variables/connections outside of `airflow.secrets.metastore.MetastoreBackend` I think most of (probably all) secrets backends which implemented in airflow or in community providers could implemented List operation. Which make possible to show this variables/connections in UI/CLI **No information for end-user where received variable/connection actually stored** Personally spend additional time last Friday just because I assume that we do not create specific connection in secrets backend. **Implicit ignore for end-users any json.Decoding errors in backend_kwargs** `¯\_(ツ)_/¯` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
