potiuk commented on issue #9713: URL: https://github.com/apache/airflow/issues/9713#issuecomment-1181479699
I think this problem might be solved differently without the need to use redis/cache. I think when this one gets into final shape https://github.com/apache/airflow/pull/23560, you will be able to configure the sequence of retrieval of variables/connecton retrievaland you wil be able to put Metadata DB before Secrets. Currently Secrets are queried always first, and only if a value is not found in secret, the fallback will read it from the DB. And if you keep just a few of those in secrets manager, that makes vast majority of such requests needless. With #23560 you will be able to reverse it and only go to secrets when you won't find it in the database. That should vastly reduce the amount of requests to Vault or any other secret storage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
