potiuk commented on pull request #19857: URL: https://github.com/apache/airflow/pull/19857#issuecomment-980644388
Ah nice. Indeed Airflow connection URI IS trricky. Comment while you are at it @dstandish, as this seems very much related However I think this does not solve a specific case that several of our users mentioned. And I am just thinking whether we maybe also try to solve it systemically rather than telling the users "Write your custom Secrets Backend. There is a case where the users already have processes and tools to automatically rotate their credentials, but only when they are standalone "values" - not part of URI, not part of dictionary. The credentials (passwords/tokens etc). are the only things that change when they are rotated - all the rest - user,URIs, extras remain as they were. That for me is really, really valid use case - where part of the connection (the non-secret one) is 'static' but token is dynamic. And the tools that the organisation has treats those two separately. Example issue https://github.com/apache/airflow/issues/19217 (but there were few other similar discussions). I do not have a "perfect" solution for that but I thought about an options of joinng the two options allow to connect "metadata" connection with "secret" credentials. One possible solution I could imagine the case that you configure your connection in Airflow DB and in the place of password you put: ``` SECRET:/connections/my_secret_password ``` Then airflow could combine thet two and retrieve the metadata for most of the conneciton and secret from the secrets manager. I think it should be rather easy to implement something like that (and have a flag in secrets which would allow to combine metadata + secrets). WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
