potiuk commented on issue #16684: URL: https://github.com/apache/airflow/issues/16684#issuecomment-870896142
Yeah. I think you are partially right @mik-laj, if you treat the _CMD as a "glue" to re-format credentials provided in a different format from a more secure way of sharing credentials and the other way has no way to format it in the way it shoudl be provided, the _CMD pattern is quite OK. But .... (And i am not trying to make my point just trying to make it more clear where my thinking comes from and maybe see what you think). The example with Vault is a good one - because it is an implementation of precisely the side-car pattern I explained (Vault injects the secrets exactly via side-car secret injector). So vault already encourages the use of the side-car pattern. And really the question is "where" the formatting glue should be added. From what I know, Vault actually support secret templates that provide very flexible way of injecting the secrets: https://www.vaultproject.io/docs/platform/k8s/injector#secret-templates - and the question here is who should adapt and provide the "glue". Should it be Airflow via _CMD pattern, or should it be secret 'injector'? Note, I do not argue to remove the _CMD pattern altogether from Airflow. It is very useful in case of on-prem/custom deployments which have no other way to provide security. I am arguing that in the Helm/K8S world there are other ways how this can be achieved - where the custom code - if needed - shoudl be written "outside" of Airlfow. And my point is that we should discourage use of _CMD for that. The case with "just reformatting" is fine, but I am afraid (and this is my whole point) that the _CMD pattern encourages people to write scripts that provide the "temporary" where the scripts themselves contain "hidden/obfuscated" credentials. For example I can imagine one can write a script to communicate with a secret manager where the "user/password" to the secret manager is embedded (and maybe obfuscated) in the script itself. This is the "false sense of security" that I am talking about. For me the example you wrote for re-formatting the secret is already the "custom code" that you need to write. And my argument is that either you do not need it (like vault + templating + maybe adding configuration to FileSystemBackend) or writing custom code (might be a bash script) in your "side-car-secret-injector". Is much better choice, which encourages much more secure patterns. Even if it is a bit more difficult. BTW. I was not aware FileSystemBackend does not support configuration (other secret backends support it and it causes some pain in configuration class). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
