potiuk commented on pull request #16404: URL: https://github.com/apache/airflow/pull/16404#issuecomment-870386738
> I'm not sure that is a good behaviour... > If the goal is to protect against a misconfigured secrets backend only, then we should change it to test the settings somehow in get_custom_secret_backend, and if it's not working there, to not use it, rather than to put the try/except around every use of the secrets backends. I quite agree. Just a bit of context @ashb We had big discussion about it in https://github.com/apache/airflow/issues/14592 and my original "extreme" statement was that we should crash Airflow if secret backend is unreachable (I had same concern). Finally we reached a bit "softer" approach (which might allow the tasks to survive secret manager temporary unavailability if the user decides that this is the case). We decideed the crash should only happen when you read configuration (because there are default fallback values which might change airflow behaviour), but there might be some valid cases of fallback for connections or variables. The idea here is that the user **might** then implement scenarios where the fallback values will be OK. It requires a deliberate action from the user - of defining both variable and connection fallback. There are no "defaults" for neither connections nor variables, so unless the users explicitly defines them, the task will not proceed/fail in case secret backend is unreachable (there will be no connection nor variable to use). While this is not as "clean" as ('crash whenever the backend is not available'), it makes sense to be implemented this way. I'd love to hear your thoughts about it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
