dstandish commented on PR #23560:
URL: https://github.com/apache/airflow/pull/23560#issuecomment-1162287832
ok @Taragolis i thought about it some more.
i am thinking maybe we're better off stcking closer to your first approach
and just making search path orderable.
i think originally the problem concern was with the language "low / medium /
high" which did not have an obvious meaning. so we suggested making it take an
abritrary list. but as you have pointed out, there are additional
complexities, and i think these may tip the balance back to the earlier
approach. namely, airflow provides support for grabbing certain "secret"
config values from secrets backend. and this logic assumes there's _one_
custom secrets backend. and given that, as well as the difficulties you've
encountered with trying to implement deprecation of the old values, it seems it
may not be worth the complexity tradeoff to go with the "arbitrary" approach,
and instead go the more limited route.
i think that simply being able to provide an ordering will itself be a huge
value. and for those users who really need to implement multiple external
secrets backends, they can still do so by adding a custom secrets backend that
hits multiple sources.
so i would suggest let's just provide some mechanism for ordering. i can
think of two approaches. probably the simplest is to add an optional key
`backend_ordering_sequence` which could take a value in `[0, 1, 2]`.
alternatively, it could be an optional key `search_path` which would take a
comma-separated `List[str]` of class names e.g. `SSMParameterStoreBackend,
MetastoreBackend, EnvVariablesBackend`.
thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
