[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-599449599 Obvs I defer to you guys on voting. By "general secret backend" I think you mean arbitrary get secret method, i.e. in addition to get connection. Personally I am not convinced of the need / value of this, but welcome your sales pitch :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-599026300 > also rebase to latest master. Pydruid fix is merged done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-598927994 update: changed config to this: ```ini [secrets] backend = backend_kwargs = ``` touched up the docs so. what's next step? does there need to be vote on this? should i do anything in particular re the AIP? @potiuk i take it from your approval that you are satisfied that your `get_secret` idea can be discussed and implemented after this AIP (e.g. nothing about current structure of this PR makes the realization of that idea less likely)? thanks a bunch to everyone for the help in getting this into shape This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-598927994 update: changed config to this: ```ini [secrets] backend = backend_kwargs = ``` touched up the docs so. what's next step? does there need to be vote on this? should i do anything in particular re the AIP? @potiuk i take it from your approval that you are satisfied that your `get_secret` idea can be discussed and implemented after this AIP (e.g. nothing about the structure of this makes the realization of that less likely)? thanks a bunch to everyone for the help in getting this into shape This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-598589810 update: i made two enhancements 1. the backend config is now passed directly to class init, e.g. [here](https://github.com/apache/airflow/pull/6376/files#diff-2e589119610e1b6ffe1ff76755881a4bR46-R49) this makes it easier to understand what configuration options are possible this also means that config parsing logic is fully contained in `secrets/__init__.py` rather than left to the subclass 2. backend classes are now [imported and instantiated at top level](https://github.com/apache/airflow/pull/6376/files#diff-3179456596a893fb9e46e105cbb4390cR77) in `secrets/__init__.py`. this way we no longer import and instantiate the backend classes once for each call to `get_connections`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-598589810 update: i made two enhancements 1. the backend config is now passed directly to class init, e.g.: https://github.com/apache/airflow/pull/6376/files#diff-2e589119610e1b6ffe1ff76755881a4bR46-R49 this makes it easier to understand what configuration options are possible this also means that config parsing logic is fully contained in `secrets/__init__.py` rather than left to the subclass 2. backend classes are now imported and instantiated at top level in `secrets/__init__.py`. this way we no longer import and instantiate the backend classes once for each call to `get_connections`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM URL: https://github.com/apache/airflow/pull/6376#issuecomment-598350630 > Have you thought about adding generic "get secret" implementation ?I am not sure if this has been rejected so far or at all :)? Not rejected at all. Been meaning to circle back to it. Just trying to chip away at the less open-ended (and more consensus-ready) items first. I will think about it soon and either add something or comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend
and support for AWS SSM
URL: https://github.com/apache/airflow/pull/6376#issuecomment-598084161
updated:
* can only specify one "alternative" secrets backend
* default search path is `env var > metastore`; if a backend is specified,
it becomes `alternative > env var > metastore`
Config looks like this:
```
[secrets_backend]
class_name = my.fave.SecretsBackendClass
config_json = {"my secrets class": "needs json to work properly"}
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend
and support for AWS SSM
URL: https://github.com/apache/airflow/pull/6376#issuecomment-598084161
updated:
* can only specify one "alternative" secrets backend
* default search path is `env var > metastore`; if a backend is specified,
it becomes `alternative > env var > metastore`
Config looks like this:
```
[secrets_backend]
class_name = my.fave.SecretsBackendClass
config_json = {"my secrets class": "needs json to work properly"}
```
One thing I liked about the earlier iteration is that airflow had no opinion
about secrets backend; now we have default secrets backends and optionally
"alternative" secrets backend, which is maybe a little confusing. But overall
makes the config simpler so I'm still in support of the now-current approach.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend
and support for AWS SSM
URL: https://github.com/apache/airflow/pull/6376#issuecomment-598084161
updated:
* can only specify one "alternative" secrets backend
* default search path is `env var > metastore`; if a backend is specified,
it is `alternative > env var > metastore`
Config looks like this:
```
[secrets_backend]
class_name = my.fave.SecretsBackendClass
config_json = {"my secrets class": "needs json to work properly"}
```
One thing I liked about the behavior before is that airflow had no opinion
about secrets backend; now we have default secrets backends and optionally
"alternative" secrets backend, which is maybe a little confusing. But overall
makes the config simpler so I'm still in support of the now-current approach.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend and support for AWS SSM
dstandish edited a comment on issue #6376: [AIRFLOW-5705] Add secrets backend
and support for AWS SSM
URL: https://github.com/apache/airflow/pull/6376#issuecomment-598084161
updated:
* can only specify one "alternative" secrets backend
* default search path is `env var > metastore`; if a backend is specified,
it becomes `alternative > env var > metastore`
Config looks like this:
```
[secrets_backend]
class_name = my.fave.SecretsBackendClass
config_json = {"my secrets class": "needs json to work properly"}
```
One thing I liked about the behavior before is that airflow had no opinion
about secrets backend; now we have default secrets backends and optionally
"alternative" secrets backend, which is maybe a little confusing. But overall
makes the config simpler so I'm still in support of the now-current approach.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
