We use a dynamic config where we iterate through a JSON file, and all sensitive info (like api keys, aws keys, etc...) are pulled from a remote k/v store when airflow starts and adds them as fields to the JSON config file.
On Tue, Sep 19, 2017 at 6:54 PM, Michael Crawford < [email protected]> wrote: > Did my message go through? I have never tried to send an email to the > list before, only silently monitored. > > Does anyone have any ideas? I would be happy to create an issue and code > up the fix myself, but I just wanted to ping here first to make sure I > wasn’t missing anything and try to get a consensus on how to handle this. > > Thanks, > Mike > > > On Sep 18, 2017, at 8:03 PM, Michael Crawford <michael.crawford@ > modernizingmedicine.com> wrote: > > > > Hi, > > > > I was wondering if anything had ever been proposed for having the aws > secret key hidden in the aws type connection. > > > > Currently passing in these credentials is done by defining the some json > in the extra params section of the connection like > > {"aws_access_key_id":"_your_aws_access_key_id_", > "aws_secret_access_key": "_your_aws_secret_access_key_”} > > > > While this does work it leaves the secret access key in plain text for > anyone that has access to the connections. > > > > I know there are other options about setting them as environment > variables, but this doesn’t help if we need to define more than one aws > connection with different access keys. > > > > Two things that immediately came to mind for how to do this: > > > > 1. use login and password sections of the connection for the access and > secret keys so that the secret gets hidden and encrypted like all the other > passwords. > > 2. have an option to encrypt the extra params > > > > Option 1 seems most logical and should be too hard to implement. > > > > Open to any ideas people might have on this. > > > > Thanks, > > Mike > >
