ashb opened a new pull request #15599:
URL: https://github.com/apache/airflow/pull/15599
Mask secret values from connections and variables
This masks secret values in logs for Connections and Variables.
It behaves as follows:
- Connection passwords are always masked, where-ever they appear.
This means, if a connection has a password of `a`, then _every_ `a` in
log messages would get replaced with `***`
- "Sensitive" keys from extra_dejson are also masked. Sensitive is
defined by the "existing" mechanism that the UI used, based upon the
name of the key.
- "Sensitive" Variables are also masked.
commit 204ca666087c223a2c29ba7dde3349fed6375c53
Author: Ash Berlin-Taylor <[email protected]>
Date: Thu Apr 29 12:33:23 2021 +0100
Add a logging filter to mask secrets from logs
This isn't used anywhere yet, but it is the first step in not printing
passwords in the logs
**Still to do**:
- [ ] Add docs for this
- [ ] Unittest automatic masking from Connection properties
- [ ] Unitest automatic masking from Variables.
**Example**
Given this Task:
```python
@task
def foo():
conn = BaseHook.get_connection('foo_bar')
print(conn.password)
logging.info("Password is %s", conn.password)
logging.info("Variable %s is %s", "api_key", Variable.get("api_key"))
logging.info("flibble is masked?")
```
We get the following lines in the log:
```
[2021-04-29 22:27:34,180] {base.py:78} INFO - Using connection to: id:
foo_***. Host: example.com, Port: None, Schema: , Login: foo, Password: ***,
extra: {'foo': '***', 'quxx': 'bazz'}
[2021-04-29 22:27:34,181] {logging_mixin.py:104} INFO - ***
[2021-04-29 22:27:34,181] {foo.py:18} INFO - Password is ***
[2021-04-29 22:27:34,182] {foo.py:19} INFO - Variable api_key is ***
[2021-04-29 22:27:34,183] {foo.py:20} INFO - *** is masked?
```
Closes #8421 8421
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
