uranusjr commented on a change in pull request #21281:
URL: https://github.com/apache/airflow/pull/21281#discussion_r798211852
##########
File path: airflow/utils/log/secrets_masker.py
##########
@@ -252,3 +253,44 @@ def add_mask(self, secret: Union[str, dict, Iterable],
name: Optional[str] = Non
elif isinstance(secret, collections.abc.Iterable):
for v in secret:
self.add_mask(v, name)
+
+
+class StdoutRedactContext:
+ """
+ This class redirects all stdout within the context through the logger
specified in the init.
+ Print statements get turned into log lines through the logger member.
+
+ """
+
+ def __init__(self):
+ self.stdout_lines = []
+ self._redirector = contextlib.redirect_stdout(self)
+
+ def write(self, stdout_line):
+ """Write stdout to the internal buffer when stdout is written to."""
+ self.stdout_lines.append(redact(stdout_line))
+
+ def flush(self):
+ """
+ No flush functionality needed here, but this method is needed to make
this class a "file-like" which
+ can be called by the context lib in self._redirector.
+
+ """
+
+ def __enter__(self):
+ """
+ At the beginning of the context, enter the "redirect mode" which is
initiated by entering the context
+ of `contextlib.redirect_stdout`.
+
+ """
+ self._redirector.__enter__()
+
+ def __exit__(self, *args):
+ """
+ On exit from the context, close the `contextlib.redirect_stdout`
context to stop capturing lines
+ from stdout, then route all lines one by one through self.redirect_to.
+
+ """
+ self._redirector.__exit__(*args)
+ for log in self.stdout_lines:
+ sys.stdout.write(log)
Review comment:
Hmm, this means all of the task’s output will be buffered, and only spit
out in one go when the task is finished, which seems wrong. I would call
`stdout.write` in `write` directly after redaction instead.
This can be significantly simplified to
```python
class StdoutRedactor:
@contextlib.contextmanager
@classmethod
def enable(cls):
with contextlib.redirect_stdout(cls()):
yield
def write(self, content):
sys.stdout.write(redact(content))
def flush(self):
sys.stdout.flush()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
