muditagrawal2007 opened a new issue, #75069: URL: https://github.com/apache/airflow/issues/75069
### Describe the bug `RedactedIO.write()` and `RedactedIO.writelines()` are supposed to mask registered secrets before any output reaches the terminal. They do — for `str` values. For `bytes` (and `bytearray`) values, the registered secret is written to stdout in cleartext, wrapped in Python's `bytes` repr (`b'…'`). The only production call site today is `airflow tasks test` ([`airflow-core/src/airflow/cli/commands/task_command.py:539`](https://github.com/apache/airflow/blob/main/airflow-core/src/airflow/cli/commands/task_command.py#L539)), which wraps task execution in `redirect_stdout(RedactedIO())`. Any task that calls `sys.stdout.writelines([b"…secret…"])` — common with subprocess wrappers, log buffers, and some logging libraries — will have its secrets printed verbatim. ### Reproducer ```python from airflow.sdk._shared.secrets_masker import mask_secret, RedactedIO mask_secret("hunter2") # str — masked RedactedIO().write("here is hunter2") # -> here is *** # bytes containing the same secret — NOT masked RedactedIO().write(b"here is hunter2") # -> b'here is hunter2' <-- secret leaked as bytes repr ``` Equivalently, after the `airflow tasks test` wrapper is active inside a Dag: ```python import sys def my_task(**_): sys.stdout.writelines([b"authorization: Bearer hunter2\n"]) # Writes "authorization: Bearer b'authorization: Bearer hunter2\n'" # to the console. The token is now in scrollback. ``` ### Expected `RedactedIO().write(b"...hunter2...")` should produce the same masked output as `RedactedIO().write("...hunter2...")` — `...***...`. ### Actual The bytes are passed to `redact()` first, and `redact()` does not understand `bytes` — they fall through `_redact`'s type checks to the default `return item` at [`secrets_masker.py:430`](https://github.com/apache/airflow/blob/main/shared/secrets_masker/src/airflow_shared/secrets_masker/secrets_masker.py#L430), so the bytes object is returned unchanged. The `str(...)` wrapper in both `write()` ([line 720-722](https://github.com/apache/airflow/blob/main/shared/secrets_masker/src/airflow_shared/secrets_masker/secrets_masker.py#L720)) and `writelines()` ([line 724-725](https://github.com/apache/airflow/blob/main/shared/secrets_masker/src/airflow_shared/secrets_masker/secrets_masker.py#L724)) then converts the bytes to their `b'…'` repr, which writes the cleartext secret to the underlying stream. ### Why this matters `RedactedIO` exists so that output produced by user code is masked before it reaches stdout. Its docstring says "Writes to stdout will be redacted." That promise is silently broken for the bytes path. Because the wrapper is applied to the entire task run, any user code that ever produces a `bytes` write through stdout is affected — and there's no error to warn the caller, the secret is just printed. This was introduced into the `bytes` path of `writelines()` by #72063 (which deliberately propagated the behavior of the pre-existing `write()`). The `write()` method has had the same issue for a long time, but the only recently-added bytes-supporting code path is `writelines()`. ### Proposed fix Decode `bytes`/`bytearray` to UTF-8 (with replacement) before calling `redact()` in both methods, so the existing pattern-based masker can see the content. Something like: ```python def _redact_io_value(value) -> str: if isinstance(value, (bytes, bytearray)): value = value.decode("utf-8", errors="replace") return str(redact(value)) ``` Both `write()` and `writelines()` would call this helper, keeping them symmetric. The permissive behavior for non-string types is preserved (per the explicit intent of #72063), but bytes are now actually scanned for registered secrets. ### Area - [x] secrets_masker - [x] CLI (`airflow tasks test` is the only reachable production call site) ### Tested Airflow version main (commit `935bac04f0`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
