galafis commented on issue #54981: URL: https://github.com/apache/airflow/issues/54981#issuecomment-4068960955
Hey, I took a look at this and I think I understand what's going on. When the operator runs in non-deferred mode, Airflow's `SecretsMasker` catches sensitive values in the logs because they go through the standard logging pipeline. But in deferred mode, the trigger gets serialized via `serialize()` which dumps the raw `variables` dict (including passwords, API tokens, etc.) into the trigger's serialized state. When the triggerer process picks it up and logs the trigger info, those values show up unmasked. I traced it through `BeamPythonPipelineTrigger.serialize()` in `providers/apache/beam/src/airflow/providers/apache/beam/triggers/beam.py` - the `variables` dict is returned as-is. The fix should ensure that sensitive keys in the variables are properly masked when the trigger state is logged. I'd like to work on this if it's available. Will have a PR up shortly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
