LukeScraper opened a new issue, #54981:
URL: https://github.com/apache/airflow/issues/54981
### Apache Airflow Provider(s)
apache-beam
### Versions of Apache Airflow Providers
apache-airflow-providers-apache-beam==6.0.4
### Apache Airflow version
2.10.5
### Operating System
Debian GNU/Linux 12 (bookworm)
### Deployment
Docker-Compose
### Deployment details
Simple deployment with docker-compose
### What happened
I am running a beam pipeline with the BeamRunPythonPipelineOperator. Some of
the pipeline options are sensitive. When i run the pipeline normally, sensitive
vars such as password or api_token are masked correctly. However when i defer
the task, those pipeline options are logged and not masked.
### What you think should happen instead
The sensitive vars should be masked when the task is deferred just how it is
done when the pipeline is not deferred
### How to reproduce
`if self.deferrable:
self.defer(
trigger=BeamPythonPipelineTrigger(
variables=self.snake_case_pipeline_options,
py_file=self.py_file,
py_options=self.py_options,
py_interpreter=self.py_interpreter,
py_requirements=self.py_requirements,
py_system_site_packages=self.py_system_site_packages,
runner=self.runner,
gcp_conn_id=self.gcp_conn_id,
),
method_name="execute_complete",
)
else:
self.beam_hook.start_python_pipeline(
variables=self.snake_case_pipeline_options,
py_file=self.py_file,
py_options=self.py_options,
py_interpreter=self.py_interpreter,
py_requirements=self.py_requirements,
py_system_site_packages=self.py_system_site_packages,
)`
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
