afonsojanu opened a new pull request, #72210: URL: https://github.com/apache/airflow/pull/72210
Fixes #72135 BeamRunJavaPipelineOperator and BeamRunPythonPipelineOperator both use `self.dataflow_config.impersonation_chain` when running the pipeline directly, but when deferrable mode is on and they build `trigger_args` for `DataflowJobStatusTrigger` / `DataflowJobStateCompleteTrigger`, that value never gets passed through. So a deferred task ends up polling Dataflow for completion with whatever credentials the worker has, not the impersonated account the operator was configured with. Both trigger classes already accept `impersonation_chain` in their constructor (see `airflow/providers/google/cloud/triggers/dataflow.py`), so this is just a matter of including it in the dict the operators build. While looking at this I noticed the exact same pattern shows up in both operators, the Java one that the issue reports and the Python one that wasn't reported but has the identical bug, so I fixed both. I added a test for each operator asserting the trigger it raises actually carries the configured impersonation_chain. I couldn't run the full provider test suite locally since it needs the breeze dev environment, but I verified the fix against the real installed package by patching in the modified module and driving it through the exact same code path the tests exercise, confirming it fails without the change (impersonation_chain comes through as None) and passes with it. Ran ruff on both changed files with no issues. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
