shahar1 opened a new pull request, #61611:
URL: https://github.com/apache/airflow/pull/61611
## Description
Fixes #22021
The `CloudDataTransferServiceUpdateJobOperator` was passing the full patch
request body to `TransferJobPreprocessor`, but that body wraps the
`TransferJob` under a `transfer_job` (or `transferJob`) key. The preprocessor
looks for `transferSpec` at the top level and thus never found it, skipping AWS
credential injection for S3 sources.
### Changes
1. **Operator Fix** (`cloud_storage_transfer_service.py`):
- Added `_get_transfer_job_body()` method that extracts the inner
`TransferJob` dict from the patch request body
- Supports both snake_case (`transfer_job`) and camelCase (`transferJob`)
key variants
- Updated `_validate_inputs()` and `execute()` to use the extracted body
for preprocessing and validation
2. **Test Coverage** (`test_cloud_storage_transfer_service.py`):
- Added `test_job_update_with_aws_s3_source_camel_case`: Verifies AWS
credentials are injected with camelCase body
- Added `test_job_update_with_aws_s3_source_snake_case`: Verifies AWS
credentials are injected with snake_case body
- Added `test_job_update_with_aws_role_arn_does_not_inject_credentials`:
Verifies role ARN credentials are not overridden
### Testing
All 74 tests in the operators test suite pass, including the 3 new tests and
the existing update operator tests.
---
- [X] Yes (please specify the tool below)
Generated-by: GitHub Copilot following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
