nickgarza-google-tse opened a new pull request, #72449:
URL: https://github.com/apache/airflow/pull/72449
<!--
Thank you for contributing! Please make sure that your PR meets the
following requirements:
- Pull Request title follows the conventional commit syntax.
- Related issues are linked.
-->
Closes: #72278
Related-to: #72144
### Description
In `BatchSensor`, when running in deferrable mode, `BatchJobTrigger` was
initialized passing only `region_name`, omitting `verify` and
`botocore_config`.
Because `BatchSensor` inherits from `AwsBaseSensor`, it already captures
these configurations during initialization. Without forwarding them to
`BatchJobTrigger`, the triggerer process reconstructs the AWS Batch client hook
using default connection settings, causing custom SSL certificates (`verify`)
and custom `botocore_config` (such as timeouts, retries, and user agents) to be
lost upon deferral.
This change:
1. Passes `verify=self.verify` and `botocore_config=self.botocore_config` to
`BatchJobTrigger` in `BatchSensor.execute()` upon deferral (aligning with
`BatchOperator`).
2. Adds unit test coverage asserting that `region_name`, `verify`, and
`botocore_config` survive and persist into the serialized trigger payload.
---
### Use Case / Motivation
Preserves custom AWS configuration (e.g., custom CA bundles/SSL settings and
botocore connection parameters) across task deferral boundaries when polling
AWS Batch jobs asynchronously.
---
### Related Issues
- Closes #72278
- Sub-issue of #72144
---
### Code Changes Summary
| File | Change |
|---|---|
| `providers/amazon/src/airflow/providers/amazon/aws/sensors/batch.py` |
Passed `verify=self.verify` and `botocore_config=self.botocore_config` to
`BatchJobTrigger`. |
| `providers/amazon/tests/unit/amazon/aws/sensors/test_batch.py` | Added
`test_execute_in_deferrable_mode_passes_aws_configs` verifying parameters in
serialized trigger dictionary. |
---
### Checklists
- [x] Are you creating a PR in a personal branch instead of `main`?
- [x] Tests are added/updated and pass locally using Breeze:
- `breeze testing providers-tests
providers/amazon/tests/unit/amazon/aws/sensors/test_batch.py`
- `breeze testing providers-tests
providers/amazon/tests/unit/amazon/aws/triggers/test_batch.py`
- [x] Static checks pass:
- `breeze static-checks --last-commit`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
