Tomeshwari-02 opened a new issue, #69903:
URL: https://github.com/apache/airflow/issues/69903
### What happened and how to reproduce it?
`HttpSensor` supports `response_error_codes_allowlist` so users can treat
selected HTTP errors as "keep poking" instead of failing the task. In the
synchronous path this is honored by checking
`self.response_error_codes_allowlist` in `HttpSensor.poke()`.
However, in deferrable mode the behavior appears to diverge after the task
is deferred:
- `providers/http/src/airflow/providers/http/sensors/http.py` stores the
configured allowlist and uses it in `poke()`.
- `HttpSensor.execute()` creates an `HttpSensorTrigger`, but does not pass
`response_error_codes_allowlist` to the trigger.
- `providers/http/src/airflow/providers/http/triggers/http.py` has
`HttpSensorTrigger.run()` retry only when the exception string starts with
`"404"`.
A DAG using something like this can behave differently depending on whether
deferrable mode is enabled:
```python
HttpSensor(
task_id="wait_for_api",
endpoint="/health",
http_conn_id="my_api",
response_error_codes_allowlist=["404", "503"],
deferrable=True,
)
```
Expected behavior: `503` should be treated the same way as in non-deferrable
mode, i.e. the sensor should keep waiting according to `poke_interval`.
Current behavior: the initial worker-side `poke()` can honor `503`, but
after deferral the trigger only handles `404`. For other allowlisted statuses
such as `503`, the trigger does not sleep via the allowlist path and does not
preserve the documented sensor behavior.
### What you think should happen instead?
`HttpSensorTrigger` should accept and serialize the configured
`response_error_codes_allowlist`, and `HttpSensor.execute()` should pass the
value when creating the trigger.
The trigger should use that allowlist instead of hard-coding `404`, so
deferrable and non-deferrable `HttpSensor` behavior are consistent.
A unit test should cover a deferrable `HttpSensor` with a custom allowlist
such as `["404", "503"]`.
### Operating System
N/A
### Versions of Apache Airflow Providers
Current `main` branch, HTTP provider code under `providers/http`.
### Deployment
N/A
### Deployment details
N/A
### Anything else?
The existing sync-path tests already cover custom
`response_error_codes_allowlist`; the missing coverage appears to be the
deferrable trigger path.
### Are you willing to submit PR?
Yes, I am willing to submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
