RNHTTR commented on issue #39463: URL: https://github.com/apache/airflow/issues/39463#issuecomment-2100642010
> State Awareness and Exception Management: It would handle different states (like success, skipped, and failed) and raise exceptions (e.g., AirflowFailException, AirflowSkipException) when a specific state is reached. This ensures that task dependencies are managed correctly. Can't that be achieved with the `HTTPSensor`'s `response_check` parameter? This allows you to pass a function that evaluates the response to the `HTTPSensor` for some condition. So, if you query the `taskInstances` [endpoint of the Airflow REST API ](https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/get_task_instance), you can configure your `response_check` accordingly. You could then use existing [TI states](https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/tasks.html#task-instances) to determine how to respond to a given state. > Advanced Error Handling: It would differentiate between transient network errors (retries with a threshold) and API-related issues, providing more robust error handling. I believe this can also be done both with `HTTPSensor` parameters (for example `tcp_keep_alive_count `) and normal Airflow retries. > Deferrable Execution: The sensor would be able to defer execution to a custom WorkflowTrigger. The `HTTPSensor` already supports deferrable execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
