SameerMesiah97 opened a new pull request, #64051: URL: https://github.com/apache/airflow/pull/64051
**Description** This change enforces `execution_timeout` for `AirbyteTriggerSyncOperator` in deferrable mode. Previously, when the operator deferred, `execution_timeout` was not enforced, allowing Airbyte jobs to continue running after the Airflow task had timed out. The operator now computes an execution deadline before deferring, the trigger emits a timeout event when the deadline is exceeded, and the operator cancels the Airbyte job and fails the task when the event is received. **Rationale** In non-deferrable mode, `execution_timeout` is enforced by the scheduler, which terminates the task process and invokes `on_kill()` to cancel the external Airbyte job. In deferrable mode, execution is handed off to a trigger running in the triggerer process, so there is no worker process to terminate. However, this does not change the expected task semantics. From a user perspective, `execution_timeout` is a **hard task-level limit and should behave consistently regardless of execution mode**. Without explicit handling in the trigger/operator interaction, `execution_timeout` becomes a no-op in deferrable mode, leading to leaked Airbyte jobs and inconsistent behavior. This is the same class of issue addressed in PR #61472 for `DbtCloudRunJobOperator`. **Notes** * The existing `timeout` parameter only controls how long the operator waits for job completion and does not imply cancellation. * When both `execution_timeout` and `timeout` are set, the earlier deadline takes precedence. * Job cancellation on timeout is best-effort. Failures during cancellation are logged and do not mask the original timeout failure. * In `execute_complete`, generic `AirflowException` raises have been replaced with more appropriate, specific exception types. * Before/after behavior screenshots can be provided upon request. **Tests** * Added a trigger-level test asserting a timeout event when the execution deadline is exceeded. * Added an operator-level test verifying job cancellation on execution timeout. * Added an operator-level test verifying error propagation when job cancellation fails. * Modified a trigger serialization test to ensure the optional `execution_deadline` field is always serialized. **Documentation** * Updated the trigger docstring to document the new `execution_deadline` parameter. * Updated the operator docstring to clarify the behavior of `timeout` vs `execution_timeout`; added an entry for `execution_timeout`. **Backwards Compatibility** This change does not modify public APIs or method signatures. Behavior is changed such that Airbyte jobs are now cancelled when `execution_timeout` is reached in deferrable mode. Previously, jobs could continue running after the task timed out. Closes: #64048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
