filipeaaoliveira opened a new pull request, #71092: URL: https://github.com/apache/airflow/pull/71092
Hello everyone. While upgrading Airflow to 3.3.0 yesterday, our Airbyte DAGs started failing on every trigger even though the syncs themselves were starting in Airbyte. The cause is the provider's move to **airbyte-api 1.x**, which replaced requests with **httpx** and with it picked up httpx's 5-second default request timeout. Provider 6.0.0 requires the 1.x SDK, but 5.5.1 hits it too: its **airbyte-api>=0.52.0** pin has no upper bound ([#69081](https://github.com/apache/airflow/pull/69081) added the <1.0.0 cap after 5.5.1 shipped), and constraints-3.3.0 resolves it to **airbyte-api==1.0.1**. On self-hosted deployments, job creation can take far longer than 5 seconds under load — we measured POST /v1/jobs at 30–130 seconds with 9 connections triggered in parallel. Every trigger task then fails with `httpx.ReadTimeout` raised from `AirbyteHook.submit_sync_connection`. And because create-job is not idempotent, the timed-out request still creates the job server-side, so retries get 409 "A sync is already running" and the attempt that eventually succeeds starts a duplicate sync. The hook currently exposes no way to change the timeout. This PR lets users set it through an `AirbyteHook` parameter or a connection extra, passing it to the SDK's existing `timeout_ms`. With neither set, the session is built exactly as before, so default behavior is unchanged. ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code - Fable 5 following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
