khuper opened a new pull request, #72228: URL: https://github.com/apache/airflow/pull/72228
S3 Dag bundles currently sync the latest objects from a prefix, so a Dag run cannot reliably load the exact code it started with. Multi-object deployments can also be observed partway through an update.\n\nThis adds an optional mode for Airflow 3.4 and later. It:\n\n- pins each release to exact S3 object VersionIds, sizes, and SHA-256 hashes\n- publishes releases through a small current pointer written last\n- verifies downloads in a staging directory before an atomic local rename\n- keeps older generations available for tasks, callbacks, and in-flight parsing\n- leaves the existing unversioned S3 behavior unchanged\n\nThe change also fixes bundle generation lease handling and keeps callback bundle metadata tied to the Dag run version.\n\nTests cover manifest validation, historical versions, partial downloads, integrity failures, concurrent initialization, requester-pays requests, parser and worker leases, stale parse results, and callback version consistency.\n\n---\n\n##### Was generative AI tooling used to co-author this PR?\n\n- [X] Yes: Codex (GPT-5)\n\nGenerated-by: Codex (GPT-5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)\n\n---\nDrafted-by: Codex (GPT-5) (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
