jingi723 opened a new pull request, #73756: URL: https://github.com/apache/airflow/pull/73756
`S3Hook.sync_to_local_dir()` can fail when a directory prefix has no trailing slash and the bucket contains a sibling key with the same string prefix. For example, syncing `dags` also lists `dags_archive/old.py`, which cannot be made relative to the `dags` directory. An object named exactly `dags` can instead be downloaded to the local directory's path, preventing subsequent downloads. This also affects `S3DagBundle` initialization and refresh. Normalize non-empty directory prefixes to end in `/` before listing objects. Empty prefixes still sync the whole bucket, and the existing local path containment check remains in place. Validation: - All six new Moto-backed regression cases fail without the fix: four with `ValueError` and two with `NotADirectoryError`. - The S3 hook and S3 Dag bundle test files pass: 170 tests on Python 3.10.21 in the official Linux/amd64 Breeze CI image. - Regular and manual prek checks pass, including provider-wide mypy. - The complete Amazon unit suite passes when run serially in Breeze: 5,204 passed, 44 skipped. An earlier non-DB run with two xdist workers had one failure in `TestGlueDataQualityHook.test_validate_evaluation_results_show_results_True`, due to Pandas table wrapping in the captured log. The same isolated test fails with two workers on the unchanged base (`7e5c580cd6858b9f873fdcde65bed26182bcf379`) as well. It passes in the complete serial run above. No Glue code or tests are changed here. The broader dependent-provider and older-Airflow compatibility matrices have not been run locally. No live AWS calls were made. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — OpenAI Codex (GPT-6). Generated-by: OpenAI Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
