jingi723 opened a new pull request, #74322: URL: https://github.com/apache/airflow/pull/74322
An S3 key such as `dags/.` passes the folder-marker check but becomes `.` after removing the configured prefix with pathlib. Its download destination is then the local bundle directory itself, so boto3 raises `IsADirectoryError` and interrupts the refresh. Skip keys whose relative path is `.` before downloading, as we already do for keys ending in `/`. Normal files, including `.airflowignore`, continue to download and stale files are removed. This addresses the [follow-up identified during review of #73756](https://github.com/apache/airflow/pull/73756#pullrequestreview-5418108795). The regressions cover prefixed, repeated-dot and bucket-root keys in the existing S3Hook tests, plus a bundle refresh after the problematic key is introduced. All four new cases fail without the fix. They use Moto responses with real boto3 downloads and local filesystem writes. Local validation with Python 3.10: - Current main and Airflow 3.0.6: 177 S3Hook/S3DagBundle tests passed in each environment. - Airflow 2.11.0: 161 S3Hook tests passed; Dag bundles require Airflow 3. - Regular and manual static checks passed, including provider mypy. - The Amazon documentation build passed. Compatibility checks used editable provider sources with the installed Airflow versions, rather than the wheel-based CI matrix. The older-core runs selected non-DB tests with `-m 'not db_test'` because the current non-DB plugin mock references an attribute absent in 3.0.6. Live AWS testing was not performed. The broader provider non-DB run stopped at the existing Glue `test_validate_evaluation_results_show_results_True` table-format assertion after 1,098 passes. That test passes in isolation both with and without this S3 change; the cause of the broader-run failure remains undetermined, so the full provider matrix is not claimed as passing locally. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Codex (GPT-6) Generated-by: Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
