marcosmartinezfco opened a new issue, #59465: URL: https://github.com/apache/airflow/issues/59465
### Apache Airflow version Other Airflow 3 version (please specify below) ### If "Other Airflow 3 version" selected, which one? 3.1.4 ### What happened? We are observing **continuous DAG version increments** for DAGs sourced from an S3 DAG bundle (`S3DagBundle`). Even though **no changes** have been made to the DAG files in S3 for several weeks, Airflow keeps generating new DAG versions—for example, the UI currently shows: > **Dag Version(s): v2516** This issue happens **without any DAG code updates**, suggesting that the scheduler or DAG processor is detecting artificial "changes" during bundle sync or serialization. #### Effects we’re seeing: - DAG version numbers increase constantly (v1000 → v2000+). - Metadata DB (`serialized_dag`, `dag_version`) grows unnecessarily. - UI makes it appear as if DAGs are constantly changing. - Scheduler and DAG processor logs do **not** indicate code changes. ### What you think should happen instead? Airflow should only create new DAG versions when: - The DAG file content changes, or - Structural changes to the DAG occur. Since the S3 objects have **unchanged ETags and last-modified timestamps**, new versions should **not** be created. ### How to reproduce No idea, this only happens with one of my Dags which doesn't have any particular special code or anthing ### Operating System AlmaLinux 9.5 (Teal Serval) ### Versions of Apache Airflow Providers apache-airflow-providers-amazon==9.18.0 apache-airflow-providers-celery==3.14.0 apache-airflow-providers-cncf-kubernetes==10.11.0 apache-airflow-providers-common-compat==1.10.0 apache-airflow-providers-common-io==1.7.0 apache-airflow-providers-common-messaging==2.0.1 apache-airflow-providers-common-sql==1.30.0 apache-airflow-providers-docker==4.5.0 apache-airflow-providers-elasticsearch==6.4.0 apache-airflow-providers-fab==3.0.3 apache-airflow-providers-ftp==3.14.0 apache-airflow-providers-git==0.1.0 apache-airflow-providers-google==19.1.0 apache-airflow-providers-grpc==3.9.0 apache-airflow-providers-hashicorp==4.4.0 apache-airflow-providers-http==5.6.0 apache-airflow-providers-microsoft-azure==12.9.0 apache-airflow-providers-mysql==6.4.0 apache-airflow-providers-odbc==4.11.0 apache-airflow-providers-openlineage==2.9.0 apache-airflow-providers-postgres==6.5.0 apache-airflow-providers-redis==4.4.0 apache-airflow-providers-sendgrid==4.2.0 apache-airflow-providers-sftp==5.5.0 apache-airflow-providers-slack==9.6.0 apache-airflow-providers-smtp==2.4.0 apache-airflow-providers-snowflake==6.7.0 apache-airflow-providers-ssh==3.14.0 apache-airflow-providers-standard==1.10.0 ### Deployment Docker-Compose ### Deployment details _No response_ ### Anything else? ## Orchestration - Docker Compose deployment. - Scheduler + webserver + DAG processor running on a r7i.8xlarge instance - Celery workers running on additional EC2 instances. ## Executor - CeleryExecutor ## Metadata Database - PostgreSQL Aurora cluster - Highly available, no failovers detected. - No write issues or DB-level instability. ## Storage - DAGs pulled from S3 DAG Bundle (S3DagBundle) - S3 object ETags and LastModified timestamps confirmed stable. ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
