LourTV commented on issue #55768: URL: https://github.com/apache/airflow/issues/55768#issuecomment-3360876034
@wjddn279 In our case, the significant growth in the dag_version and serialized_dag tables was caused by the use of a dynamic start_date in the DAG definition (start_date = datetime.now() - timedelta(days=1)). This resulted in Airflow generating a different hash on each execution, thereby creating a new DAG version every time. After identifying this issue, we updated the DAG to use a fixed start_date. Since making this change, the number of DAG versions has stabilized and is no longer increasing. Following this adjustment, we restarted Airflow and monitored the system for several hours to assess whether the memory usage issue persisted. Unfortunately, the results remain the same, although Airflow is no longer generating excessive DAG versions, we continue to observe a gradual increase in RAM and swap usage over time, eventually leading to an out-of-memory condition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
