Nataneljpwd commented on issue #50708: URL: https://github.com/apache/airflow/issues/50708#issuecomment-3691155824
Hello, I seem to be getting the same issue, we have multiple airflow deployments, all using the same version, and we have 1 deployment that is experiencing a memory peak, while the other does not, the airflow version used is 2.11, though we are experiencing a memory leak on both the scheduler and the dag processor. If someone has any additional insights, it will aid me in trying to debug the issue. As of now, I am trying to find differences in dags or in the workflows that are used. We have also noticed that when the dag processor faces memory pressure, it starts deleting dags from the db as if they do not exist, logging that the dag is deleted because the file is not present (even though it is), and so if anyone is experiencing a similar issue along with the memory leak, it could help in uncovering a solution to a different issue as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
