wjddn279 commented on issue #58509: URL: https://github.com/apache/airflow/issues/58509#issuecomment-3595236737
@Lee-W Yes. Based on my current observations, I did not find any memory leak in the dag-processor when only the example DAGs were present. I tested this locally using Docker, and even when analyzing with Memray, I could not observe any actual increase in heap memory. However, zachliu observed a memory leak under the same conditions and is suspecting two possible causes. First, since he is using a custom image, the issue may be caused by some change introduced in that image. Second, as mentioned above, he suspects that the increase might not be in process memory but in auxiliary kernel memory. From my own experience, I have seen kernel memory grow due to slab memory increasing when writing and closing local log files. While testing with the LocalExecutor, many log files were created, and they occupied several hundred MB per hour. After deleting the log files, the slab memory clearly decreased. This is not exactly a leak, but normal kernel memory management—though from a container perspective, it appears as if memory usage is increasing. The two images below show the difference in memory reported by docker stats before and after deleting the files under log/*. After removing all log files, the container’s memory usage becomes similar to the total memory consumed by the processes inside it. <img width="450" height="225" alt="Image" src="https://github.com/user-attachments/assets/4c957e77-1287-42f8-b55e-c7953e633cae"; /> <img width="450" height="225" alt="Image" src="https://github.com/user-attachments/assets/f4e7830d-9cd8-4c4e-bb93-c491328acadd"; /> [Considering this behavior](https://github.com/apache/airflow/issues/58509#issuecomment-3563803195), the issues appear to be related. It would be helpful for those experiencing the symptoms to confirm whether the memory increase is actually coming from kernel memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
