[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 Bug Janitor Service changed: What|Removed |Added Resolution|WAITINGFORINFO |WORKSFORME Status|NEEDSINFO |RESOLVED --- Comment #10 from Bug Janitor Service --- This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging Thank you for helping us make KDE software even better for everyone! -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 --- Comment #9 from Bug Janitor Service --- Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone! -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 Justin Zobel changed: What|Removed |Added Status|REPORTED|NEEDSINFO Resolution|--- |WAITINGFORINFO --- Comment #8 from Justin Zobel --- Thank you for the crash report. As it has been a while since this was reported, can you please test and confirm if this issue is still occurring or if this bug report can be marked as resolved. I have set the bug status to "needsinfo" pending your response, please change back to "reported" or "resolved/worksforme" when you respond, thank you. -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 --- Comment #7 from Milian Wolff --- @Sven, your idea can still fail. When we schedule, the clang parse jobs may not yet use a lot of memory. But later when they hit $insane-template-magic-code they suddenly sky-rocket memory wise and then we fail. @JDF: Imposing a 512MB memory limit on Clang will probably lead to "OOM" kills of the parse jobs as soon as you encounter non-trivial template magic. That said, this is still the best idea so far, I think. Someone should try that out and see what happens and how clang behaves in that scenario. If a parse thread dies, we should be able to recover from that gracefully. It's certainly better than hogging up more memory than is available. So we could maybe start off by dividing the total available memory over the number of configured parse jobs and see how it goes. -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 --- Comment #6 from Sven Brauch --- After a bit of quick looking around, I think one realistic option would be to read current memory usage from /proc/1234/mem before starting a parse job, and defer starting the job if that is already above a certain threshold and jobs are running. For Windows, we can do something similar, there isn't going to be a cross-platform solution I think. -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 --- Comment #5 from JDF --- Here below are some links that might be useful. It seems that prlimit and timeout can be interesting. If you know that you have let's say 16GB on the machine, and the user wants that 8GB maximum will be used for parsing and that he wants 16 threads to do the work, then only 512 MB will be given for each clang process. That looks too easy, and since I'm not inside kdevelop code (and know nothing about how clang is doing code parsing) I might be missing important things that will kill these ideas... But other solutions might exist: https://github.com/valloric/youcompleteme/issues/184 https://bugreports.qt.io/browse/QTCREATORBUG-11640 https://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process https://unix.stackexchange.com/questions/1424/is-there-a-way-to-limit-the-amount-of-memory-a-particular-process-can-use-in-uni https://letitknow.wordpress.com/2014/12/11/how-to-limit-memory-usage-of-a-process-under-linux/ -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 --- Comment #4 from Milian Wolff --- Yes, I agree. The question is: how? ulimit on the whole process is one way to do $something, but it will just lead to a crash earlier. We somehow need to figure out a way to limit clang's memory consumption, or guesstimate the memory consumption per thread and limit the number of threads based on that then... Suggestions welcome! -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 Sven Brauch changed: What|Removed |Added CC||m...@svenbrauch.de --- Comment #3 from Sven Brauch --- Maybe this is a good idea, yes. Other potentially memory-hungry projects do this as well (e.g. krita). -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 JDF changed: What|Removed |Added CC||dage...@free.fr --- Comment #2 from JDF --- The bug is still present (kdevelop 5.2.1). On a 16 cores CPU, when setting 16 cores, the parser will use all the available memory (8GB + 16 GB swap) until it crashes. Reducing the parser to 12 cores will not show up the problem on the above architecture (the parser will tend to use up to 6GB or RAM and few GB of swap). Can be interesting to set a maximum of memory usage along with a maximum number of cores. -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 --- Comment #1 from gnu...@gmail.com --- Hi, started to test KDevelop 5.0.3 and behaviour seems to be much better now. Only 3GB memory usage instead of >20GB! for the same large project. Thanks for your effort! Best, gnux -- You are receiving this mail because: You are watching all bug changes.
[kdevelop] [Bug 373017] Background parser memory on large project
https://bugs.kde.org/show_bug.cgi?id=373017 gnu...@gmail.com changed: What|Removed |Added CC||gnu...@gmail.com -- You are receiving this mail because: You are watching all bug changes.
