[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #104 from Ash --- I agree with Knut Hildebrandt The original bug which was reported, the one which I am experiencing, and commented on (#63), is not fixed. A _different_ memory leak was found, and fixed, (changing the title of this bug to reflect that was a nice touch) but the bug matching the description of the original report is not fixed. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #103 from Knut Hildebrandt --- Back with a short summery of my observations. 1. with the moment of its start a dolphin instance begins to constantly allocate memory 2. the amount of memory allocated as well as the CPU time used correlates with the number of tabs and views open and grows over time 3. the memory allocation does not depend on user interaction. Opening files several GB big does not cause any jump in memory usage. On the other hand the memory usage increases constantly even without any user interaction at all. 4. hibernating the machine brings the used memory down to about 20 to 30 MB for each instance, but within minutes, sometimes seconds, it is back to the level before hibernation and than further rising 5. over time and with many dolphins, tabs and views open not only the dolphin instances get very slow but the machine too, due to constant swapping For me there arise a few questions: 1. why does dolphin constantly allocate memory even though there is no user interaction? 2. what does dolphin need the saved data for, since it obviously works well with memory usage far below 100 MB even if there are many tabs open? 3. Why once allocated memory never gets freed? All the data I mention here I gathered in KSysGuard. I've no idea what tools I have to use to collect more useful data. One last thing. The original title of the bug report was something like "Dolphin uses huge amounts of memory" and has been changed since a potential memory leak has been identified. Since this bug was fixed the bug report has been closed even though the problem I reported three years ago still exists. Thus I'd propose to reopen the bug and change it's title back to the original one. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #102 from Knut Hildebrandt --- Just noticed that there is a correlation between the number of tabs and views open in a dolphin instance and the CPU time and memory it is accumulating. After only about six hours of total run time the situation looks like this: 1. 9 tabs x 2 views -> 326 MB 2. 8 tabs x 2 views -> 215 MB 3. 6 tabs x 2 views -> 208 MB 4. 4 tabs x 2 views -> 160 MB 5. 2 tabs x 2 views -> 101 MB 6. 2 tabs x 2 views -> 99 MB 7. 1 tabs x 2 views -> 66 MB 8. 1 tabs x 1 views -> 46 MB Most of the time I was watching movies, answering emails or surfing the internet and not using any of the dolphins. Some I did not touch at all but even though they are not used they accumulate memory over time. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #101 from Knut Hildebrandt --- (In reply to Harald Sitter from comment #100) Sorry for the late reply, I was out of town for a while. > Is that actually the same defect though? I do not know, but it at least the same behavior I described in my first bug report. All dolphin instances use up more and more memory over time, particularly if the system has been suspended or hibernated various times. > When you start from a new dolphin and click a super large file, like say an > ISO, does the consumption jump up and stay high even after the file is fully > opened? Just restarted the machine after having to shut it down since dolphin used up to 4 GB of memory. After the restart I opened an extra instance an started a 3.2 GB movie. When I started the movie the respective dolphin instance used at bit more than 20 MB and slowly gained 10 MB until the movie finished. Meanwhile some of the other dolphin instances had gained more than 100 MB each. The ones showing the highest memory usage at the same time show the highest CPU time - up to 3:50 over two hours - too, whereas the one I used to start the movie only has 0:21. Maybe I should mention that all instances showing high memory usage and CPU time have several tabs and split view, only the one opened to start the movie has a single tab and simple view. Well, I got the impression that my description of the behavior does not help to narrow down the problem. What data could be more helpful and how can I retrieve it? Operating System: Manjaro Linux KDE Plasma Version: 5.22.4 KDE Frameworks Version: 5.84.0 Qt Version: 5.15.2 Kernel Version: 5.10.56-1-MANJARO (64-bit) Graphics Platform: X11 -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Paul changed: What|Removed |Added CC|holle...@cs.dal.ca | -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #100 from Harald Sitter --- Is that actually the same defect though? When you start from a new dolphin and click a super large file, like say an ISO, does the consumption jump up and stay high even after the file is fully opened? -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #99 from Knut Hildebrandt --- Updated KDE Frameworks and nothing seems to have changed. I even got the impression it got worse. Hibernated the machine last night, what brought all dolphin instances back to below 100 MB each, but now about three hours later three of them are in between 0.5 and 1 GB, four between 100 and 500 MB and only two below 100 MB. And Ooe of the ones above 500 MB sometimes uses between 10 and 15% CPU. BTW, the more CPU time an instance has accumulated the higher the memory it occupies. I still double click all files - including video and audio - to open them. Only exceptions: audio playlists which I open from amarok and if I do not want to open a file in it's standard application. One change I noticed, sometimes memory seems to get released. But if I'm not mistaken, this effect I already noticed earlier, for instance with Frameworks 5.82.0. Operating System: Manjaro Linux KDE Gear Version: 21.04.3 KDE Plasma Version: 5.22.3 KDE Frameworks Version: 5.84.0 Qt Version: 5.15.2 Kernel Version: 5.10.49-1-MANJARO (64-bit) Graphics Platform: X11 -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #98 from Knut Hildebrandt --- (In reply to Harald Sitter from comment #97) > The bug was fixed for 5.83, you are still on 5.82. Sorry, then I'll check again when the respective update is offered. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Harald Sitter changed: What|Removed |Added Status|RESOLVED|CLOSED -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Harald Sitter changed: What|Removed |Added Status|NEEDSINFO |RESOLVED -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #97 from Harald Sitter --- The bug was fixed for 5.83, you are still on 5.82. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #96 from Knut Hildebrandt --- The problem still prevails and somehow I got the impression that it is getting worse again. Last night I had to shut down my laptop since it was getting very slow due to endless swapping. Dolphin was consuming about 3.5 GB at that moment. Operating System: Manjaro Linux KDE Plasma Version: 5.21.5 KDE Frameworks Version: 5.82.0 Qt Version: 5.15.2 Kernel Version: 5.10.42-1-MANJARO Processors: 4 × Intel® Core™ i5-3317U CPU @ 1.70GHz Memory: 9.6 GiB of RAM -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #95 from Bug Janitor Service --- Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone! -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Andrew Rembrandt changed: What|Removed |Added CC|pub...@rembrandt.dev| -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Alexander Meshcheryakov changed: What|Removed |Added Status|CONFIRMED |NEEDSINFO Resolution|--- |FIXED --- Comment #94 from Alexander Meshcheryakov --- I still observe Dolphin memory leak with plasma-framework 5.83.0: in just two days memory usage of running dolphin instance increased from ~210 MB to ~740 MB RSS There must be something else. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Ahmad Samir changed: What|Removed |Added Version|21.04.2 |5.82.0 Assignee|dolphin-bugs-n...@kde.org |kio-bugs-n...@kde.org Product|dolphin |frameworks-kio Component|general |general -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Ahmad Samir changed: What|Removed |Added CC||unkn...@skynet.be --- Comment #92 from Ahmad Samir --- *** Bug 427168 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #91 from Paul --- *** Bug 436505 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Ahmad Samir changed: What|Removed |Added CC||rikmi...@kde.org --- Comment #90 from Ahmad Samir --- That reminds me, I should let distro maintainer know about this patch to backport it to 5.82. (About 5.80, the KUbuntu maintainers can rebase the patch manually?) -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #89 from Fonic --- Great that this got sorted out, many thanks! It seems, however, that the patch is incompatible with KIO's 5.80 codebase and KIO can't be updated individually without updating all KDE packages due to dependencies, or am I missing something? -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 J.D changed: What|Removed |Added CC|o...@gmail.com | -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #88 from David Faure --- Git commit c19876052ecec18a87a82f5950e8909e22e895ba by David Faure, on behalf of Ahmad Samir. Committed on 13/05/2021 at 23:15. Pushed by dfaure into branch 'master'. MimeTypeFinderJob: the StatJob details should include the mimetype Apparently we forgot to specify that we want the UDS_MIME_TYPE field in the statFile() method (both when it lived in OpenUrlJob and when it was moved to MimeTypeFinderJob). And now there is a dedicated StatJob flag, StatMimeType, that we can use. Not passing KIO::StatMimeType when creating the StatJob meant the code always used a get job to determine the mime type, which mean that e.g. opening an ISO file from Dolphin, which supposedly just needs to launch Ark, had the whole file read into memory, which means that opening a couple of ISO's and you're out of memory... Thanks to sitter for doing a big chunk of the investigative work in the bug report. M +5-1src/core/mimetypefinderjob.cpp https://invent.kde.org/frameworks/kio/commit/c19876052ecec18a87a82f5950e8909e22e895ba -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #87 from Ahmad Samir --- This bug report is actually two bugs, one of them that the mimetype finding code was falling back to KIO::get() which, as sitter's investigation shows, meant the whole file was read into memory which meant spikes in memory consumption used by Dolphin; hopefully this is fixed by the MR I linked above. (Note that the issue with the code in MimeTypeFinderJob was probably the same in OpenUrlJob, because that code lived in OpenUrlJob before getting split to a separate MimeTypeFinderJob). Now the other issue, the original report seems to hint that some of the files in question are on remote filesystems? -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 --- Comment #86 from Ahmad Samir --- https://invent.kde.org/frameworks/kio/-/merge_requests/444 -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Michael D changed: What|Removed |Added CC||nortex...@gmail.com --- Comment #85 from Michael D --- Just had Dolphin using 1GB of memory with a few tabs open and 1 day uptime. Dolphin version 21.04.0. I have an information panel open but no terminal or anything. I don't recall doing anything unusual, but I'll keep an eye on it. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 398908] MimeTypeFinderJob can use huge amounts of memory reading unnecessary data
https://bugs.kde.org/show_bug.cgi?id=398908 Nate Graham changed: What|Removed |Added Severity|normal |major Summary|Dolphin uses up huge|MimeTypeFinderJob can use |amounts of memory |huge amounts of memory ||reading unnecessary data Priority|NOR |HI CC||a.samir...@gmail.com, ||ahiems...@heimr.nl -- You are receiving this mail because: You are watching all bug changes.
