[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #48 from Attila --- (In reply to Wolfgang Bauer from comment #42) > (In reply to Attila from comment #41) > > Firefox can display this e-mail within 2 seconds and I can scroll very > > smooth through the content by dragging the scrollbar. > Have you tried in Chromium though? (which is what QtWebEngine is based upon) I have tried Google Chrome today. It is comparable to Firefox. Chrome displays the same e-mail in 2 seconds. There is just one difference. Chrome takes 15 seconds more to scroll down to the bottom of the e-mail. This would be acceptable as well. Chrome eats 347.676 K. This is not bad, is it? -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #47 from Wolfgang Bauer --- (In reply to RJVB from comment #45) > (In reply to Wolfgang Bauer from comment #44) > > > I do use QtWebKit with konqueror, and opening large OBS logs (which are just > > text files) do take a while. > > TBH, Konqueror is such a multipurpose tool that I have no idea what > backend/engine/kpart it uses for rendering pure text. I'm talking about opening a text file in the HTML renderer, whatever is the default. Konqueror can of course use other kparts to display files (katepart e.g.), but that's not the topic here. (In reply to Greg Rivers from comment #46) > From a user perspective, it's more than reasonable to expect KMail to > display large plain text messages as well as other common MUAs do. Rather > than using QtWebEngine to display plain text, shouldn't an efficient method > such as the one employed by "View Source" be used for all text/plain MIME > parts? I don't think so, no. Without a large rewrite anyway, as mentioned. KMail does add HTML stuff to an EMail even if it's just plain text, such as the header, links to attachments, and so on. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #46 from Greg Rivers --- >From a user perspective, it's more than reasonable to expect KMail to display large plain text messages as well as other common MUAs do. Rather than using QtWebEngine to display plain text, shouldn't an efficient method such as the one employed by "View Source" be used for all text/plain MIME parts? -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #45 from RJVB --- (In reply to Wolfgang Bauer from comment #44) > I do use QtWebKit with konqueror, and opening large OBS logs (which are just > text files) do take a while. TBH, Konqueror is such a multipurpose tool that I have no idea what backend/engine/kpart it uses for rendering pure text. Actually, when I open /var/log/syslog I get a view that suggest very strongly that the Kate text editor kpart is used. That is, if I open the file from within Konqueror. If I try to start konqueror with a file it will start but open the file in the system handler (Kate for text files), very funny. > That's what I meant with a "large rewrite" amongst other things. > Please note that I'm not a kmail developer though. Neither am I, and you're right in assuming it could be more work than you'd expect. > I'm not sure though, nor if it frees the resources when you switch to a > different mail (I think it does). In my experience that doesn't always mean the resources are available immediately, for RAM at least. But I digress... -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #44 from Wolfgang Bauer --- (In reply to RJVB from comment #43) > > TBH, I think even QtWebKit would struggle with these large mails > > That would be simple to test in konqueror with the webkit backend, or > Otter-Browser, and the rebooted QtWebKit. Or in epiphany. > 1 caveat emptor: we don't actually know what clever tricks the web email > interface pulls. I do use QtWebKit with konqueror, and opening large OBS logs (which are just text files) do take a while. > > Anyway, this is kind of a different problem than comment#35 (or comment#0). > > I don't see how it could be fixed from the kmail side, except for not trying > > to display mails larger than a certain size at all. (or maybe just display > > it as plain text instead then, but that probably means a large rewrite of > > the code, and I'm not sure it's possible at all) > > - Don't dump the entire message into the rendering widget but use a paged > approach > - wrapper code can be written that connects to different backends. If you > don't want to support QtWebkit, fine, but there's also QTextBrowser which > should be sufficient even for simple HTML emails (it's what Qt's assistant > uses by default since whatever Qt version that obsoleted QtWebkit). I've > done a compile-time version of such wrapper code for KDevelop's help browser > (which supports both QWE and QWK). I think it should be possible at least to > chose between backends as a startup option. That's what I meant with a "large rewrite" amongst other things. Please note that I'm not a kmail developer though. > Out of curiosity: does that humongous QWE process exit when you close the > message view, or do you have to quit KMail to recuperate all that RAM? I think that the QWE process is reused. I'm not sure though, nor if it frees the resources when you switch to a different mail (I think it does). -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #43 from RJVB --- (In reply to Wolfgang Bauer from comment #42) > Have you tried in Chromium though? (which is what QtWebEngine is based upon) Current versions of Firefox tend to consume more memory than Chromium - in fact,I understand that the Firefox Quantum engine uses bits and pieces from all 3 open source web engines and picks the fastest one. > TBH, I think even QtWebKit would struggle with these large mails That would be simple to test in konqueror with the webkit backend, or Otter-Browser, and the rebooted QtWebKit. Or in epiphany. 1 caveat emptor: we don't actually know what clever tricks the web email interface pulls. > Anyway, this is kind of a different problem than comment#35 (or comment#0). > I don't see how it could be fixed from the kmail side, except for not trying > to display mails larger than a certain size at all. (or maybe just display > it as plain text instead then, but that probably means a large rewrite of > the code, and I'm not sure it's possible at all) - Don't dump the entire message into the rendering widget but use a paged approach - wrapper code can be written that connects to different backends. If you don't want to support QtWebkit, fine, but there's also QTextBrowser which should be sufficient even for simple HTML emails (it's what Qt's assistant uses by default since whatever Qt version that obsoleted QtWebkit). I've done a compile-time version of such wrapper code for KDevelop's help browser (which supports both QWE and QWK). I think it should be possible at least to chose between backends as a startup option. Out of curiosity: does that humongous QWE process exit when you close the message view, or do you have to quit KMail to recuperate all that RAM? -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #42 from Wolfgang Bauer --- (In reply to Attila from comment #41) > Firefox can display this e-mail within 2 seconds and I can scroll very > smooth through the content by dragging the scrollbar. Have you tried in Chromium though? (which is what QtWebEngine is based upon) TBH, I think even QtWebKit would struggle with these large mails, according to my experiences with OBS build logs... (KHTML served better in this regard, but that's completely dead now) Anyway, this is kind of a different problem than comment#35 (or comment#0). I don't see how it could be fixed from the kmail side, except for not trying to display mails larger than a certain size at all. (or maybe just display it as plain text instead then, but that probably means a large rewrite of the code, and I'm not sure it's possible at all) -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #41 from Attila --- (In reply to RJVB from comment #39) > A pure text email of 35Mb is massive too, but eating around 4Gb of RAM for > that isn't just massive. That's what you meant, right, not 4Mb for > displaying just the part that fits in your window (which would seem > reasonable)? 4Gb, that's about 120 times the size of the email. Yes I mean 4.317.684 K (QtWebEngineProcess). > I knew KMail would become a powerkids' tool with the design decision to use > a full-fledged-browser-in-a-widget (QtWebEngine) for rendering everything > including pure text emails but I wasn't expecting this. Then again, I don't > usually get this kind of long letters by email so I have no idea how any > other MUA would handle them. I get this kind of long emails every day. This is the output from backup scripts executed by Cron. > If the account you got this email on has a web/browser interface it would be > interesting to estimate the resources an actual browser (GChrome or > Chromium) would use to display it, for comparison. Unfortunately this account doesn't have a Web-Interface, but I did a test on another account with Web-Interface. The plain text has a size of 20 MiB. In Comparison: QtWebEngineProcess (KMail) is eating 1.609.232 K. Firefox (Web-Interface) is eating 156.204 K. Firefox can display this e-mail within 2 seconds and I can scroll very smooth through the content by dragging the scrollbar. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #40 from Rodney Baker --- It is working for me (plain text emails around 41MB - log files from an automated backup script), but it is very slow to load the email. The CPU fan does ramp up slightly when doing so, but at least it loads, and once it loads, scrolling is usable. By comparison, "View Message Source" uses almost no CPU or RAM, and displays the message pretty much instantly. Thunderbird displays the message absolutely fine, too, with no noticeable hit on system resources or CPU load. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #39 from RJVB --- A pure text email of 35Mb is massive too, but eating around 4Gb of RAM for that isn't just massive. That's what you meant, right, not 4Mb for displaying just the part that fits in your window (which would seem reasonable)? 4Gb, that's about 120 times the size of the email. I knew KMail would become a powerkids' tool with the design decision to use a full-fledged-browser-in-a-widget (QtWebEngine) for rendering everything including pure text emails but I wasn't expecting this. Then again, I don't usually get this kind of long letters by email so I have no idea how any other MUA would handle them. If the account you got this email on has a web/browser interface it would be interesting to estimate the resources an actual browser (GChrome or Chromium) would use to display it, for comparison. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #38 from Attila --- Additional information: I have observed the system activity today by opening an ASCII-email by the size of 35 MB. The application "System Activity" shows me a CPU usage of 13% (QtWebEngineProcess) and RAM consumption of 4.317.684 K. I think this is massive, isn't it? Scrolling through the email is almost impossible. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #37 from Attila --- The bug has not been fixed as far as I can see. KMail 5.15.3 is available for Fedora 33. It doesn't work when I try to open ASCII-emails larger then 2 MB. I got ASCII emails from 30 MB up to 55 MB. The system starts to swap by opening such an email (8 GB RAM is installed and a CPU with 4 cores). All I can do is to delete this email as soon as KMail responds not to overload the system. This isn't a good user experience. Is there anything I can do? Let me know if you still need more informtaion. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Wolfgang Bauer changed: What|Removed |Added CC||wba...@tmo.at --- Comment #36 from Wolfgang Bauer --- (In reply to Axel Braun from comment #35) > Looks like the issue is back: The issue us not really "back". Rather, it was actually not *fully* fixed, for every possible email. As proposed in comment#29, messagelib does use a temporary file now for larger emails, because setting the content with QWebEngineView::setContent() or QWebEngineView::setHTML() has a 2MB limit (see comment#5). But the problem is that AFAICT it is impossible to know in advance how big the message exactly is *after* QWebEngine's internal processing. To quote from the docs: > Note: Content larger than 2 MB cannot be displayed, because setHtml() > converts > the provided HTML to percent-encoding and places data: in front of it to > create the URL that it navigates to. Thereby, the provided code becomes a URL > that exceeds the 2 MB limit set by Chromium. So apparently your mail is an "edge case" that is smaller than 200 bytes (the current threshold in messagelib), but that still becomes too large for QtWebEngine/Chromium after the percent-encoding done internally. Larger emails, like the one mentioned in the title, do (still) display fine now though, because they are correctly detected as being too large. It should be easy to fix for that particular mail by lowering the size limit, but the lower limit may still be too high for certain other emails. OTOH, setting it too low is not good either, as that way of displaying mails seems to cause problems with encoding (see comment#28). But I just noticed that the docs also say: > If the content is too large, the loadFinished() signal is triggered with > success=false. Maybe this could be exploited as trigger for using a temporary file instead of a size check? Just an idea, I don't know if it's really possible to do that (it would probably be quite tricky to implement at least, I fear). -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Axel Braun changed: What|Removed |Added Resolution|FIXED |--- Status|RESOLVED|REOPENED --- Comment #35 from Axel Braun --- Looks like the issue is back: Operating System: openSUSE Tumbleweed 20210108 KDE Plasma Version: 5.20.5 KDE Frameworks Version: 5.77.0 Qt Version: 5.15.2 -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #34 from Axel Braun --- Created attachment 134744 --> https://bugs.kde.org/attachment.cgi?id=134744=edit message not displayed -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #33 from Rodney Baker --- Sorry, it's taken a while for me to be able to verify on my system due to other issues, but I can confirm it is now working as expected. Thanks for your work - much appreciated! -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Wolfgang Bauer changed: What|Removed |Added Status|ASSIGNED|RESOLVED Latest Commit|https://commits.kde.org/mes |https://invent.kde.org/pim/ |sagelib/991eb9c20286bdce245 |messagelib/commit/706aad16c |8d7dbc17765dd7b0d7b38 |5927cee5ff6a6bf6d608f8842c4 ||e9db Resolution|--- |FIXED Version Fixed In|5.14.0 |5.15.0 --- Comment #32 from Wolfgang Bauer --- Git commit 706aad16c5927cee5ff6a6bf6d608f8842c4e9db by Wolfgang Bauer. Committed on 30/07/2020 at 08:31. Pushed by mlaurent into branch 'release/20.08'. Fix size threshold for "big email" codepath The maximum size supported by QWebEngineView::setContent() is 2 MB, not 2 GB. FIXED-IN: 5.15.0 M +1-1messageviewer/src/htmlwriter/webengineparthtmlwriter.cpp https://invent.kde.org/pim/messagelib/commit/706aad16c5927cee5ff6a6bf6d608f8842c4e9db -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Bug Janitor Service changed: What|Removed |Added Status|REOPENED|ASSIGNED --- Comment #31 from Bug Janitor Service --- A possibly relevant merge request was started @ https://invent.kde.org/pim/messagelib/-/merge_requests/3 -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #30 from Laurent Montel --- it can be a good idea. I will look at if it's possible. it can break encoding for some big email but it's less critical as not be able to load it I think -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #29 from Rodney Baker --- So, is it possible to make it conditional i.e. below a certain size threshold, use the preferred method, but if a message is above that threshold, use the alternative (working but reverted) method from this patch? -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Laurent Montel changed: What|Removed |Added CC||mon...@kde.org --- Comment #28 from Laurent Montel --- Yep confirmed. It created too many encoding problem so I decided to reverse this patch. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 --- Comment #27 from Greg Rivers --- (In reply to Rodney Baker from comment #26) Confirmed. I see this regression as well. -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Rodney Baker changed: What|Removed |Added Version|5.6.0 |5.14.1 -- You are receiving this mail because: You are the assignee for the bug.
[kmail2] [Bug 387061] REGRESSION: Large messages don't display in the viewer pane (eg. New Tumbleweed snapshot 20171117 released!)
https://bugs.kde.org/show_bug.cgi?id=387061 Rodney Baker changed: What|Removed |Added Summary|Large messages don't|REGRESSION: Large messages |display in the viewer pane |don't display in the viewer |(eg. New Tumbleweed |pane (eg. New Tumbleweed |snapshot 20171117 |snapshot 20171117 |released!) |released!) -- You are receiving this mail because: You are the assignee for the bug.
