[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #55 from Tore Anderson --- > In my experience Akonadi based KMail with IMAP never really worked Assuming KMail wasn't always «Akonadi based», I would agree with this. I used to be a happy KMail user years ago, then I upgraded my distro and found it to have become unusable. This is not an exaggeration, it was literally unusable. I had to immediately find another MUA in order to continue to use my e-mail accounts. I've occasionally tried to go back to KMail since then, but there has been no improvement. So I've been using other clients like Evolution, Thunderbird, Claws, etc. - they all work fine. I don't remember the exact version when it became unusable myself, since it was so long ago. The OP puts it at KDE 4.14, though, which I have no reason to doubt is correct. Thus, if KMail became «Akonadi based» in KDE 4.14 but was «SomethingElse based» in KDE 4.13, then the Akonadi stuff does indeed appear to be the problem here. Tore -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 Tore Anderson changed: What|Removed |Added Resolution|WAITINGFORINFO |--- Status|NEEDSINFO |CONFIRMED --- Comment #52 from Tore Anderson --- OK, I have now tested KMail 5.14.1/Akonadi 20.04 in a freshly installed VM running Fedora 33 Rawhide. Then I compared it to the behaviour of Thunderbird. tl;dr: * KMail/Akonadi spends ~63 minutes on opening a large IMAP folder (133k messages) that Thunderbird spends ~8 minutes on. 688% increase with KMail/Akonadi. * KMail/Akonadi generates an overall higher load average than Thunderbird does (even without taking into account that it does so for a much longer period of time) * KMail/Akonadi ends up requiring 1703 MiB of disk space, Thunderbird only 124 MiB. 1273% increase with KMail/Akonadi. So by every single observable metric, KMail/Akonadi is *extremely* inefficient and slow. In fact it manages to use more disk space than Thunderbird does in the end, even *before* any IMAP account has been created (cf. comment #49). That is just ridiculous. Here is the exact test procedure I used. I'm using the IETF public IMAP server, so anyone can try to do the same. 1. Launched KMail, dismissed the initial account setup wizard. 2. Settings -> KMail settings -> Receiving tab -> Add...custom account -> IMAP 3. General tab: * Account name: IETF * IMAP Server: imap.ietf.org * Username: anonymous * Password: anonym...@ietf.org * Uncheck download all messages for offline use * Uncheck regular checking of e-mail Advanced tab: * Encryption: none 4. Pressed OK, waited for folder list to be downloaded 5. Found the folder named just "ietf" in the list, clicked on it as I started a stopwatch (@11:40). This folder has about 133.000 messages in it. 6. Waited until folder sync and threading had completed After ~15 minutes: folder sync 22% completed, 15-min load average 0.52 After ~30 minutes: folder sync 47% completed, 15-min load average 0.88 After ~45 minutes: folder sync 73% completed, 15-min load average 1.39 After ~60 minutes: folder sync 100% completed, 15-min load average 1.63 At this point, KMail is still busy doing stuff. Status bar says «grouped X of Y threads», «processing X of Y messages», etc. After ~63 minutes: KMail finally idles. 15-min load average 1.61 7. Closed KMail 8. du -ms ~/.local/share/akonadi -> 1703 MiB I then compared with Thunderbird's behaviour: 9. Launched Thunderbird (version 68.8.0) 10. In inital account wizard: * E-mail address: anonym...@ietf.org * Password: anonymous Click "Manual settings": * Incoming server name: imap.ietf.org * Incoming SSL: none * Incoming authentication: regular password * Configured a random outgoing SMTP account (since it's required) * Clicked "Test Again" * Clicked "Advanced settings" * Server settings page: Unchecked "Look for new messages on startup" * Server settings page: Unchecked "Look for new messages every 10 minutes" * Synchronisation and storage page: Unchecked "Keep messages in all folder for this account on this computer" Press OK 11. In advanced config editor: set mail.server.default.using_subscription to false 12. Restart Thunderbird (apparently necessary for mail.server.default.using_subscription to take effect), let it load the list of shared folders 13. Clicked the "ietf" folder and started stopwatch Waiting until folder sync was done: After ~5m: 98k of 133k messages processed (~74%), 5-min load average 0.68 After ~8m: completed. 5-min load average 0.62 14: du -ms ~/.thunderbird -> 124 MiB Tore -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #49 from Tore Anderson --- (In reply to Martin Steigerwald from comment #47) > Tore, Gunther or someone else who commented here: Could you please check > whether you still have those performance issues you reported there with > KDEPIM/Akonadi 20.04? I can try. I obviously haven't been using KMail in the last few years, but now I installed a Fedora 32 KDE VM to perform, upgraded it to Rawhide to get KDEPIM/Akonadi 20.04, rebooted it and deleted all files (including hidden) in my home directory from a console login (just to make sure what I am about to report was not a holdover from the previous Akonadi version in Fedora 32) and logged back in. The result - just from logging in - is that ~/.local/share/akonadi/db_data contains 158 MiB of stuff. To be clear: I have *not* started KMail, *not* edited any Akonadi settings, or anything like that. The *only* thing I have done is to log in from a completely empty user account, start Konsole, run 'du' to check disk usage. Yet Akonadi has managed to store 158 MiB of what has to be completely pointless stuff. That is beyond comprehension, and certainly does not give me any hope that this bug is anywhere close to being fixed, quite the opposite. Akonadi's disk and resource usage seems to still be beyond insane. That said, I will test towards an IMAP account later and update this bug with my findings. > If so, it might be good to report it in a new bug since – with my > contribution unfortunately, this bug report has so many comments that it may > be difficult to make sense out of it for a developer. What's the point? The reason why this bug has many comments is that it has languished here or almost six years with no developer taking any interest in actually fixing it. Why assume a new bug report will be treated any differently KMail is unusable with IMAP accounts with more than a few thousand messages, and it's been like this for years. I can only surmise that this is a use case the developers care about (which is fair enough, don't get me wrong, I get what I pay for here). Tore -- You are receiving this mail because: You are watching all bug changes.
[kdelibs] [Bug 277134] Shortcut grabbing doesn't work correctly when «Make Caps Lock an additional Ctrl» and «Meta on Left Ctrl» is set
https://bugs.kde.org/show_bug.cgi?id=277134 --- Comment #4 from Tore Anderson --- Yes, this is still not working correctly, but it behaves a bit different than in 2011. With the two options enabled (which are now named «Caps Lock is also a Ctrl» and «Left Ctrl as Meta»), this is the following results when attempting to grab/learn a new shortcut: - Physically pressing Caps Lock + Space: --- expected result: «Ctrl+Space» --- actual: «CapsLock, Ctrl+Space,Ctrl + ...» (the «...» indicates that it is still expecting more keypresses to complete the shortcut sequence) - Physically pressing Left Ctrl + Space: --- expected result: «Meta+Space» --- actual result: «Meta+Alt+Space,Alt+ ...» (again «...» - it is still listening for more keystrokes» Physicall pressing Left Windows + Space: --- expected result: «Meta+Space» --- actual result: «Meta+Space, Meta+ ..» (again «...»). I'm using Fedora 28 (kdelibs-4.14.38-6.fc28.x86_64). Tore -- You are receiving this mail because: You are watching all bug changes.
[plasma-nm] [Bug 350521] [RFE] [OpenVPN] kdeplasma-applets-plasma-nm does not support OTP Tokens for OpenVPN connections
https://bugs.kde.org/show_bug.cgi?id=350521 Tore Anderson changed: What|Removed |Added CC||t...@fud.no -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 387936] NFSv4 mount is incorrectly marked as read-only
https://bugs.kde.org/show_bug.cgi?id=387936 --- Comment #7 from Tore Anderson --- Noticed something interesting now: $ test -w /nfshome && echo rw || echo ro ro $ test -w /nfshome/bin && echo rw || echo ro rw This seems to match Dolphin's behaviour. I can use Dolphin to manage files inside subdirectories just fine, just not at the top-level mount point. But even though "test -w" suggests otherwise, the top-level mount point is writeable just fine. Curious. -- You are receiving this mail because: You are watching all bug changes.
[frameworks-kio] [Bug 387936] NFSv4 mount is incorrectly marked as read-only
https://bugs.kde.org/show_bug.cgi?id=387936 --- Comment #6 from Tore Anderson --- (In reply to Nate Graham from comment #5) > Thanks for the new info. Can you provide details about your method of > mounting it so we can try to figure out what combination of options triggers > this bug? Standard mounting from /etc/fstab: nas2-osl1:/volume1/homes/@LH-REDPILL-LINPRO.COM/0/tore-7097 /nfshome nfs defaults 0 0 The resulting /proc/mounts entry: nas2-osl1:/volume1/homes/@LH-REDPILL-LINPRO.COM/0/tore-7097 /nfshome nfs4 rw,relatime,vers=4.0,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp6,timeo=600,retrans=2,sec=sys,clientaddr=2a02:c0:(snip),local_lock=none,addr=2a02:c0:(snip) 0 0 The mount is writable just fine from other applications, such as the shell: $ stat /nfshome File: /nfshome Size: 6654Blocks: 0 IO Block: 32768 directory Device: 38h/56d Inode: 4727201 Links: 1 Access: (0755/drwxr-xr-x) Uid: ( 7097/tore) Gid: ( 7097/tore) Context: system_u:object_r:nfs_t:s0 Access: 2018-07-04 14:16:34.555886284 +0200 Modify: 2018-07-04 14:16:33.518885060 +0200 Change: 2018-07-04 14:16:33.518885060 +0200 Birth: - $ touch /nfshome/test $ rm /nfshome/test $ I believe the NFS server is a Synology RackStation RS3617xs. Tore -- You are receiving this mail because: You are watching all bug changes.
[dolphin] [Bug 387936] Can't write on nfs mounted dir from Dolphin but can from terminal
https://bugs.kde.org/show_bug.cgi?id=387936 Tore Anderson changed: What|Removed |Added CC||t...@fud.no Ever confirmed|0 |1 Status|UNCONFIRMED |CONFIRMED --- Comment #4 from Tore Anderson --- Confirmed. Same issue on Fedora 28 with dolphin-17.12.2-2.fc28.x86_64. It seems to believe a NFSv4 mount is read-only. (It isn't.) -- You are receiving this mail because: You are watching all bug changes.
[konsole] [Bug 361791] Konsole ignores browser settings and opens some URLs in their associated applications
https://bugs.kde.org/show_bug.cgi?id=361791 Tore Anderson <t...@fud.no> changed: What|Removed |Added CC||t...@fud.no --- Comment #2 from Tore Anderson <t...@fud.no> --- I had the same problem and searched my way to this bug. Changing it in "kcmshell5 componentchooser" did however fix it for me. I'm using Konsole 17.04.1 on Fedora 26, so perhaps the issue is already resolved? -- You are receiving this mail because: You are watching all bug changes.
[KAccounts] [Bug 363260] Can't create twitter account
https://bugs.kde.org/show_bug.cgi?id=363260 Tore Anderson <t...@fud.no> changed: What|Removed |Added CC||t...@fud.no -- You are receiving this mail because: You are watching all bug changes.
[kmail2] [Bug 371228] enable remote search on IMAP servers
https://bugs.kde.org/show_bug.cgi?id=371228 Tore Anderson <t...@fud.no> changed: What|Removed |Added CC||t...@fud.no -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 367892] During folder synchronisation Akonadi blocks out other operations like deleting or viewing mails
https://bugs.kde.org/show_bug.cgi?id=367892 Tore Anderson <t...@fud.no> changed: What|Removed |Added CC||t...@fud.no -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #46 from Tore Anderson <t...@fud.no> --- By the way, if you need an IMAP account to test with, try the IETF mailing list archives, which is available through anonymous IMAP (cf. https://trac.tools.ietf.org/group/tools/trac/wiki/Imap): Server: imap.ietf.org port 143 Username: anonymous Password: anonym...@ietf.org Encryption: none Authentication: Clear text (note: auto-detection does NOT work, as the CAPABILITY output of the server is incorrect) There is a ton of folders on this server and probably several million messages. The fact of the matter is that this bug is quite simply making impossible to use KMail to access this server. "Use Less Bandwidth" or "Download all messages for offline use" doesn't help. Try it and see. -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #45 from Tore Anderson <t...@fud.no> --- Created attachment 106920 --> https://bugs.kde.org/attachment.cgi?id=106920=edit Temperature graph - KMail tests at 21:30-22:00 and 12:35-13:05 -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #43 from Tore Anderson <t...@fud.no> --- Created attachment 106918 --> https://bugs.kde.org/attachment.cgi?id=106918=edit I/O throughput graph - KMail tests at 21:30-22:00 and 12:35-13:05 -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #42 from Tore Anderson <t...@fud.no> --- Created attachment 106917 --> https://bugs.kde.org/attachment.cgi?id=106917=edit IOOPS graph - KMail tests at 21:30-22:00 and 12:35-13:05 -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #44 from Tore Anderson <t...@fud.no> --- Created attachment 106919 --> https://bugs.kde.org/attachment.cgi?id=106919=edit Load graph - KMail tests at 21:30-22:00 and 12:35-13:05 -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #41 from Tore Anderson <t...@fud.no> --- Created attachment 106916 --> https://bugs.kde.org/attachment.cgi?id=106916=edit CPU graph - KMail tests at 21:30-22:00 and 12:35-13:05 -- You are receiving this mail because: You are watching all bug changes.
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 16.08 (actually in any release, starting with KDE 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #40 from Tore Anderson <t...@fud.no> --- Following the release of Fedora 26, I decided to test KMail again and compare it with Mozilla Thunderbird, both using a completely empty UNIX account. tl;dr: Compared to Thunderbird, KMail/Akonadi ends up using a absolutely ridiculous amount of time, CPU, disk space, and network traffic to connect to an IMAP account. Main observations: - KMail needed ~30 minutes to perform the initial sync of all folders, while Thunderbird used <5 minutes*: - The system was completely swamped by Akonadi and MySQL processes. Thunderbird was very modest. - KMail exchanged 1.19 GB of data with the IMAP server. Thunderbird exchanged 128 MB. - Afterwards, the home directory of the KMail test account contained 3842 MB of data, Thunderbird's contained 128 MB. * Unlike KMail, Thunderbird only synced INBOX automatically and its UI usable within seconds. The <5 minutes included manually opening and syncing every single IMAP folder. More details: The test system has a quad-core Intel i5-7300U and 16 GB of RAM. The IMAP server is running Dovecot 2.2.22 and requires TLS. The IMAP accounts contains approx 224.500 messages (4.3 GB of data). The largest folder contains about 43.400 messages. When starting KMail at about 12:30 I cancelled the initial Account Assistant wizard, and selected both "Work Offline" and "Use Less Bandwidth" from the File menu. I then created the IMAP account, making sure to uncheck "Download all messages for offline use" and "Interval mail checking". At 12:35 I selected "Work Online" from the File menu and clicked the "Check Mail" button. At this point, KMail goes to work. The status indicator is showing a progress bar titled "Syncing folder 'x'" which cycles through every folder on the IMAP server. While this goes on, I can see the following processes consistently occupying the top-5 spots in the output of top(1). They are sorted according to which position they typically was found at (although it does change around a bit from refresh to refresh): 1) mysqld 2) akonadiserver 3) akonadi_indexing_agent 4) akonadi_imap_resource 5) kmail The load average jumps up to around 5, and the laptop's fan revs up to max speed. At 13:02 the last "Syncing folder 'x'" progress indicator vanishes. At this point, IMAP network traffic stops and only the "akonadi_indexing_agent" process continue to hog a CPU core. At around 13:05 this process also calms down, the load eventually drops to below 1, and the laptop fan silences. As mentioned above, at this point there had been 1,19 GB of network traffic between the client and the IMAP server. Almost exclusively in the server->client direction. This was measured with iftop(8). The test user's (previously empty) home directory had grown to contain 3842 MB of data. This was almost all divided into these three directories: /home/kmailtest/.local/share/akonadi/db_data : 791 MB /home/kmailtest/.local/share/akonadi/file_db_data : 1360 MB /home/kmailtest/.local/share/akonadi/search_db : 1674 MB The Thunderbird process was started at 13:30. As with KMail, I cancelled the initial wizard and entered offline mode. Then I created the IMAP account, making sure to deselect "Keep messages for this account on this computer". At 13:35 I entered online mode and clicked "Get Messages". This caused it to download all the headers in the modestly-sized INBOX folder, which was done in maybe five seconds. This is IMHO a vastly superior approach to the KMail one of syncing all the remote folders immediately. In order to get a comparable test, I then proceeded to manually open every single IMAP folder, one after the other. Upon entering each folder Thunderbird would download the headers of all messages contained in it. I had walked through all the folders on the servers before 13:40. After this /home/tbirdtest had grown to contain 128 MB of data and iftop(8) had observed 210MB of network traffic between the client and the IMAP server. I will be attaching a few Munin graphs gathered from the laptop as these tests were run. The Thunderbird test isn't really visible, while the KMail one clearly is. I also ran this KMail test yesterday at around 21:30-22:00, which also clearly stands out. The KMail/Akonadi versions installed were: kf5-libkdepim-akonadi-17.04.1-1.fc26.x86_64 kf5-akonadi-notes-17.04.1-1.fc26.x86_64 kf5-akonadi-server-mysql-17.04.1-3.fc26.x86_64 kf5-akonadi-mime-17.04.1-1.fc26.x86_64 kf5-akonadi-contacts-17.04.1-1.fc26.x86_64 kf5-akonadi-search-17.04.1-1.fc26.x86_64 kmail-account-wizard-17.04.1-1.fc26.x86_64 kf5-kmailtransport-17.04.1-1.fc26.x86_64 akonadi-1.13.0-103.fc26.x86_64 kmail-17.04.1-3.fc26.x86_64 kf5-akonadi-calendar-17.04.1-1.fc26.x86_64 kf5-mailimporter-akonadi-17.04.1-1.fc26.x86_64 kdepimlibs-akonadi-4.14.10-18.fc26.x86_64 akonadi-import-wizard-17.04
[Akonadi] [Bug 338571] Performance Regression: Folder synchronisation in Akonadi 15.12 (actually in any release, starting with 4.14) very slow, compared to kMail from KDE 4.13.x
https://bugs.kde.org/show_bug.cgi?id=338571 --- Comment #15 from Tore Anderson <t...@fud.no> --- Just upgraded to Fedora 24 and tried a fresh setup of KMail with two IMAP accounts. Versions: akonadi-1.13.0-102.fc24.x86_64 kmail-15.12.3-1.fc24.x86_64 No improvement is noticeable. Synchronising folders takes hours, and my laptop is completely swamped - everything gets extremely slow. The mysqld and akonadi_indexing_agent processes appears to be the biggest resource hogs. Neither of the accounts have «Download all messages for offline use» selected. In spite of this, after the initial sync has completed, ~/.local/share/akonadi/ contains almost 7 GiB of data. The largest contributors to that are: file_db_data/* ~3 GiB search_db/email/termlist.DB ~1 GiB search_db/email/postlist.DB ~1 GiB db_data/akonadi/parttable.ibd ~1 GiB KMail is essentially unusable for me due to this issue. -- You are receiving this mail because: You are watching all bug changes.
