Bug#727800: kmail: KMail goes bonkers following Qt update
So, I've tried again, this time restarting Akonadi when the migration script said it was done. I tried two variations: 1) akonadictl restart before pressing ok on the window showing the kmail resource migration. Surprisingly, this did not cause KMail to shut down. It also didn't really work -- I ended up with two KMail Folders collections, repeated item belongs to two top-level collections error messages, and other misbehaviors. The two KMail Folders collections contained, each, all my folders, but it seemed the folders didn't quite work in any of them; I think inbox looked like it had the right contents in one, but I couldn't get contents in any other. Maybe I just didn't wait enough (see below). 2) When the migration script says it's done, press ok, close KMail, and then restart Akonadi. This worked -- in the sense that I got a single KMail Folders collection with all my folders. However, it postponed the action of actual migration -- when I tried to access the folders, only then KMail did the actual loading and syncing or whatever takes so long. Opening a folder with ~22000 mails took well over 5 minutes, and during these minutes, no other mail could be retrieved (even from the inbox which had already been synced before). Still, no filters were migrated. None whatsoever. I have ~30 of them. I am writing this message from the upgraded user -- as a test to see if the other bug I'm experiencing (sending requires akonadi restart) is specific to my regular user. I encountered another issue -- although my Identity has sign messages automatically set to False, when I started writing the sign message option was on. Perhaps it was on in the last message I sent before doing the backup from which this user is restored, I'm not sure. Either way, it was surprising. Hope this helps, Shai. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727800: kmail: KMail goes bonkers following Qt update
On Sunday 14 December 2014 21:35:45 Diane Trout wrote: The migration bug is the one kmail bug I've been able to reproduce repeteadly. I have a work-around patch that basically does akonadictl restart when the migrator says Migration Done Oh, I haven't followed that advice. I'll try again, hopefully this week. I haven't figured out how to reproduce the other kmail bugs. And yes, one of KDE's design choices is that everything is integrated, so not shipping kmail2 would be very difficult if not impossible. You might also want to check out the current versions of evolution or thunderbird. Both are also quite decent mail clients. This is becoming increasingly off-topic, I'll respond on the mailing list. Thanks, Shai. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727800: kmail: KMail goes bonkers following Qt update
Finally got to it. The upgrade was not smooth at all; the update script ended quickly enough, but the real upgrade only happened when I opened KMail. I had the same experience as last time -- KMail shows blank folders, nothing seems to be going on, except if you look for it. At some point KMail's little operation in progress bar (bottom right progress bar when doing something) lit up with 0%. I tried to click the up-arrow next to it, to see what KMail was doing. After that, looking from another X session, I saw mysqld, akonadiserver and kwin(!) all taking 100% CPU (I have four cores). Pretty soon after that, kwin managed to make the X server completely unresponsive -- not even Ctrl-Alt-Fn to go to another session. This happened twice, so I'm calling it reproducible. Perhaps it should just be reported elsewhere -- I suspect a grave bug in the upgrade. It seems like the mails themselves made it through the upgrade; I haven't yet verified the filters and identities. The computer that runs Sid serves the whole family, and I had to reboot it to restore it to functional after running the imported user. A little off-topic, w.r.t. a comment Lisandro made earlier -- I understand that not shipping Akonadi is not an option. Would you say the same about KMail2? Thanks, Shai. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727800: kmail: KMail goes bonkers following Qt update
The migration bug is the one kmail bug I've been able to reproduce repeteadly. I have a work-around patch that basically does akonadictl restart when the migrator says Migration Done I've been hoping that upstream would say if that's a reasonable work around. I haven't figured out how to reproduce the other kmail bugs. And yes, one of KDE's design choices is that everything is integrated, so not shipping kmail2 would be very difficult if not impossible. You might also want to check out the current versions of evolution or thunderbird. Both are also quite decent mail clients. Diane On Monday, December 15, 2014 01:21:18 Shai Berger wrote: Finally got to it. The upgrade was not smooth at all; the update script ended quickly enough, but the real upgrade only happened when I opened KMail. I had the same experience as last time -- KMail shows blank folders, nothing seems to be going on, except if you look for it. At some point KMail's little operation in progress bar (bottom right progress bar when doing something) lit up with 0%. I tried to click the up-arrow next to it, to see what KMail was doing. After that, looking from another X session, I saw mysqld, akonadiserver and kwin(!) all taking 100% CPU (I have four cores). Pretty soon after that, kwin managed to make the X server completely unresponsive -- not even Ctrl-Alt-Fn to go to another session. This happened twice, so I'm calling it reproducible. Perhaps it should just be reported elsewhere -- I suspect a grave bug in the upgrade. It seems like the mails themselves made it through the upgrade; I haven't yet verified the filters and identities. The computer that runs Sid serves the whole family, and I had to reboot it to restore it to functional after running the imported user. A little off-topic, w.r.t. a comment Lisandro made earlier -- I understand that not shipping Akonadi is not an option. Would you say the same about KMail2? Thanks, Shai. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727800: kmail: KMail goes bonkers following Qt update
I contacted Upstream and although they're too busy right now he offered this clue, in response to my bug description. Diane [Diane] With akonadiconsole under job tracker I can see a vast number of SpecialCollectionsRequestJob's being scheduled and with the query browser see a bunch of Select ... from CollectionTable queries being generated. Screenshot of query browser posted: https://bugs.debian.org/cgi-bin/bugreport.cgi?msg=40;filename=akonadi-quer ie s.png;att=1;bug=727800 I can stop the loop by restarting akonadi, though I'm not sure if that's the best solution to this. Would you have any suggestions? [upstream] From a quick look I'd say the problem is the part when the migrator tries to replace the local maildir resource, created as a requirement for the maildispatcher, with the resource it has just created for the user's previously used folder structure. It might be possible to disable that through a configuration option, but I can't remember with certainty. signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
Hi, On Wednesday 26 November 2014 15:21:15 you wrote: Hi! As Sandro pointed out, this seems fixed in newer versions. Can you please test if this is still an issue for you? I have no means to reconstruct the update as it was. I am using two systems: One of them is a sid which I keep updating. Upgrades ever since that woefull upgrade have been smooth as far as I can tell, although I haven't been watching closely -- ever since kmail2 came into sid, mail there has not been really reliable, so if I suddenly have a couple hundred more unread mails, I don't think I'd notice. The filters haven't been broken again, as far as I can remember. The other system is mostly testing, but since I use kmail for all my critical mail, and kmail2 has not been production-ready (at least before 4.14, though I find it hard to trust even now) I have pinned kmail there; I am writing this message from kmail 1.13.7 on kde 4.14.2. I can try to copy the user from testing to sid and see how the upgrade goes, although this would be a lengthy process; would that be helpful? On a side note -- I see that mails sent to the bug are no longer sent to the submitter automatically (except when sent to -submitter specifically); is this on purpose? It is a bit of a problem. Feel free to include my address explicitly in replies. Thanks, Shai. -- To UNSUBSCRIBE, email to debian-bugs-rc-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#727800: kmail: KMail goes bonkers following Qt update
On Saturday 29 November 2014 15:05:58 Shai Berger wrote: Hi, On Wednesday 26 November 2014 15:21:15 you wrote: Hi! As Sandro pointed out, this seems fixed in newer versions. Can you please test if this is still an issue for you? [snip] I can try to copy the user from testing to sid and see how the upgrade goes, although this would be a lengthy process; would that be helpful? Yes, this could really help. Please note that you should copy ~/.kde and *maybe* ~/.config to the new user. Thanks a lot!!! On a side note -- I see that mails sent to the bug are no longer sent to the submitter automatically (except when sent to -submitter specifically); It never was as far as I remember :) is this on purpose? Yes, sometimes mainatiners discuss the bug but it would be quite noisy for the submitter. It is a bit of a problem. Feel free to include my address explicitly in replies. You can always subscribe to the bug by: - Mailing nn-subscr...@bugs.debian.org where nn is the bug number - Clicking on the subscribe link in the bug page (which will try to send the above mail). -- You know it's love when you memorize her IP number to skip DNS overhead. Anonymous Lisandro Damián Nicanor Pérez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
Shai: please also note Diane's workaround: maybe restarting akonadi after the upgrade solves the issue. That would be: - Unlog from KDE - Do the upgrade (in you case, create the new user and copy the data) - Log in Then wait some minutes, if you see the process doesn't ends open a console and restart akonadi. I'm CCing Diane to see if I missed or made a mistake at some point. Kinds regards, Lisandro. -- Lisandro Damián Nicanor Pérez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
The sequence for me was: * do the upgrade * log in * the knotes upgrade script runs * the kmail2 upgrade script runs * akonadi goes a bit crazy toward the end of the kmail 2 upgrade script, but the upgrade script does report upgrade complete * after the upgrade finished then I restarted akonadi. (Either by doing akonadictl restart or just logging out and back in). Diane On Saturday, November 29, 2014 20:48:33 Lisandro Damián Nicanor Pérez Meyer wrote: Shai: please also note Diane's workaround: maybe restarting akonadi after the upgrade solves the issue. That would be: - Unlog from KDE - Do the upgrade (in you case, create the new user and copy the data) - Log in Then wait some minutes, if you see the process doesn't ends open a console and restart akonadi. I'm CCing Diane to see if I missed or made a mistake at some point. Kinds regards, Lisandro. signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
Some more details about trying to debug this: I did akonadictl restart in a terminal so I could see log messages from akonadi. Then I ran /usr/bin/kmail-migrator --interactive After a bit it started constantly printing: akonadi_maildispatcher_agent($pid)/libakonadi: Resource id's don't match: akonadi_mixedmaildir_resource_0 akonadi_maildir_resource_0 the kmail-migrator seems to be from kdepim-runtime and seems to be the binary automatically launched when you start kmail the first time after upgrading. The mail spool had been configured with both imap and a local maildir. Diane signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
tag 727800 moreinfo thanks Hi! As Sandro pointed out, this seems fixed in newer versions. Can you please test if this is still an issue for you? Kinds regards, Lisandro. -- Without us [Free Software developers], people would study computer science and programming without ever having seen a real program in its entirety. That's like becoming writers without ever having read a complete book. Matthias Ettrich, founder of the KDE project. http://www.efytimes.com/efytimes/25412/news.htm Lisandro Damián Nicanor Pérez Meyer http://perezmeyer.com.ar/ http://perezmeyer.blogspot.com/ signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
Control: reassign 727800 akonadi Hey, there are some newer versions and the bug you mentioned is marked as resolved fixed. The biggest issue with akonadi is, that older akonadi databases are often in kind of a broken state (something like 1.10). That is/was not visible in ( KDE 4.9), 'cause only kaddressbook uses akonadi so far. So for many users the upgrade process is kind of crual, and they have to reset akonadi database + config. Unfortunatelly the package system can't help here 'cause these are living inside the home folder... Well speaking as an upstream dev, hunting update errors is one of the hard ones. 'Cause normally users can't give you the state from before the upgrade and mail setups are so different... But something was synchronizing, so I let it go. When it was done (didn't follow closely, but ~20 minutes later) it had restored my mails -- sort of. Yes akonadi has to index all mails. It managed to forget the read status of ~3 months of mails, so I suddenly had more than 4000 unread mails in my inbox. The 3 months weren't the last ones, but mid April to mid July; I suspect these were the first 3 months of KMail2 on this system. what kind of email folder do you use? imap, maildir, mboxes I never heard about this issue. I have ~20 filters, most of them just send messages to different folders. When I looked at the filter configuration, I realized all the filter target folders were wrong. It wasn't too hard to resotre them, but it had to be done manually. Well the same can/is true for sent/draft folders for identities etc. I know it is annoying, but you to do it only once. A) There was real (though minor) data loss -- the read status, and the filters configurations; B) There was what looked like data loss -- at first, seeing the empty mail folders was scary; it is very easy to panic and kill the program at that stage, which is likely to cause more serious data loss. if you keep akonadi running everything will sync correctly :) btw. akonadi only indexes first and not touching the mails. So no you won't end up in a serios data loss. But I would advertise backuping all pim data before upgrade kdepim-kdepim 4.14. And yes leaving akonadi time to sync is also a good idea :) Regards, sandro signature.asc Description: This is a digitally signed message part.
Bug#727800: kmail: KMail goes bonkers following Qt update
Package: kmail Version: 4:4.10.5-2 Severity: grave Justification: causes non-serious data loss Dear Maintainer, Today, following a safe-upgrade which updated the Qt4 libraries, I found the whole KMail/Akonadi system quite broken. I should note that, after the upgrade, I rebooted the system, as it included updates to the kernel. At first, things seemed normal, except filters weren't working. I closed KMail, and tried to re-open it. It wouldn't start, saying it Failed to fetch resource collection. Now, I am not sure exactly when I restarted what. At some point, akonadiconsole wouldn't start with the same complaint. Later, it opened, but KMail still refused to run. Reports on the web suggested the problem was a misconfigured mail folder, but the mail folder was configured correctly. I noticed that the Akonadi self-test was reporting an error in the mysql server startup -- missing tables. After consulting the web, I decided to try to recreate the database with mysql_install_db --datadir=$HOME/.local/share/akonadi/db_data/ --user=shai --force This seems to have worked -- after a few more false starts, logoff-logon cycles, akonadi restart, and KMail failures, something finally clicked into place and KMail started. At first it seemed like it had forgotten all my mail. But something was synchronizing, so I let it go. When it was done (didn't follow closely, but ~20 minutes later) it had restored my mails -- sort of. It managed to forget the read status of ~3 months of mails, so I suddenly had more than 4000 unread mails in my inbox. The 3 months weren't the last ones, but mid April to mid July; I suspect these were the first 3 months of KMail2 on this system. And then I noted the filters were still not working. I have ~20 filters, most of them just send messages to different folders. When I looked at the filter configuration, I realized all the filter target folders were wrong. It wasn't too hard to resotre them, but it had to be done manually. So -- A) There was real (though minor) data loss -- the read status, and the filters configurations; B) There was what looked like data loss -- at first, seeing the empty mail folders was scary; it is very easy to panic and kill the program at that stage, which is likely to cause more serious data loss. References: https://bbs.archlinux.org/viewtopic.php?pid=695037 for the tables https://bugs.kde.org/show_bug.cgi?id=259355 for the startup fail (comment 10 there seems to be suggested often as a workaround; as noted above, was not relevant for my case. I believe the problem I encountered is similar in effects, but different in cause). -- System Information: Debian Release: jessie/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing') Architecture: amd64 (x86_64) Kernel: Linux 3.11-1-amd64 (SMP w/4 CPU cores) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/dash Versions of packages kmail depends on: ii kde-runtime 4:4.10.5-1 ii kdepim-runtime4:4.10.5-2 ii kdepimlibs-kio-plugins4:4.10.5-1 ii libakonadi-contact4 4:4.10.5-1 ii libakonadi-kde4 4:4.10.5-1 ii libakonadi-kmime4 4:4.10.5-1 ii libakonadiprotocolinternals1 1.9.2-2+b1 ii libc6 2.17-93 ii libcalendarsupport4 4:4.10.5-2 ii libgcc1 1:4.8.2-1 ii libgpgme++2 4:4.10.5-1 ii libincidenceeditorsng44:4.10.5-2 ii libkabc4 4:4.10.5-1 ii libkcalcore4 4:4.10.5-1 ii libkcalutils4 4:4.10.5-1 ii libkcmutils4 4:4.10.5-1+b1 ii libkdecore5 4:4.10.5-1+b1 ii libkdepim44:4.10.5-2 ii libkdeui5 4:4.10.5-1+b1 ii libkio5 4:4.10.5-1+b1 ii libkleo4 4:4.10.5-2 ii libkmime4 4:4.10.5-1 ii libknotifyconfig4 4:4.10.5-1+b1 ii libkontactinterface4 4:4.10.5-1 ii libkparts44:4.10.5-1+b1 ii libkpgp4 4:4.10.5-2 ii libkpimidentities44:4.10.5-1 ii libkpimtextedit4 4:4.10.5-1 ii libkpimutils4 4:4.10.5-1 ii libkprintutils4 4:4.10.5-1+b1 ii libksieveui4 4:4.10.5-2 ii libktnef4 4:4.10.5-1 ii libmailcommon44:4.10.5-2 ii libmailimporter4 4:4.10.5-2 ii libmailtransport4 4:4.10.5-1 ii libmessagecomposer4 4:4.10.5-2 ii libmessagecore4 4:4.10.5-2 ii libmessagelist4 4:4.10.5-2 ii libmessageviewer4 4:4.10.5-2 ii libnepomukcore4 4:4.10.5-1+b1 ii libpimcommon4 4:4.10.5-2 ii libqt4-dbus 4:4.8.5+git121-g2a9ea11+dfsg1-2 ii libqt4-network4:4.8.5+git121-g2a9ea11+dfsg1-2 ii libqt4-xml
