[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=336581

missol...@protonmail.com changed:

   What|Removed |Added

 CC||missol...@protonmail.com

--- Comment #14 from missol...@protonmail.com ---
Just to confirm that the issue is still there for Kmail 5.20.2. The akonadi
database gets corrupted randomly (happened to me after a distro upgrade that
otherwise went perfectly fine) and after removing Akonadi's corrupted data
folders and syncing again, all folder associations were messed up. 

Can't way for the day when our whole company can start relying on KDE PIM's
framework. It looks very promising and the vision behind akonadi is brilliant,
but this data corruption issue is a deal breaker. I hope you manage to fix this
soon, stability and durability are so important for this kind of application. 

If a solution is known but what is lacking is manpower, please let the
community know so that we can try to organize sponsorship. This is such a
critical issue (I am referring to the data corruption. Folder assignment is
more a symptom. If data corruption wasn't there in the first place, folder
assignment mix up wouldn't be an issue. I feel it would probably be better to
fix the fragility of the storage system to make such symptoms non-issues by
removing a vast class of failure modes rather than rather than trying to come
up with global ids).

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Andrey]

https://bugs.kde.org/show_bug.cgi?id=336581

Andrey  changed:

   What|Removed |Added

 CC||kdeb...@openaliasbox.org

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=336581

--- Comment #13 from Martin Steigerwald  ---
(In reply to Frits Spieker from comment #12)

Dear Frits, please refrain from writing unrelated stuff into this bug report.
The bug report is about the effects of accidental database loss on folder
assignments and nothing else. It is not a discussion forum or a mailing list.
Using it for any random comment makes it more difficult to read by any of the
developers. Please use KDE Forum or preferably kdepim-users mailing list to
vent your frustration.

Have a Merry Christmas,
Martin

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Frits Spieker]

https://bugs.kde.org/show_bug.cgi?id=336581

Frits Spieker  changed:

   What|Removed |Added

 CC||s.pa...@spiekerbros.com

--- Comment #12 from Frits Spieker  ---
As to broken architecture... The dreaded KMail hanging with the message
"Retrieving message/ folder content screen" showing indefinitely is back. On
top of that KMail now always crashes when writing a new email using autofill
for the receiver (start typing the email address and when trying to select one
of the suggestions made, KMail will crash). I will search if a bug report has
been filed for that one and if not I will file it myself.

KMail/ Akonadi had been "OK-ish" for a couple of months (to the extent I even
removed my backup solution of Thunderbird), but it is back to being near
useless again.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Kai Maerzhaeuser]

https://bugs.kde.org/show_bug.cgi?id=336581

Kai Maerzhaeuser  changed:

   What|Removed |Added

 CC||ka...@freenet.de

--- Comment #11 from Kai Maerzhaeuser  ---
A short comment on this, since Martin perfectly summarizes my frustration. A
user's opinion, I am not a developer.

Since introduction of akonadi, I experience broken database issue about 1-2
times a Year (last time some days ago) and none of the proposed solutions
anywhere in posts resolves my given issues. Because I don't have the time for
extensive investigations I have to delete ~/.local/share/akonadi which then
totally mixes up folder associations with the described impacts on filters etc.
as described above.

Email is among the most critical applications for daily use and this weakness
in architecture (as I learned from above) is a total mess.

This is not a bug, but a totally broken architecture !!!
Martin's suggestions seem already very reasonable to me, and promising.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=336581

Martin Steigerwald  changed:

   What|Removed |Added

   Platform|Compiled Sources|Debian unstable
Version|GIT (master)|5.2.3

--- Comment #10 from Martin Steigerwald  ---
This is indeed still the case with KMail 5.2.3 (Debian Unstable unfortunately
does not have any newer packages due to the difficulty to properly package Qt
Webengine in order to enable newer KDEPIM versions, which likely means that
Debian will go with KDEPIM 16.04 for next stable version Stretch).

The principal issue of relying on database IDs outside of the database itself
and thus using wrong folders in case of a database loss, potentially causing
data loss, is still present.

For mailfilters:

~/.config> grep action- akonadi_mailfilter_agentrc | head -20
[…]
action-args-0=95
action-name-0=transfer
action-args-0=209
action-name-0=transfer
action-args-0=209
action-name-0=transfer
action-args-0=208
action-name-0=transfer
action-args-0=211
action-name-0=transfer
action-args-0=422
action-name-0=transfer

Action "transfer" means to move the mail into a folder. The corresponding
args-0 is the numerical id of the folder.

For folder settings:

~/.config> fgrep '[Folder-' kmail2rc | head
[Folder-1]
[Folder-10]
[Folder-100]
[Folder-101]
[Folder-102]
[Folder-103]
[Folder-104]
[Folder-105]
[Folder-106]
[Folder-107]

For default folders for mail identities:

~/.config> egrep -i "draft|template" emailidentities | head -10 
Drafts=21
Templates=31
Drafts=21
Templates=31
Drafts=21
Templates=31
Drafts=21
Templates=31
Drafts=411
Templates=378

Default inbox folder for POP mail resources:

martin@merkaba:~/.config> grep targetCollection akonadi_pop3_resource_0rc
targetCollection=361

I bet is similar for IMAP resources.


My expected result is this:

1) Either keep / rebuild / reassign references after a database loss properly
*or*

2) *loose* the references and require the user to reconfigure.

Yet *never ever* use *wrong* references, just do to a database loss.

Use a unique hash for identification, not just the index number of the database
entry. In that way you can at least *know* when the reference is no more valid.
*Or* store *all* of these reference into the database itself, so that they are
gone for good if the database is rebuilt.

For user convenience I would aim for approach 1 wherever possible which is
easier to do with a hash based approach. For maildir resource, you can store
the folder identification in the maildir filesystem, or IMAP it becomes more
difficult tough, yet you could at least also store the name and then if the
reference by hash is gone, you can show the folder name to the user and
possibly even autoselect it for confirmation.


You can argue that a database loss should not happen and with proper MariaDB
configuration, proper filesystem and proper storage media you might well be
right about that – *yet* in practice it did happen to a lot of users. So please
make Akonadi *failure proof* and more *robust* in cases like this.

Another approach would be to use something else than a database for metadata or
rethink the Akonadi caching concept completely like Aaron and others do with
Sink for Kubemail. I don´t see this happening for KMail from KDEPIM soon, so I
think it makes sense to make it more failure proof even before that may happen.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Martin Steigerwald]

https://bugs.kde.org/show_bug.cgi?id=336581

Martin Steigerwald  changed:

   What|Removed |Added

 Resolution|UNMAINTAINED|---
 Status|RESOLVED|REOPENED

--- Comment #9 from Martin Steigerwald  ---
Thanks, Denis.

This is a principal issue of the current Akonadi design and still present, as I
am not aware of any design change. Thus reopening.

There is no need to open a new bug report about it.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Denis Kurz]

https://bugs.kde.org/show_bug.cgi?id=336581

Denis Kurz  changed:

   What|Removed |Added

 Resolution|WAITINGFORINFO  |UNMAINTAINED
 Status|NEEDSINFO   |RESOLVED

--- Comment #8 from Denis Kurz  ---
Just as announced in my last comment, I close this bug. If you encounter it
again in a recent version (at least 5.0 aka 15.08), please open a new one
unless it already exists. Thank you for all your input.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Denis Kurz via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=336581

Denis Kurz  changed:

   What|Removed |Added

 Status|CONFIRMED   |NEEDSINFO
 Resolution|--- |WAITINGFORINFO

--- Comment #7 from Denis Kurz  ---
This bug has only been reported for versions older than KDEPIM 4.14 (at most
akonadi-1.3). Can anyone tell if this bug still present?

If noone confirms this bug for a recent version of akonadi (part of KDE
Applications 15.08 or later), it gets closed in about three months.

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[Martin Steigerwald via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=336581

Martin Steigerwald  changed:

   What|Removed |Added

 Status|UNCONFIRMED |CONFIRMED
 Ever confirmed|0   |1

--- Comment #6 from Martin Steigerwald  ---
David, that is exactly what happens. It is known to the developers. Challenge
is to get unique ids working with all Akonadi resources, say for example IMAP.
How do uniquely identify a mail with IMAP? Always check for the message-id.

We discussed this on KDE Randa Meetings last year and we didn´t come to an easy
implementable solution so far. I do not recall all of the discussion. If you
want to move this further along I suggest you bring up the topic on kde-pim
mailing list.

Other than that thank you that you create bug relations. There are several
duplicates of it. I am pretty sure I reported it as well..

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[David Tonhofer via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=336581

--- Comment #5 from David Tonhofer  ---
Also bug#340529

-- 
You are receiving this mail because:
You are watching all bug changes.

[Akonadi] [Bug 336581] accidental database loss causes Akonadi / KMail to silently break correct folder assignments

[David Tonhofer via KDE Bugzilla]

https://bugs.kde.org/show_bug.cgi?id=336581

David Tonhofer  changed:

   What|Removed |Added

 CC||bugh...@gluino.name

--- Comment #4 from David Tonhofer  ---
Same  as bug#283345 I guess

I have a similar one with kaddressbook bug#357266 - and my guess is that the
application uses short numeric ids to reference akonadi database records which,
after a rebuild, have completely different ids at random. Just a hunch.

-- 
You are receiving this mail because:
You are watching all bug changes.