Re: Bring back ANNOTATEMORE support
On 09/16/2018 08:29 AM, Bron Gondwana wrote: > Right! OK, we'll add it back in 3.0 and 3.2 then (with a config option > to turn it on), and deprecate it after that :) That's great, thanks Bron. Kind regards, Samir Aguiar
Re: Bring back ANNOTATEMORE support
Hi Bron, On 09/12/2018 03:49 AM, Bron Gondwana wrote: > Samir: what's your plan for moving those users to something which > supports standard METADATA? Because the downside of keeping on > supporting a legacy ANNOTATEMORE in Cyrus is that somebody has to > maintain that forever if you're never planning to get off it, either you > or everyone else touching that code! Yes, that's a fair point. The plan is to update our client to use METADATA instead of ANNOTATEMORE, while keeping support for both in the server. After this we deprecate ANNOTATEMORE to signal customers that they should upgrade their clients. This deprecation should happen mostly likely within two years, and in the next years we will be removing it completely from the server. Kind regards, Samir Aguiar
Re: Bring back ANNOTATEMORE support
Hi Dilyan, On 09/11/2018 06:17 PM, Dilyan Palauzov wrote: > it does not make sense to keep the ANNOTATEMORE code just for your specific > case. > > You are entitled to issue an updated client software dealing with METADATA > and ask users to update, or for your server to revert the respective change. Yes, absolutely. We will need to revert that change and restore ANNOTATEMORE support anyway since it's not possible at the moment for all of our clients to upgrade. But I believe my question was ill-phrased. I actually meant to ask if such revert, after done by us, would be accepted upstream as an opt-in feature, or if the team has already decided to drop that implementation for good. Kind regards, Samir Aguiar
Re: Bring back ANNOTATEMORE support
Hi Bron, On 09/10/2018 08:26 PM, Bron Gondwana wrote: > The ANNOTATEMORE was a draft which got replaced with METADATA. If there > a reason why you are still using the draft rather than the spec which > was published in 2009? Yes, when our groupware client was planned and developed only ANNOTATEMORE was available. This client is now is in use at several thousand customers and we have no control over which software version they use. Samir
Bring back ANNOTATEMORE support
Hi all, We are currently upgrading to 3.0.8, but 3.1.x will probably be stable by the time we are finished. From what I could see ANNOTATEMORE support was dropped in 3.1. Would it possible to bring it back and maybe make it opt-in in imapd.conf? Our groupware clients depend heavily on it and it would take a while until we can fully deprecate it. I ran some tests and apparently reverting #26fbf999312 seems to be enough, but please correct me if I'm wrong. Thank you in advance. Kind regards, Samir Aguiar
Re: Mailbox deletion race: folders and files are never deleted
On Friday 27 February 2015 09:31:00 Bron Gondwana wrote: On Fri, Feb 27, 2015, at 06:31 AM, Jeroen van Meeuwen (Kolab Systems) wrote: On 2015-02-26 16:51, Samir Aguiar wrote: Proposed solution: - Patch the mailbox_close() function to reload the index before trying to clean the files (and merge the current index changes with the ones found) - Because the above would still be skipped when shutting down Cyrus: expand cyr_expire to find mailboxes with the deleted flag set (by looping through the filesystem, and not by using the database) and try to remove them. This could then be run periodically by administrators. What do you think? Please do not take my word for it, but I think Cyrus IMAP 2.5 introduces a state flag of MBTYPE_DELETE for mailboxes [1] in the actual mboxlist, so the related functions would still be able to find it after the facts of the case. IIRC, I ran in to this (particular MBTYPE) because it purges ACLs, and rendered (discrete) murder topologies unable to ctl_mboxlist -m (as well as made ctl_mboxlist -d segfault). I think the intention is exactly what you describe is the scenario, but if it is I'm not certain it also properly addresses it. Yes, MBTYPE_DELETE would help for this, that was my first thought. Its main purpose is to enable true master/master replication and all the other good things like efficient JMAP mailboxListUpdates, but safe cleanup is a win too. Bron. That's great, we will then revisit this in 2.5. Best regards, Samir Aguiar
Mailbox deletion race: folders and files are never deleted
Hi,
we found a race condition problem in Cyrus IMAP 2.4.17 where mailbox folders
(and the files inside them) are not removed when deleting a mailbox with delete
mode set to immediate.
The steps to reproduce are:
- Open a mailbox in one session (session A);
- Delete that mailbox with another session (session B).
Session B updates the index with the OPT_MAILBOX_DELETED flag and removes the
mailbox from the database. It expects session A to wipe out the files from the
filesystem later in mailbox_close().
Because the database now has no record of the mailbox, no further sessions
will be able to clean up those files. Neither will cyr_expire, because that
mailbox won't be listed by the mboxlist* functions. So session A is the one
that should remove the leftovers.
However, when closing the mailbox, session A executes the following piece of
code:
/* drop the index lock here because we'll lose our right to it
* when try to upgrade the mboxlock anyway. */
mailbox_unlock_index(mailbox, NULL);
/* do we need to try and clean up? (not if doing a shutdown,
* speed is probably more important!) */
if (!in_shutdown (mailbox-i.options MAILBOX_CLEANUP_MASK)) {
int r = mailbox_mboxlock_reopen(listitem, LOCK_NONBLOCKING);
if (!r) r = mailbox_open_index(mailbox);
if (!r) r = mailbox_lock_index(mailbox, LOCK_EXCLUSIVE);
if (!r) {
/* finish cleaning up */
if (mailbox-i.options OPT_MAILBOX_DELETED)
mailbox_delete_cleanup(mailbox-part, mailbox-name);
else if (mailbox-i.options OPT_MAILBOX_NEEDS_REPACK)
mailbox_index_repack(mailbox);
else if (mailbox-i.options OPT_MAILBOX_NEEDS_UNLINK)
mailbox_index_unlink(mailbox);
/* or we missed out - someone else beat us to it */
}
/* otherwise someone else has the mailbox locked
* already, so they can handle the cleanup in
* THEIR mailbox_close call */
}
There are two problems here:
- if the in_shutdown flag is set, session A will exit and the files will be
forgotten forever;
- the index file is only reloaded by the call to mailbox_open_index, which is
inside the if. That means that session A will be unaware of the clean up mask
set by session B when checking this if and again the files will never be
cleaned.
Proposed solution:
- Patch the mailbox_close() function to reload the index before trying to
clean the files (and merge the current index changes with the ones found)
- Because the above would still be skipped when shutting down Cyrus: expand
cyr_expire to find mailboxes with the deleted flag set (by looping through the
filesystem, and not by using the database) and try to remove them. This could
then be run periodically by administrators.
What do you think?
Best regards,
Samir Aguiar
