Re: Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Hi,

forgot the cyrus version: 2.4.12 on Ubuntu 12.04 LTS

Ciao!



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Replication problem do_folders(): failed to rename

[Marcus Schopen via Info-cyrus]

Hi,

I have a problem with a single mailbox. The user's Outlook crashed and
since then the sync_client is running wild on this user account and
produces high load on the master. I stopped sync_client on master side
for the moment.

When I try to sync the user by hand 

/bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v
-u testuser

I do get the following error.

Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO
response: Operation is not supported on mailbox
Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to
rename: user.elsa-secgen -> user.testuser.Archives.Gesch&AOQ-ftsjahr
2014-15.25-Jahr-Feier 
Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote
Server(s) denied the operation
Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in
do_user(testuser): bailing out!

Comparing master and slave on filesystem I do see the subfolder
"25-Jahr-Feier" in "user.testuser.Archives.Gesch&AOQ-ftsjahr 2014-15.",
but only on master but not on slave side. And why does sync_client want
to rename and where does it get this order from?

I can login into the users' mailbox on master side and new message are
shown in the INBOX.

How can I fix it? 

Should I try a "reconstruct -r user.testuser" on master and slave or
just on slave? (do I have to shutdown cyrus for a reconstruct -r on a
user box?)

Or can I delete the complete mailbox on slave side start an "sync_client
-S replicaserver -v -u testuser"?

Thanks for helping
Marcus



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Trouble with Outlook 2013, especially with the upgrade from 12/8

[Sebastian Hagedorn via Info-cyrus]

Please disregard my second mail. I forgot that we have backups of the 
mailbox list. With those backups we were able to see that the user wasn't 
telling the truth after all. The folders didn't exist on the server prior 
to Wednesday, so it's exactly the same problem as with other Outlook 2013 
users: locally stored folders that are created on the server, but deleted 
locally after the upgrade from 12/8.


--On 11. Dezember 2015 um 17:04:49 +0100 Sebastian Hagedorn via Info-cyrus 
 wrote:



I have a follow-up question after talking to a user on the phone. Here's
her current directory layout (partially redacted):

--
   .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
.:.Regionales Rechenzentrum (RRZK).:.
  .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.

pgpz5uFfYt84I.pgp
Description: PGP signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Trouble with Outlook 2013, especially with the upgrade from 12/8

[Andrea Venturoli via Info-cyrus]

On 12/11/15 14:18, Sebastian Hagedorn via Info-cyrus wrote:

Hi,

we've always had more problems with users of Outlook than other clients,
but in recent days they have become much graver. A number of users have
lost mail in a way that we couldn't recover, even though we are using
expunge_mode: delayed. There seems to be some correlation to this
upgrade from Microsoft:



However, details are vague. There seem to be two types of problems:


As others pointed out, OL and IMAP never put up with each other well and 
2013 is a lot worse than earlier versions, to the point it's not usable.


Troubles I've encountered include:
_ inability to choose folders for Sent mail, Drafts, etc...;
_ as a cool addition to the above, if you add an account which already 
has a Sent mail folder which is not called exactly "Sent", the new 
"Sent" folder will be "This computer only";
_ on some installations, long delays (tens of minutes) before mail 
starts arriving after the program is launched;
_ hours of no mail arriving (not even pushing the fetch button or 
restarting the program); of course new mail is on the server and it will 
suddenly start downloading again for no reason;
_ creating folders usually work (provided their name is legal), but 
sometimes they won't show up until a restart;
_ folders with a legal name often fail to be created: either no error 
message is produced or the folder is created only locally (without any 
further sync);
_ creating folders with unallowed characters fails on the server, but 
not on the client and won't be synchronized thereafter; if you are 
lucky, "This computer only" is appended to the name, so you know, but 
don't count on this too much;
_ moving messages into a folder works most of the time, but you might 
see the folder full, then suddenly empty, then possibly filling up again 
after a restart;
_ legal renaming of folders might work or not; again no error, but it 
might succeed locally (and stop uploading messages);
_ ability to rename "Other Users" or "Other Users.mailbox": of course 
this will fail on the server, but no indication of that is given; 
instead any subfolder will stop uploading new messages to the server 
(hope I don't need to describe the risks of this!);
_ frequent corruption of .pst/.ost files or whatever else, so you need 
to recreate the profile (but this was also true in 2007 and 2010);
_ mail disappearing for no reason (*) and if you are lucky it had been 
pushed to the server, so you have backups.


(*) I might think user1 accidentally hit the Delete button, I might 
think user2 somehow delete its mail by mistake, but when user3, 4, 5, 
etc... call and tell me the same thing...





I agree with Andre': such an aggravation over previous versions can only 
be intentional (to help selling Exchange).

In any case bottom line is: either drop OutLook or Cyrus.

 bye
av.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Trouble with Outlook 2013, especially with the upgrade from 12/8

[Sebastian Hagedorn via Info-cyrus]

--On 11. Dezember 2015 um 14:18:32 +0100 Sebastian Hagedorn via Info-cyrus 
 wrote:



we've always had more problems with users of Outlook than other clients,
but in recent days they have become much graver. A number of users have
lost mail in a way that we couldn't recover, even though we are using
expunge_mode: delayed. There seems to be some correlation to this upgrade
from Microsoft:



However, details are vague. There seem to be two types of problems:

1. Folders aren't synched with the IMAP server. There seems to be no
indication of that in the UI, but has the consequence that mail for that
folder is stored locally on the PC only. When the upgrade above is
installed, something resets, which has the result that the local copies
are purged. Since there are no copies on the server, the mails are lost.

2. Folders are moved unexpectedly when you move or rename a folder. This
is mentioned in the release notes above as a supposedly fixed bug. It
certainly has happened prior to the upgrade, but it's less clear if the
bug is really fixed. This is especially harmful if the folders are moved
to the Trash.

We were able to undelete some such folders, but according to at least one
user (some) mail or folders inside those folders were lost. We can find
no evidence of that in our logs.

Has anybody else observed issues like these? We are running 2.4.18, FWIW.


Sorry, it's 2.4.17. I have a follow-up question after talking to a user on 
the phone. Here's her current directory layout (partially redacted):


$ tree -d
.
|-- 01_ArbeitDiversity Trouble
|-- 01_ArbeitFuturologie
|-- 01_ArbeitGermanistentag 2016
|-- 01_ArbeitHB Literatur und Recht
|-- 01_ArbeitKosmologien
|-- 02_K&APY-ln1_CO_Poststrukturalismus
|-- 02_K&APY-ln1_HS_Atlantis
|-- 02_K&APY-ln1_HS_Pikaro
|-- 02_K&APY-ln1_HS_Science Fiction 1900
|-- 02_K&APY-ln1_VL_Literatur und Medien 1600 - 1900
|-- 02_K&APY-lnHausarbeiten WS 2014_15
|-- 02_K&APY-lnKlausuren
|-- 02_K&APY-lnVerwaltung
|-- 02_K&APY-lnmdl Pr&APw-fungen
|-- 03_Uni
|-- 03_UniPromotion xxx
|-- 03_UniPromotion yyy
|-- 04_Stellenmarkt
|-- 04_Stellenmarktxxx
|-- 04_Stellenmarktyyy
|-- Deleted Messages
|-- Drafts
|-- ESET Antispam
|-- Gel&APY-schte Elemente
|-- Gesendete Elemente
|   `-- Notes
|-- Infected Items
|-- Junk-E-Mail
|-- Notes
|-- Sent
|-- Trash
`-- sent-mail

32 directories

She claims that prior to Wednesday afternoon those folders were 
hierarchically ordered. There was a folder "01_Arbeit" that contained a 
folder named "Futurologie" and so on. I believe her, because she says she 
worked with a webmailer, her iPhone and Outlook 2013, and was able to see 
that hierarchy and all her mails on all clients. What baffles me is that 
all I find in our imapd.log files is this:


/var/log/imapd.log.2.gz:Dec  9 17:18:40 clotho imap[9137]: Rename: 
user.xxx.Gesendete Elemente.Notes -> DELETED.user.xxx.Gesendete 
Elemente.Notes.56685460
/var/log/imapd.log.2.gz:Dec  9 17:18:41 clotho imap[9137]: Rename: 
user.xxx.Gel&APY-schte Elemente.xxx -> DELETED.user.xxx.Gel&APY-schte 
Elemente.xxx.56685461
/var/log/imapd.log.1.gz:Dec 10 13:13:33 clotho imap[30707]: Rename: 
user.xxx.01_Arbeit -> user.xxx.Gel&APY-schte Elemente.01_Arbeit
/var/log/imapd.log.1.gz:Dec 10 13:13:40 clotho imap[30707]: Rename: 
user.xxx.02_K&APY-ln -> user.xxx.Gel&APY-schte Elemente.02_K&APY-ln
/var/log/imapd.log.1.gz:Dec 10 13:13:50 clotho imap[507]: Rename: 
user.xxx.03_Uni -> user.xxx.Gel&APY-schte Elemente.03_Uni


Our logs go back 7 days, and there are no other Renames previous to those.

Is it at all possible that some operation changed the folder layout without 
corresponding Rename entries in imapd.log? I'm a little scared that an 
Outlook bug triggered another bug in Cyrus IMAP.

--
   .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
.:.Regionales Rechenzentrum (RRZK).:.
  .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.


pgpiBlI3yIKlX.pgp
Description: PGP signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Trouble with Outlook 2013, especially with the upgrade from 12/8

[Leena Heino via Info-cyrus]

On Fri, 11 Dec 2015, André Schild via Info-cyrus wrote:

Outlook and IMAP is just pain, it's not intended to work, otherwise who would 
buy the exchange server?


1) We see this rather seldom, but sometimes mails come in after 2-10 hours, 
not even a "Fetch mail" helps


2) Definitively a major problem of Outlook 2013, working with IMAP folders in 
copy/rename operations is random at best


We might have had similar problems with Outlook 2013:
- Outlook would sometimes fail to create folders or recognize folders it
  had created earlier
- Messages would get deleted or simply lost when they were copied between
  folders

Attached is a patch that we used locally with Cyrus IMAPD 2.4.17 to 
improve compatibility with Outlook 2013.


--
  Leena Heino  University of Tampere / Computer Centre
  ( liinu at uta.fi )  ( http://www.uta.fi/laitokset/tkk )Patch to make Cyrus Imapd compatible with Outlook 2013.
This patch changes XLIST behaviour so that it works the same as in LIST
in a special case. After this patch the output should be:
. LIST "" ""
* LIST (\Noselect) "/" ""
. OK Completed (0.000 secs)
. XLIST "" ""
* XLIST (\Noselect) "/" ""
. OK Completed (0.000 secs)

diff -Naur cyrus-imapd-2.4.17.orig/imap/imapd.c cyrus-imapd-2.4.17/imap/imapd.c
--- cyrus-imapd-2.4.17.orig/imap/imapd.c2012-12-01 21:57:54.0 
+0200
+++ cyrus-imapd-2.4.17/imap/imapd.c 2014-07-24 13:51:26.885893900 +0300
@@ -6029,10 +6029,14 @@
 
 list_callback_calls = 0;
 
-if (!listargs->pat->s[0] && !(listargs->cmd & LIST_CMD_LSUB)) {
+if (!listargs->pat->s[0] && (listargs->cmd == LIST_CMD_LIST)) {
/* special case: query top-level hierarchy separator */
prot_printf(imapd_out, "* LIST (\\Noselect) \"%c\" \"\"\r\n",
imapd_namespace.hier_sep);
+} else if (!listargs->pat->s[0] && (listargs->cmd == LIST_CMD_XLIST)) {
+   /* special case: query top-level hierarchy separator */
+   prot_printf(imapd_out, "* XLIST (\\Noselect) \"%c\" \"\"\r\n",
+   imapd_namespace.hier_sep);
 } else if (((listargs->sel & LIST_SEL_SUBSCRIBED) ||
(listargs->ret & LIST_RET_SUBSCRIBED)) &&
   (backend_inbox || (backend_inbox = 
proxy_findinboxserver(imapd_userid {

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Re: Trouble with Outlook 2013, especially with the upgrade from 12/8

[André Schild via Info-cyrus]

Am 11.12.2015 um 14:18 schrieb Sebastian Hagedorn via Info-cyrus:

Hi,

we've always had more problems with users of Outlook than other 
clients, but in recent days they have become much graver. A number of 
users have lost mail in a way that we couldn't recover, even though we 
are using expunge_mode: delayed. There seems to be some correlation to 
this upgrade from Microsoft:




However, details are vague. There seem to be two types of problems:

1. Folders aren't synched with the IMAP server. There seems to be no 
indication of that in the UI, but has the consequence that mail for 
that folder is stored locally on the PC only. When the upgrade above 
is installed, something resets, which has the result that the local 
copies are purged. Since there are no copies on the server, the mails 
are lost.


2. Folders are moved unexpectedly when you move or rename a folder. 
This is mentioned in the release notes above as a supposedly fixed 
bug. It certainly has happened prior to the upgrade, but it's less 
clear if the bug is really fixed. This is especially harmful if the 
folders are moved to the Trash.


We were able to undelete some such folders, but according to at least 
one user (some) mail or folders inside those folders were lost. We can 
find no evidence of that in our logs.


Has anybody else observed issues like these? We are running 2.4.18, FWIW.


Outlook and IMAP is just pain, it's not intended to work, otherwise who 
would buy the exchange server?


1) We see this rather seldom, but sometimes mails come in after 2-10 
hours, not even a "Fetch mail" helps


2) Definitively a major problem of Outlook 2013, working with IMAP 
folders in copy/rename operations is random at best


André



Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Trouble with Outlook 2013, especially with the upgrade from 12/8

[Sebastian Hagedorn via Info-cyrus]

Hi,

we've always had more problems with users of Outlook than other clients, 
but in recent days they have become much graver. A number of users have 
lost mail in a way that we couldn't recover, even though we are using 
expunge_mode: delayed. There seems to be some correlation to this upgrade 
from Microsoft:




However, details are vague. There seem to be two types of problems:

1. Folders aren't synched with the IMAP server. There seems to be no 
indication of that in the UI, but has the consequence that mail for that 
folder is stored locally on the PC only. When the upgrade above is 
installed, something resets, which has the result that the local copies are 
purged. Since there are no copies on the server, the mails are lost.


2. Folders are moved unexpectedly when you move or rename a folder. This is 
mentioned in the release notes above as a supposedly fixed bug. It 
certainly has happened prior to the upgrade, but it's less clear if the bug 
is really fixed. This is especially harmful if the folders are moved to the 
Trash.


We were able to undelete some such folders, but according to at least one 
user (some) mail or folders inside those folders were lost. We can find no 
evidence of that in our logs.


Has anybody else observed issues like these? We are running 2.4.18, FWIW.
--
   .:.Sebastian Hagedorn - Weyertal 121 (Gebäude 133), Zimmer 2.02.:.
.:.Regionales Rechenzentrum (RRZK).:.
  .:.Universität zu Köln / Cologne University - ✆ +49-221-470-89578.:.

pgpfjSFgyxXRk.pgp
Description: PGP signature

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus