Disappearing Mailbox Content
We have an odd situation that I cannot track down regarding all content in a mailbox disappearing (minus sub mailboxes). CentOS release 6.5 Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6 A while back we setup a Cyrus install for a client to use as a mail repository for customer contact. They have one account which is shared across all users in the office. Approximate size of the account is 85GB. At times when a user tries to move email into a Customers mailbox the action is denied. On the client side they see do see email in the mailbox (cached). Upon further review everything is gone for that mailbox including: cyrus.cache, cyrus.header, cyrus.index. Backups contain everything including past emails and the core Cyrus files. If the mailbox contained other mailboxes they are not affected. Reconstructing it corrects the issue, luckily our backups do not propagate deletes. Problem is that we can't find any record of the mailbox being deleted. The content just disappears. We have been running Cyrus for years and have never seen anything like this. I am leaning towards user error but I can't identify what that would be. Any insight would be appreciated. ___ Robert Covell Rolet Hosting & Interactive www.rolet.com Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Disappearing Mailbox Content
I have verified that the user is allowed to delete the mailbox. If I change it at the root level will it propagate down? If not how can this be done without writing a script? As stated before they have thousands of folders. ___ Robert Covell Rolet Hosting & Interactive www.rolet.com -Original Message- From: info-cyrus-bounces+rcovell=rolet@lists.andrew.cmu.edu [mailto:info-cyrus-bounces+rcovell=rolet@lists.andrew.cmu.edu] On Behalf Of Joseph Brennan Sent: Wednesday, September 9, 2015 2:09 PM To: info-cyrus@lists.andrew.cmu.edu Subject: RE: Disappearing Mailbox Content --On September 9, 2015 at 13:49:40 -0500 "Robert T. Covell" <rcov...@rolet.com> wrote: > That is the problem. I cannot reproduce (reliably or at all). It > might be months before we hear about it. It has been happening for > about two years. Always chalked it up to user error. But I can't say > that it is or isn't. > Check the ACL on the mailbox and make sure no user has permission to delete the mailbox-- no "x" right. Loss of all content including the cyrus* files is mailbox deletion. (This is the voice of experience talking) Too bad you didn't implement delayed delete. You can just rename the deleted folder to its original name. Joseph Brennan Columbia University Information Technology Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Disappearing Mailbox Content
That is the problem. I cannot reproduce (reliably or at all). It might be months before we hear about it. It has been happening for about two years. Always chalked it up to user error. But I can't say that it is or isn't. -Original Message- From: Dan White [mailto:dwh...@olp.net] Sent: Wednesday, September 9, 2015 1:44 PM To: Robert T. Covell <rcov...@rolet.com> Cc: signaldevelo...@gmail.com; info-cyrus@lists.andrew.cmu.edu Subject: Re: Disappearing Mailbox Content If you can reliably reproduce the problem on a test account, enable telemetry logging to capture what the client is doing. On 09/09/15 13:07 -0500, Robert T. Covell wrote: >Nope. On the original user they might have a cached view. So to them it >looks like there is no issue until they try to interact with the folder. If >you were to go to a new/fresh client it is empty (minus potential sub-folders). > >-Original Message- >From: signaldevelo...@gmail.com [mailto:signaldevelo...@gmail.com] >Sent: Wednesday, September 9, 2015 1:02 PM >To: Robert T. Covell <rcov...@rolet.com> >Cc: info-cyrus@lists.andrew.cmu.edu >Subject: Re: Disappearing Mailbox Content > >And if you connect some type of IMAP client up to the account do the messages >show? > >Sent from my iPhone > >> On Sep 9, 2015, at 1:18 PM, Robert T. Covell <rcov...@rolet.com> wrote: >> >> Is the user in the RC db still? Are the folders displaying properly in >> Roundxube and they are just empty? >> >> >> On Sep 9, 2015, at 11:03 AM, Robert T. Covell <rcov...@rolet.com> wrote: >> We have an odd situation that I cannot track down regarding all content in a >> mailbox disappearing (minus sub mailboxes). >> >> CentOS release 6.5 >> Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6 >> >> A while back we setup a Cyrus install for a client to use as a mail >> repository for customer contact. They have one account which is shared >> across all users in the office. Approximate size of the account is 85GB. >> >> At times when a user tries to move email into a Customers mailbox the action >> is denied. On the client side they see do see email in the mailbox >> (cached). Upon further review everything is gone for that mailbox >> including: cyrus.cache, cyrus.header, cyrus.index. Backups contain >> everything including past emails and the core Cyrus files. If the mailbox >> contained other mailboxes they are not affected. Reconstructing it corrects >> the issue, luckily our backups do not propagate deletes. >> >> Problem is that we can’t find any record of the mailbox being deleted. The >> content just disappears. We have been running Cyrus for years and have >> never seen anything like this. I am leaning towards user error but I can’t >> identify what that would be. >> >> Any insight would be appreciated. -- Dan White Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Disappearing Mailbox Content
Is the user in the RC db still? Are the folders displaying properly in Roundxube and they are just empty? On Sep 9, 2015, at 11:03 AM, Robert T. Covell <rcov...@rolet.com> wrote: We have an odd situation that I cannot track down regarding all content in a mailbox disappearing (minus sub mailboxes). CentOS release 6.5 Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6 A while back we setup a Cyrus install for a client to use as a mail repository for customer contact. They have one account which is shared across all users in the office. Approximate size of the account is 85GB. At times when a user tries to move email into a Customers mailbox the action is denied. On the client side they see do see email in the mailbox (cached). Upon further review everything is gone for that mailbox including: cyrus.cache, cyrus.header, cyrus.index. Backups contain everything including past emails and the core Cyrus files. If the mailbox contained other mailboxes they are not affected. Reconstructing it corrects the issue, luckily our backups do not propagate deletes. Problem is that we can’t find any record of the mailbox being deleted. The content just disappears. We have been running Cyrus for years and have never seen anything like this. I am leaning towards user error but I can’t identify what that would be. Any insight would be appreciated. ___ Robert Covell Rolet Hosting & Interactive www.rolet.com Yes, the folders do still display but are empty. If the folder had children they are not affected, just the primary contents of the folder in question. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Disappearing Mailbox Content
Nope. On the original user they might have a cached view. So to them it looks like there is no issue until they try to interact with the folder. If you were to go to a new/fresh client it is empty (minus potential sub-folders). -Original Message- From: signaldevelo...@gmail.com [mailto:signaldevelo...@gmail.com] Sent: Wednesday, September 9, 2015 1:02 PM To: Robert T. Covell <rcov...@rolet.com> Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: Disappearing Mailbox Content And if you connect some type of IMAP client up to the account do the messages show? Sent from my iPhone > On Sep 9, 2015, at 1:18 PM, Robert T. Covell <rcov...@rolet.com> wrote: > > Is the user in the RC db still? Are the folders displaying properly in > Roundxube and they are just empty? > > > On Sep 9, 2015, at 11:03 AM, Robert T. Covell <rcov...@rolet.com> wrote: > We have an odd situation that I cannot track down regarding all content in a > mailbox disappearing (minus sub mailboxes). > > CentOS release 6.5 > Cyrus IMAP v2.4.17-Invoca-RPM-2.4.17-6.el6 > > A while back we setup a Cyrus install for a client to use as a mail > repository for customer contact. They have one account which is shared > across all users in the office. Approximate size of the account is 85GB. > > At times when a user tries to move email into a Customers mailbox the action > is denied. On the client side they see do see email in the mailbox (cached). > Upon further review everything is gone for that mailbox including: > cyrus.cache, cyrus.header, cyrus.index. Backups contain everything including > past emails and the core Cyrus files. If the mailbox contained other > mailboxes they are not affected. Reconstructing it corrects the issue, > luckily our backups do not propagate deletes. > > Problem is that we can’t find any record of the mailbox being deleted. The > content just disappears. We have been running Cyrus for years and have never > seen anything like this. I am leaning towards user error but I can’t > identify what that would be. > > Any insight would be appreciated. > > ___ > Robert Covell > Rolet Hosting & Interactive > www.rolet.com > > > > Yes, the folders do still display but are empty. If the folder had children > they are not affected, just the primary contents of the folder in question. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: Disappearing Mailbox Content
I will check the ACL, good call. We do have delayed delete implemented but when we get there it has always been too late. They have thousands of folders. What would be the best way to propagate that change via cyradm? Will report back. -Original Message- From: info-cyrus-bounces+rcovell=rolet@lists.andrew.cmu.edu [mailto:info-cyrus-bounces+rcovell=rolet@lists.andrew.cmu.edu] On Behalf Of Joseph Brennan Sent: Wednesday, September 9, 2015 2:09 PM To: info-cyrus@lists.andrew.cmu.edu Subject: RE: Disappearing Mailbox Content --On September 9, 2015 at 13:49:40 -0500 "Robert T. Covell" <rcov...@rolet.com> wrote: > That is the problem. I cannot reproduce (reliably or at all). It > might be months before we hear about it. It has been happening for > about two years. Always chalked it up to user error. But I can't say > that it is or isn't. > Check the ACL on the mailbox and make sure no user has permission to delete the mailbox-- no "x" right. Loss of all content including the cyrus* files is mailbox deletion. (This is the voice of experience talking) Too bad you didn't implement delayed delete. You can just rename the deleted folder to its original name. Joseph Brennan Columbia University Information Technology Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
RE: 2.6 Kernel and POP issues
As of this time I am happy to report that adding --with-devrandom=/dev/urandom to the sasl config has solved the issue for a 2.6 kernel. I was finally able to implement the compile time feature last week and seem to be running fine. Just a follow-up to the thread... -Bob From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Robert T. Covell Sent: Saturday, December 02, 2006 1:24 PM To: Sebastian Hagedorn Cc: info-cyrus@lists.andrew.cmu.edu Subject: RE: 2.6 Kernel and POP issues Well it figures that I recomplied cyrus imap only and NOT sasl. I will recompile that and put in the allowapop statement. Report to follow... -Thanks Bob From: Sebastian Hagedorn [mailto:[EMAIL PROTECTED] Sent: Sat 12/2/2006 12:20 PM To: Robert T. Covell Cc: info-cyrus@lists.andrew.cmu.edu Subject: RE: 2.6 Kernel and POP issues -- Robert T. Covell [EMAIL PROTECTED] is rumored to have mumbled on 1. Dezember 2006 18:50:11 -0600 regarding RE: 2.6 Kernel and POP issues: I have recompiled my cyrus implementation (--with-devrandom=/dev/urandom) and my kernel with no luck. I reboot from the newest 2.4 kernel and my pop clients hang. When checking the 2.6 kernel I looked at /proc/sys/kernel/random/entropy_avail and saw that it very low (like 5-30, not sure on specifics). Is there something I am not doing right with either the kernel or cyrus? Well, recompiling SASL *should've* taken care of that, but you could also try disabling APOP in /etc/imapd.conf: allowapop: 0 Restart Cyrus after you make the change. If that helps it means that, for whatever reason, Cyrus still uses /dev/random instead of /dev/urandom ... if it doesn't help, something else must be wrong. I have no idea what that might be. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: how to backup a cyrus server?
I have found a link to a process that might be helpful: http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/Backup Aside from this, we have done it the old fashion way: tar zip on a live system late at night. We just had a major server failure on 10/25/2006 which we were able to recover from. Since this worked we most likely will go with this (Knock on wood), but the above is a bit more streamlined (rsync). -Bob From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Rafael Mahecha Sent: Monday, December 04, 2006 4:50 PM To: info-cyrus@lists.andrew.cmu.edu Subject: how to backup a cyrus server? I recently moved all my users from proprietary wemail (using local sendmail) to latest cyrus and sendmail using lmtp. I used to use tivoli to backup the old server (which was ok since no data bases were involved)... but since cyrus has databases and such, I am concern about file-locking and database corruption. What is the best way to back up the server? shutdown cyrus for a while, then snap shot it, and then back up to tivoli or should I just be able to back up the running server directly to tivoli? what other software can I use to backup? Thanks : Rafael Mahecha Email / Systems Administrator JSU OIM - Web Services [EMAIL PROTECTED] http://www.jsums.edu : Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: 2.6 Kernel and POP issues
Well it figures that I recomplied cyrus imap only and NOT sasl. I will recompile that and put in the allowapop statement. Report to follow... -Thanks Bob From: Sebastian Hagedorn [mailto:[EMAIL PROTECTED] Sent: Sat 12/2/2006 12:20 PM To: Robert T. Covell Cc: info-cyrus@lists.andrew.cmu.edu Subject: RE: 2.6 Kernel and POP issues -- Robert T. Covell [EMAIL PROTECTED] is rumored to have mumbled on 1. Dezember 2006 18:50:11 -0600 regarding RE: 2.6 Kernel and POP issues: I have recompiled my cyrus implementation (--with-devrandom=/dev/urandom) and my kernel with no luck. I reboot from the newest 2.4 kernel and my pop clients hang. When checking the 2.6 kernel I looked at /proc/sys/kernel/random/entropy_avail and saw that it very low (like 5-30, not sure on specifics). Is there something I am not doing right with either the kernel or cyrus? Well, recompiling SASL *should've* taken care of that, but you could also try disabling APOP in /etc/imapd.conf: allowapop: 0 Restart Cyrus after you make the change. If that helps it means that, for whatever reason, Cyrus still uses /dev/random instead of /dev/urandom ... if it doesn't help, something else must be wrong. I have no idea what that might be. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: 2.6 Kernel and POP issues
I have recompiled my cyrus implementation (--with-devrandom=/dev/urandom) and my kernel with no luck. I reboot from the newest 2.4 kernel and my pop clients hang. When checking the 2.6 kernel I looked at /proc/sys/kernel/random/entropy_avail and saw that it very low (like 5-30, not sure on specifics). Is there something I am not doing right with either the kernel or cyrus? -Bob -Original Message- From: Sebastian Hagedorn [mailto:[EMAIL PROTECTED] Sent: Tuesday, November 28, 2006 11:31 AM To: Robert T. Covell Cc: info-cyrus@lists.andrew.cmu.edu Subject: RE: 2.6 Kernel and POP issues -- Robert T. Covell [EMAIL PROTECTED] is rumored to have mumbled on 28. November 2006 10:35:40 -0600 regarding RE: 2.6 Kernel and POP issues: Looking over my start up scripts I found that the system is using urandom and not random. That doesn't have to do anything with anything. Should I be using the compile option: --with-egd-socket From the configure explanation I would not think so. No. You should use --with-devrandom=/dev/urandom. -- Sebastian Hagedorn - RZKR-R1 (Flachbau), Zi. 18, Robert-Koch-Str. 10 Zentrum für angewandte Informatik - Universitätsweiter Service RRZK Universität zu Köln / Cologne University - Tel. +49-221-478-5587 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: 2.6 Kernel and POP issues
Looking over my start up scripts I found that the system is using urandom and not random. Should I be using the compile option: --with-egd-socket From the configure explanation I would not think so. If in fact urandom is being used and the entropy pool is working would there be anything else I can look at to gleam light on the issue? Thanks, -Bob rc.S Snippet # Carry an entropy pool between reboots to improve randomness. if [ -f /etc/random-seed ]; then echo Using /etc/random-seed to initialize /dev/urandom. cat /etc/random-seed /dev/urandom fi # Use the pool size from /proc, or 512 bytes: if [ -r /proc/sys/kernel/random/poolsize ]; then dd if=/dev/urandom of=/etc/random-seed count=1 bs=$(cat /proc/sys/kernel/random/poolsize) 2 /dev/null else dd if=/dev/urandom of=/etc/random-seed count=1 bs=512 2 /dev/null fi chmod 600 /etc/random-seed -Original Message- From: Wesley Craig [mailto:[EMAIL PROTECTED] Sent: Monday, November 27, 2006 6:17 PM To: Robert T. Covell Cc: info-cyrus@lists.andrew.cmu.edu Subject: Re: 2.6 Kernel and POP issues On 27 Nov 2006, at 18:20, Robert T. Covell wrote: Other articles I have read reference how to set it up. My setup seems to be similar to what I have seen. From the below article it is possible that I need to fill up the entropy pool. Are their other ways to do this aside from running find /.? Or examples of how to tune it (if possible). Assuming this is the problem, the fix is to use /dev/urandom rather than /dev/random. The difference is that /dev/urandom always returns bytes, whether or not there is sufficient entropy. Another fix would be to run on a machine with a hardware entropy generator. If this turns out to be the issue does anyone know what would have changed from 2.4 to 2.6 (which I know is a lot), but in regards to entropy? Linux 2.4 didn't have a secure /dev/random, at least not be default. 2.6 does by default. :wes Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
2.6 Kernel and POP issues
Hello, On a 2.6 Kernel (linux-2.6.18.1) pop connections stop working completely. People are able to connect once (sometimes) and then the pop3 server will not let them connect again. The connection hangs until timeout. When we switch back to a 2.4 Kernel (linux-2.4.33.3) everything works like a charm. IMAP does not seem to be affected. We told our client base to use this until we determined the problem (couldn't believe it was the kernel). We built the 2.6 kernel from scratch and from the generic smp config provided with the distro (thinking we missed something). Both had the same result. In syslog we are seeing the following error on either a 2.4/2.6 kernel, so I don't think this is an issue but it might be. Nov 26 10:10:23 mail1 master[2665]: setrlimit: Unable to set file descriptors limit to -1: Operation not permitted Nov 26 10:10:23 mail1 master[2665]: retrying with 1024 (current max) Cyrus Version: cyrus-imapd-2.2.12 Distro: Slackware 11 CPU:AMD 64 4600 Dual Core Ram:2GB Drives: 3ware 7506-4LP Raid 5 (320 GB x 4) Would really like to be on a new kernel if at all possible. Any ideas? -Bob Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
RE: 2.6 Kernel and POP issues
Before I posted this I too read other posts on entropy but could not reliably say this was the cause at the time, or find determine how to fix it if it was. The article below gave me some things to look at and try (find /.) which I will test out shortly. Other articles I have read reference how to set it up. My setup seems to be similar to what I have seen. From the below article it is possible that I need to fill up the entropy pool. Are their other ways to do this aside from running find /.? Or examples of how to tune it (if possible). If this turns out to be the issue does anyone know what would have changed from 2.4 to 2.6 (which I know is a lot), but in regards to entropy? Thanks, -Bob -Original Message- From: Wesley Craig [mailto:[EMAIL PROTECTED] Sent: Monday, November 27, 2006 1:15 PM To: Robert T. Covell Subject: Re: 2.6 Kernel and POP issues I would suspect entropy as your problem: http://www.burtonini.com/blog/computers/cups-2006-08-14-18-00 is an interesting reference to the problem. The error below is unrelated, BTW. :wes On 27 Nov 2006, at 10:50, Robert T. Covell wrote: Nov 26 10:10:23 mail1 master[2665]: setrlimit: Unable to set file descriptors limit to -1: Operation not permitted Nov 26 10:10:23 mail1 master[2665]: retrying with 1024 (current max) Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
