Re: limit connections
Hi Shawn, Am Freitag, den 23.12.2016, 16:01 + schrieb Shawn Bakhtiar via Info-cyrus: > Well... you really should take this to the Ubuntu iptables; Start > there and they may be able to give you a better mail list that deals > specifically with iptables: > https://lists.ubuntu.com/mailman/listinfo/ubuntu-users > > > Having said that you don't necessarily need to use > the iptables-extensions --rsource option, you can just use --limit > which is in the standard iptables. This is a debian example but works > for any iptables based system: > https://debian-administration.org/article/187/Using_iptables_to_rate-limit_incoming_connections > > > However, I digress, back to the topic at hand, doing this, is going to > send the users client (based on the client) into a tif. It's going to > assume the server has gone down and warn the user about a lost > connection, I imaging a user who is unwilling to change his pop > interval is going to be even more pissed at having his client pop up > with connection lost messages. > > > IMHO it would be far more professional to first implement a rule > change (update their agreement, or make a corporate policy change) > than inform users not following those policies/agreements that if they > don't comply their accounts will be disabled. This is not only good > corporate governance but it is also non-discriminatory, which means > the offending user is far less likely to be mad, and far more > embarrassed that it had to come to this. > > > Doing what your trying to do, is going to make it look like your > servers are not working. The user may not understand why every so > often the client complains that the connection to the server is not > working, and in fact may result in breach of contract/policy if no > previous policy/agreement has been put into place regarding the issue. > Thanks for time. I definitely agree to your arguments and I'm on not that kind of BOFH, who harms users. But if you ask over weeks to stop that nonsense and the user reduces the interval from 30 seconds to 2 seconds, it's time to play a harder game. And yes, there is an contract, which allows me to disable accounts or block connections, if a user harms, attacks or penetrates the system. Technically the client is an exchange server using popcon to fetch emails. So I guess they will have some errors in log files of their wild running box, if I block new established connections in between. Happy Christmas to all, have a good time and hope to read you next year! Cheers, Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
limit connections
Hi Bron, I have a user, who logs in every 3 seconds(!) to pop3s with 20 accounts, completely resistent to change his pop interval. I'd like to limit him in the way to allow 20 new connections within 5 minutes, then block his IP for 5 minutes (he is using a static IP) and open the port after five minutes again. I tried the following rule, but that opens the port only if the client keeps quiet and doesn't connect while the block is set. Example: iptables -A INPUT -p tcp -m tcp --dport 995 -m state --state NEW -m recent --set --name pop3s --rsource iptables -A INPUT -p tcp -m tcp --dport 995 -m state --state NEW -m recent --rcheck --seconds 300 --hitcount 20 --name pop3s --rsource -j REJECT --reject-with icmp-port-unreachable I know this is off topic and not cyrus specific, but any help would be great. cyrus: 2.4.17 on Ubuntu 14.04 LTS. Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyr_expire on 2.3.16
Hi Ken, Am Samstag, den 10.12.2016, 14:52 + schrieb Ken Smith: > Thank you for your quick reply Marcus > > When I run that (like this for Centos) I get > > su cyrus -c "/usr/lib/cyrus-imapd/cyr_expire -E 1 -X 0 -D 0 -v -p user.kens" > > Expunged 0 out of 0 messages from 0 mailboxes > > I think through the backup anomaly the result is that the messages in > the surplus files have long since been deleted and expunged from a Cyrus > point of view and through the backup mistake the files that Cyrus knows > nothing about any more have been restored. > > Is there a way to resolve that? EG: Extract a list of valid message > files from the database and remove those that are not referred to in there? Hmmm ... looks to me, that these additional mails in your mailboxes are not known by cyrus' database. Have you tried a reconstruct? https://cyrusimap.org/imap/admin/sop/reconstructing.html Ciao Marcus -- , [ Marcus Schopen ] | (0> | //\ | V_/_ D-33602 Bielefeld | ` Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cyr_expire on 2.3.16
Am Samstag, den 10.12.2016, 14:15 + schrieb Ken Smith via Info-cyrus: > Hi All, > > I'm running 2.3.16 cyrus-imap on Centos 6. > > Following a mother board failure on our primary Cyrus server we have > been using a standby server that normally gets a daily backup from the > primary. I now have the primary server running again and I'm > transferring the mailbox data back from the standby. > > Through a backup anomaly there a quite a few more mail files in the > mailbox store than cyrus has in its database. > > The primary server is now fine, and preparing to be live again. If I > check the content of the imap store with mutt all is well on the primary > after updating it from the standby, there's just a whole load of surplus > files in /var/spool/imap/.. that I'd like to delete. > > I believe I need the -D option in cyr_expire but when I run it I get > this help text > > ./cyr_expire > cyr_expire [-C ] -E [-X ] [-p prefix] > [-a] [-v] > > Is the -D option not valid in this version? > > What's the simplest way to tidy the filesystem up? I do this on some mailboxes, if I want to physically remove all mails with deleted flag from file system: su - cyrus -c "/usr/sbin/cyrus expire -E 1 -X 0 -D 0 -v -p user.usermailbox" Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: fetching spam from users junk folder
Hi Nic, Am Dienstag, den 06.12.2016, 11:25 -0600 schrieb Nic Bernstein: > Marcus, > In my original response I forgot to mention that for this, or any > similar approach, to work, you need to use the misnamed "proxyservers" > setting in imapd.conf to grant access to an administrative user. >proxyservers: > A list of users and groups that are allowed to proxy for other > users, separated by spaces. Any user listed in this will be > allowed to login for any other user: use with caution. > For example: > proxyservers: adminuser > Or, alternately, assign rights to the appropriate mailboxes in cyradm: > > sam user.%.Should\ Be\ Spam adminuser all Without using proxyservers I tried setaclmailbox user.testusera.j...@domain.de testus...@domain.de all This works fine to give testuserb full access to testusera's Junk folder. But I run virtual domains where I get this error message setaclmailbox user.testusera.j...@domain.de j...@anotherdomain.de all setaclmailbox: j...@anotherdomain.de: lrswipkxtea: Invalid identifier How do I solve that? I found this in doc/text/install-virtdomains * Domains are mutually exclusive - Users only have access to mailboxes within their own domain (intra-domain). The following example will not work: setacl user.j...@herdomain.com r...@hisdomain.com read. I don't want to work with a "Global administrator" with access to all mailboxes and put that password in a script. Ciao! -- , [ Marcus Schopen ] | (0> | //\ | V_/_ D-33602 Bielefeld | ` Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: fetching spam from users junk folder
Hi Vladki, Am Dienstag, den 06.12.2016, 22:22 +0100 schrieb Vladislav Kurz via Info-cyrus: > Dne Út 6. prosince 2016 08:34:39, Marcus Schopen via Info-cyrus napsal(a): > > Hi, > > > > I'm looking for an easy way to fetch spams, which were moved into a > > special junk subfolder by users in their accounts. I'd like to move > > those messages from there to my account, so I can analyse them to adjust > > anti spam rules. How would you do that? > > > > Ciao! > > I use this to feed spamassassin automatically: > https://packages.debian.org/jessie/sa-learn-cyrus Ah yes, thanks. I've seen that package a while a ago. My problem is, that SA and Cyrus are not running on the same host. Ciao! Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
fetching spam from users junk folder
Hi, I'm looking for an easy way to fetch spams, which were moved into a special junk subfolder by users in their accounts. I'd like to move those messages from there to my account, so I can analyse them to adjust anti spam rules. How would you do that? Ciao! -- "It's great to have a friend who appreciates an earnest discussion of ideas." -Calvin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
expire and remove deleted mailboxes from DELETED.user
Hi, is there a way to immediately remove deleted mailboxes from DELETED.user without calling expire? su - cyrus -c "/usr/sbin/cyrus expire -E 1 -X 0 -D 0 -v -p DELETED.user.testuser" needs about 10 minutes because of "-E 1". Cyrus: 2.4.12 Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cannot unlink .... mboxkey: No such file or directory
Hi, Am Dienstag, den 08.11.2016, 23:56 +1100 schrieb Bron Gondwana via Info-cyrus: > If you're running rolling replication, it SHOULD have logged both the old > mailbox name and the DELETED mailbox name, and calculate the move when > syncing. > > If you're running sync_client -m on just the mailbox, sure - it will get > deleted, because sync_client can't tell what name it was renamed to (there > are plans afoot to make rename detection more clever with some mailboxes.db > changes, but that's still a while away as part of a fully unified murder > environment) > > Anyway, you didn't say which Cyrus version you're using there. If it's pre > 2.4 all bets are off! Pretty sure this was solid in 2.4.0 and above though, > with rolling replication. > > On Tue, 8 Nov 2016, at 23:36, Marcus Schopen via Info-cyrus wrote: > > Hi, > > > > I'm running cyrus in master and slave. When deleting a user's mailbox I > > see the following error on master and slave log: > > > > Nov 8 12:00:44 master cyrus/imap[26104]: cannot > > unlink /var/lib/cyrus/user/t/test20161108.mboxkey: No such file or > > directory > > > > Another strange thing is, that the deleted mailbox is completely removed > > and not moved to DELETED on slave side whereas on master it's moved to > > DELETED directory. > > > > delete_mode and expunge_mode on both sides: > > > > delete_mode: delayed > > expunge_mode: delayed Some more logs. This is what I see on master and slave when deleting an account. Mailboxes are deleted successfully and moved to DELETED on both sides. Anything to be worried about "cannot unlink ..." or "Inbox missing on master for ..."? master: -- Nov 9 16:43:53 master cyrus/imap[21385]: Deleted mailbox user.example-net-02 Nov 9 16:43:53 master cyrus/imap[21385]: Remove of supposedly empty directory /var/spool/cyrus/mail/s/user/example-net-02 failed: Directory not empty Nov 9 16:43:53 master cyrus/imap[21385]: Deleted mailbox user.example-net-02.Drafts Nov 9 16:43:53 master cyrus/imap[21385]: Remove of supposedly empty directory /var/spool/cyrus/mail/s/user/example-net-02 failed: Directory not empty Nov 9 16:43:53 master cyrus/imap[21385]: Deleted mailbox user.example-net-02.Sent Nov 9 16:43:53 master cyrus/imap[21385]: Remove of supposedly empty directory /var/spool/cyrus/mail/s/user/example-net-02 failed: Directory not empty Nov 9 16:43:54 master cyrus/imap[21385]: Deleted mailbox user.example-net-02.Trash Nov 9 16:43:54 master cyrus/imap[21385]: cannot unlink /var/lib/cyrus/user/s/example-net-02.mboxkey: No such file or directory Nov 9 16:43:55 master cyrus/sync_client[5078]: Inbox missing on master for example-net-02 -- slave: -- Nov 9 16:43:54 slave cyrus/syncserver[14647]: Deleted mailbox user.example-net-02 Nov 9 16:43:54 slave cyrus/syncserver[14647]: Remove of supposedly empty directory /var/spool/cyrus/mail/s/user/example-net-02 failed: Directory not empty Nov 9 16:43:55 slave cyrus/syncserver[14647]: Deleted mailbox user.example-net-02.Drafts Nov 9 16:43:55 slave cyrus/syncserver[14647]: Remove of supposedly empty directory /var/spool/cyrus/mail/s/user/example-net-02 failed: Directory not empty Nov 9 16:43:55 slave cyrus/syncserver[14647]: Deleted mailbox user.example-net-02.Sent Nov 9 16:43:55 slave cyrus/syncserver[14647]: Remove of supposedly empty directory /var/spool/cyrus/mail/s/user/example-net-02 failed: Directory not empty Nov 9 16:43:55 slave cyrus/syncserver[14647]: Deleted mailbox user.example-net-02.Trash Nov 9 16:43:55 slave cyrus/syncserver[14647]: cannot unlink /var/lib/cyrus/user/s/example-net-02.mboxkey: No such file or directory -- Cheers! Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: cannot unlink .... mboxkey: No such file or directory
Hi Bron, Thanks for your time! Am Dienstag, den 08.11.2016, 23:56 +1100 schrieb Bron Gondwana via Info-cyrus: > If you're running rolling replication, it SHOULD have logged both the old > mailbox name and the DELETED mailbox name, and calculate the move when > syncing. > > If you're running sync_client -m on just the mailbox, sure - it will get > deleted, because sync_client can't tell what name it was renamed to (there > are plans afoot to make rename detection more clever with some mailboxes.db > changes, but that's still a while away as part of a fully unified murder > environment) > > Anyway, you didn't say which Cyrus version you're using there. If it's pre > 2.4 all bets are off! Pretty sure this was solid in 2.4.0 and above though, > with rolling replication. I've seen this on 2.4.12 (master/slave - Ubuntu 12.04 LTS) and on 2.4.17 (master/slave - Ubuntu 14.04 LTS), both rolling replication. Ciao Marcus -- "All this modern technology just makes people try to do everything at once." -Hobbes Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
cannot unlink .... mboxkey: No such file or directory
Hi, I'm running cyrus in master and slave. When deleting a user's mailbox I see the following error on master and slave log: Nov 8 12:00:44 master cyrus/imap[26104]: cannot unlink /var/lib/cyrus/user/t/test20161108.mboxkey: No such file or directory Another strange thing is, that the deleted mailbox is completely removed and not moved to DELETED on slave side whereas on master it's moved to DELETED directory. delete_mode and expunge_mode on both sides: delete_mode: delayed expunge_mode: delayed Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
limit max child per user or IP
Hi, I'm running cyrus behind a perdition proxy and from time to time one of my users is eating up "maxchild" which is set to 500 (it's just a small server for friends and family) and no further connections are accepted for other users. I checked /var/run/cyrus/proc and mostly "Archives" folders are opened. I talked to the user to check his client settings, but it's not clear why his client behaves like this. Is there a way to limit max connections per user or IP? Perdition itself doesn't support this (only limit on the number of simultaneous connections). Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
backup cyrus spool with bup
Hi, may be a helpful hint for all who are looking for a really fast method to backup the cyrus spool: during the last weeks a I tested bup [1] as an alternativ to my current backup setup based up on rsnaphot and rdiff-backup. Combining bup with lvm snapshots it's a great way to backup big mail storages with millions for files, qcow2 images or even complete partition dumps. Deduplication is working perfectly on cyrus' internal hardlinks or moved/renamed mail folders between backup cycles. The community is very warmhearted and helpful. I've found my definitive backup tools for my mailserver. Cheers, M. [1] https://github.com/bup/bup Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
migrating mails between two cyrus servers keeping seen states
Hi, I'm planning to move some mailboxes from a cyrus server (2.1.18 - which is a kind of mail archive) to another one (2.4.17+caldav~beta9-3). Playing around on a test system with copying a user's mailbox and doing a following "reconstruct -r" came out with a mailbox without seen states on read mails on the new system; all mails unread. Normally I move mailboxes using imapsync to keep states etc., but on a number of really large mailboxes the initial sync takes days. Any other suggestions than (slow) imapsync tools? Ciao! Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: migrating mails between two cyrus servers keeping seen states
Am Freitag, den 18.12.2015, 14:09 +0100 schrieb Michael Menge via Info-cyrus: > Hi, > > > Quoting Marcus Schopen via Info-cyrus <info-cyrus@lists.andrew.cmu.edu>: > > > Hi, > > > > I'm planning to move some mailboxes from a cyrus server (2.1.18 - which > > is a kind of mail archive) to another one (2.4.17+caldav~beta9-3). > > Playing around on a test system with copying a user's mailbox and doing > > a following "reconstruct -r" came out with a mailbox without seen states > > on read mails on the new system; all mails unread. Normally I move > > mailboxes using imapsync to keep states etc., but on a number of really > > large mailboxes the initial sync takes days. Any other suggestions than > > (slow) imapsync tools? > > > Befor 2.4.x the seen information was stored in extra files in /var/imap/user > Did you copy these files too? Ah, it's an old Debian system. These files are stored in eg. /var/lib/cyrus/user/t/testuser.seen. Where do I have to copy this file on the new server to? And there is a testuser.sub in the same dir, which contains the subfolders. What about this file? Ciao! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Replication problem do_folders(): failed to rename
Am Montag, den 14.12.2015, 07:31 -0400 schrieb Patrick Boutilier via Info-cyrus: > On 12/14/2015 06:25 AM, Marcus Schopen via Info-cyrus wrote: > > Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via > > Info-cyrus: > >> Hi, > >> > >> I have a problem with a single mailbox. The user's Outlook crashed and > >> since then the sync_client is running wild on this user account and > >> produces high load on the master. I stopped sync_client on master side > >> for the moment. > >> > >> When I try to sync the user by hand > >> > >> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v > >> -u testuser > >> > >> I do get the following error. > >> > >> Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO > >> response: Operation is not supported on mailbox > >> Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to > >> rename: user.elsa-secgen -> user.testuser.Archives.Gesch > >> 2014-15.25-Jahr-Feier > >> Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote > >> Server(s) denied the operation > >> Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in > >> do_user(testuser): bailing out! > >> > >> Comparing master and slave on filesystem I do see the subfolder > >> "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.", > >> but only on master but not on slave side. And why does sync_client want > >> to rename and where does it get this order from? > >> > >> I can login into the users' mailbox on master side and new message are > >> shown in the INBOX. > >> > >> How can I fix it? > >> > >> Should I try a "reconstruct -r user.testuser" on master and slave or > >> just on slave? (do I have to shutdown cyrus for a reconstruct -r on a > >> user box?) > >> > >> Or can I delete the complete mailbox on slave side start an "sync_client > >> -S replicaserver -v -u testuser"? > >> > >> Thanks for helping > >> Marcus > > > > I did a reconstruct on the replica whichs runs through on the 12 GB > > mailbox ot the user within a second (too fast?). > > > > /bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser" > > > > A following sync ended up with the same error: > > > > /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v > > -u testuser > > > > Any ideas? > > No, but this seems weird. Was this user ever renamed? > > user.testuser -> user.testuser.Archives.Gesch No, sorry, forgot to hide the real user name ;) Ciao! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Replication problem do_folders(): failed to rename
Hi, Am Montag, den 14.12.2015, 12:53 +0100 schrieb Michael Menge via Info-cyrus: > Hi, > > > Quoting Patrick Boutilier via Info-cyrus <info-cyrus@lists.andrew.cmu.edu>: > > > On 12/14/2015 06:25 AM, Marcus Schopen via Info-cyrus wrote: > >> Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via > >> Info-cyrus: > >>> Hi, > >>> > >>> I have a problem with a single mailbox. The user's Outlook crashed and > >>> since then the sync_client is running wild on this user account and > >>> produces high load on the master. I stopped sync_client on master side > >>> for the moment. > >>> > >>> When I try to sync the user by hand > >>> > >>> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v > >>> -u testuser > >>> > >>> I do get the following error. > >>> > >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO > >>> response: Operation is not supported on mailbox > >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to > >>> rename: user.elsa-secgen -> user.testuser.Archives.Gesch > >>> 2014-15.25-Jahr-Feier > >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote > >>> Server(s) denied the operation > >>> Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in > >>> do_user(testuser): bailing out! > >>> > >>> Comparing master and slave on filesystem I do see the subfolder > >>> "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.", > >>> but only on master but not on slave side. And why does sync_client want > >>> to rename and where does it get this order from? > >>> > >>> I can login into the users' mailbox on master side and new message are > >>> shown in the INBOX. > >>> > >>> How can I fix it? > >>> > >>> Should I try a "reconstruct -r user.testuser" on master and slave or > >>> just on slave? (do I have to shutdown cyrus for a reconstruct -r on a > >>> user box?) > >>> > >>> Or can I delete the complete mailbox on slave side start an "sync_client > >>> -S replicaserver -v -u testuser"? > >>> > >>> Thanks for helping > >>> Marcus > >> > >> I did a reconstruct on the replica whichs runs through on the 12 GB > >> mailbox ot the user within a second (too fast?). > >> > >> /bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser" > >> > >> A following sync ended up with the same error: > >> > >> /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v > >> -u testuser > >> > >> Any ideas? > > > > No, but this seems weird. Was this user ever renamed? > > > > user.testuser -> user.testuser.Archives.Gesch > > > > Cyrus uses the folder "Unique ID" for syncronisation. If this "Unique > ID" is NOT unique > it will confuse syncronisation. I connected with an imap client to the mailbox and removed all folders on the master to which sync_client was bailing out. After that sync_client was runnig through without problems. Strange ... Ciao! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
strange difference between master and replica account
Hi, another problem on another account: on the master on the file system there is an IMAP account with about 6000 mails inside its INBOX (~ 5,8 GB account size). Connecting with an IMAP client shows 60 mails and an "unexpunge -l" shows 400 delete mails in the INBOX. On replica side the INBOX there are exactly and only those 60 INBOX mails and exactly the 400 deleted mails, which seems to be correct (~ 2,8 GB account size). sync_client doesn't show any errors and from what I see using an imap client, master and replica are in sync. But why are there about 5500 mails on the master inside the INBOX on the filesystem, so there is a difference of 3 GB between the two INBOXes on filesystem? Shouldn't those mails have been deleted? Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: strange difference between master and replica account
Hi, Am Montag, den 14.12.2015, 16:36 +0100 schrieb venc: > On 2015-12-14 16:16, Marcus Schopen via Info-cyrus wrote: > > > Hi, > > > > another problem on another account: on the master on the file system > > there is an IMAP account with about 6000 mails inside its INBOX (~ 5,8 > > GB account size). Connecting with an IMAP client shows 60 mails and an > > "unexpunge -l" shows 400 delete mails in the INBOX. > > Are you realy sure that you client uses the master server for IMAP > connection? Yes, the imap(s) ports on the slave are closed for public access behind a firewall. Users can't access the replica directly. > Do you run "unexpunge -l" on the replica or on the master? On both sides; an "unexpunge -l" on master and slave shows exactly the same number of deleted files in the INBOX. > Is sync_server and sync_client configured and running on both systems? Yes. I run nagios and monit checks on the /var/lib/cyrus/sync/log file. If it exists and increase above a specific size I get warned. > I had some unexpected sync_client crashes and due to this master and > backup > going "out of sync". Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Replication problem do_folders(): failed to rename
Am Freitag, den 11.12.2015, 19:10 +0100 schrieb Marcus Schopen via Info-cyrus: > Hi, > > I have a problem with a single mailbox. The user's Outlook crashed and > since then the sync_client is running wild on this user account and > produces high load on the master. I stopped sync_client on master side > for the moment. > > When I try to sync the user by hand > > /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v > -u testuser > > I do get the following error. > > Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO > response: Operation is not supported on mailbox > Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to > rename: user.elsa-secgen -> user.testuser.Archives.Gesch > 2014-15.25-Jahr-Feier > Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote > Server(s) denied the operation > Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in > do_user(testuser): bailing out! > > Comparing master and slave on filesystem I do see the subfolder > "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.", > but only on master but not on slave side. And why does sync_client want > to rename and where does it get this order from? > > I can login into the users' mailbox on master side and new message are > shown in the INBOX. > > How can I fix it? > > Should I try a "reconstruct -r user.testuser" on master and slave or > just on slave? (do I have to shutdown cyrus for a reconstruct -r on a > user box?) > > Or can I delete the complete mailbox on slave side start an "sync_client > -S replicaserver -v -u testuser"? > > Thanks for helping > Marcus I did a reconstruct on the replica whichs runs through on the 12 GB mailbox ot the user within a second (too fast?). /bin/su - cyrus -c "/usr/lib/cyrus/bin/reconstruct -r user.testuser" A following sync ended up with the same error: /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v -u testuser Any ideas? Ciao Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: strange difference between master and replica account
Am Montag, den 14.12.2015, 16:16 +0100 schrieb Marcus Schopen via Info-cyrus: > Hi, > > another problem on another account: on the master on the file system > there is an IMAP account with about 6000 mails inside its INBOX (~ 5,8 > GB account size). Connecting with an IMAP client shows 60 mails and an > "unexpunge -l" shows 400 delete mails in the INBOX. > On replica side the INBOX there are exactly and only those 60 INBOX > mails and exactly the 400 deleted mails, which seems to be correct (~ > 2,8 GB account size). sync_client doesn't show any errors and from what > I see using an imap client, master and replica are in sync. But why are > there about 5500 mails on the master inside the INBOX on the filesystem, > so there is a difference of 3 GB between the two INBOXes on filesystem? > Shouldn't those mails have been deleted? I copied the complete mailbox to another cyrus test system and did a "reconstruct -f -r", which cleaned up the INBOX to the 60 visible mails and 400 deleted mails. Question to me is now how or what happend on the live system that the INBOX stored this massive number of odd(?) mail files in the filesystem? Ciao! Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: Replication problem do_folders(): failed to rename
Hi, forgot the cyrus version: 2.4.12 on Ubuntu 12.04 LTS Ciao! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Replication problem do_folders(): failed to rename
Hi, I have a problem with a single mailbox. The user's Outlook crashed and since then the sync_client is running wild on this user account and produces high load on the master. I stopped sync_client on master side for the moment. When I try to sync the user by hand /bin/su - cyrus -c "/usr/lib/cyrus/bin/sync_client -S replicaserver -v -u testuser I do get the following error. Dec 11 17:54:48 master cyrus/sync_client[22727]: RENAME received NO response: Operation is not supported on mailbox Dec 11 17:54:48 master cyrus/sync_client[22727]: do_folders(): failed to rename: user.elsa-secgen -> user.testuser.Archives.Gesch 2014-15.25-Jahr-Feier Dec 11 17:54:48 master cyrus/sync_client[22727]: IOERROR: The remote Server(s) denied the operation Dec 11 17:54:48 master cyrus/sync_client[22727]: Error in do_user(testuser): bailing out! Comparing master and slave on filesystem I do see the subfolder "25-Jahr-Feier" in "user.testuser.Archives.Gesch 2014-15.", but only on master but not on slave side. And why does sync_client want to rename and where does it get this order from? I can login into the users' mailbox on master side and new message are shown in the INBOX. How can I fix it? Should I try a "reconstruct -r user.testuser" on master and slave or just on slave? (do I have to shutdown cyrus for a reconstruct -r on a user box?) Or can I delete the complete mailbox on slave side start an "sync_client -S replicaserver -v -u testuser"? Thanks for helping Marcus Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: delprune on a single mailbox
Am Mittwoch, den 04.11.2015, 06:36 -0500 schrieb Adam Tauno Williams via
Info-cyrus:
> globally in cyrus.conf delprune is set to
> > > > > delprune cmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7"
> > > > > at=0501
> > > > > For a single mailbox I don't want to keep deleted mails for 7
> > > > > days,
> > > > > but
> > > > > expire them immediately or once a day per cron. How to do that?
> > > > Forogt to say that delete_mode and expunge_mode is set to
> > > > delayed.
> > > > Via cron this should work for an immediate cleanup/expire:
> > > You can set an expire annotation per mailbox.
> > How do I do that? From cyr_expire manpage:
> > "The value of the /vendor/cmu/cyrus-imapd/expire annotation is
> > inherited by all children of the given mailbox, so an entire mailbox
> > tree can be expired by seting a single annotation on the root of that
> > tree. If a mailbox does not have a /vendor/cmu/cyrus-imapd/expire
> > annotation set on it (or does not inherit one), then no messages are
> > expired from the mailbox."
>
> Via cyradm -
>
> cyrus.example.com> mboxcfg user.adam expire 365
> cyrus.example.com> info user.adam
> {user.adam}:
> condstore: false
> duplicatedeliver: false
> expire: 365
> lastpop:
> lastupdate: 13-Aug-2008 19:37:31 -0400
> partition: default
> sharedseen: false
> size: 12325671
>
> AFAIK the annotations supported by cyradm/mboxcfg are:
>
> * comment – A free-form text comment or description to be attached to
> the mailbox.
> * condstore – This annotation is only supported in the 2.3.x release
> series starting with 2.3.3 although its use is not recommended until
> 2.3.8. As of the 2.4.x release series CONDSTORE functionality is
> enabled on all mailboxes regardless of annotation and attempting to set
> this annotation will result in a permission denied message. On releases
> where this annotation is supported setting a value of “true” will
> enable CONDSTORE functionality1.
> * expire – If an expire value is provided messages will be
> automatically deleted from the mailbox once the specified number of
> days has elapsed.
> * news2mail -
> * sharedseen - Enables the use of a shared \Seen flag on messages
> rather than a per-user \Seen flag. The 's' right in the mailbox ACL
> still controls whether a user can set the shared \Seen flag.
> * sieve – In the case of a shared folder the “sieve” parameter
> specifies the name of a global SIEVE script that will be used for every
> message delivered to the folder. This value is ignored for personal
> mailboxes (mailboxes including and subordinate to a user's INBOX).
> * squat – Flags the mailbox to be included for indexing when the SQUAT
> process performs index generation.
>
>
> > But is it possible to expunge a message immediately when it's deleted
> > by client and not with the next expire run?
>
> Not if delayed expunge is enabled AFAIK; that would defeat the purpose.
I set "mboxcfg user.test expire 1" on a test mailbox, but it has no
effect on nightly delprune set in cyrus.conf EVENT:
delprune cmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7" at=0501"
Messages deleted two days ago are still in the file system.
localhost> info user.test
{user.test}:
duplicatedeliver: false
expire: 1
lastpop:
lastupdate: 4-Nov-2015 17:14:20 +0100
partition: default
pop3newuidl: true
sharedseen: false
size: 0
Any ideas?
Ciao!
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: delprune on a single mailbox
Am Dienstag, den 03.11.2015, 08:24 +0100 schrieb Marcus Schopen via
Info-cyrus:
> Hi,
>
> Am Montag, den 02.11.2015, 10:58 -0500 schrieb Adam Tauno Williams via
> Info-cyrus:
> > On Sun, 2015-11-01 at 14:40 +0100, Marcus Schopen via Info-cyrus wrote:
> > > Am Sonntag, den 01.11.2015, 13:35 +0100 schrieb Marcus Schopen via
> > > Info-cyrus:
> > > > Hi,
> > > > globally in cyrus.conf delprune is set to
> > > > delprunecmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7" at=0501
> > > > For a single mailbox I don't want to keep deleted mails for 7 days,
> > > > but
> > > > expire them immediately or once a day per cron. How to do that?
> > > Forogt to say that delete_mode and expunge_mode is set to delayed.
> > > Via cron this should work for an immediate cleanup/expire:
> >
> > You can set an expire annotation per mailbox.
>
> How do I do that? From cyr_expire manpage:
>
> "The value of the /vendor/cmu/cyrus-imapd/expire annotation is inherited
> by all children of the given mailbox, so an entire mailbox tree can be
> expired by seting a single annotation on the root of that tree. If a
> mailbox does not have a /vendor/cmu/cyrus-imapd/expire annotation set on
> it (or does not inherit one), then no messages are expired from the
> mailbox."
>
> Is this correct?
>
> setannotation "user.myuser"
> "/vendor/cmu/cyrus-imapd/expire" ("value.shared" "0")
>
> But is it possible to expunge a message immediately when it's deleted by
> client and not with the next expire run?
Sorry, but could someon explain how and where to set annotation?
cyradmin seems not to be the right tool.
Ciao!
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: delprune on a single mailbox
Hi,
Am Montag, den 02.11.2015, 10:58 -0500 schrieb Adam Tauno Williams via
Info-cyrus:
> On Sun, 2015-11-01 at 14:40 +0100, Marcus Schopen via Info-cyrus wrote:
> > Am Sonntag, den 01.11.2015, 13:35 +0100 schrieb Marcus Schopen via
> > Info-cyrus:
> > > Hi,
> > > globally in cyrus.conf delprune is set to
> > > delprune cmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7" at=0501
> > > For a single mailbox I don't want to keep deleted mails for 7 days,
> > > but
> > > expire them immediately or once a day per cron. How to do that?
> > Forogt to say that delete_mode and expunge_mode is set to delayed.
> > Via cron this should work for an immediate cleanup/expire:
>
> You can set an expire annotation per mailbox.
How do I do that? From cyr_expire manpage:
"The value of the /vendor/cmu/cyrus-imapd/expire annotation is inherited
by all children of the given mailbox, so an entire mailbox tree can be
expired by seting a single annotation on the root of that tree. If a
mailbox does not have a /vendor/cmu/cyrus-imapd/expire annotation set on
it (or does not inherit one), then no messages are expired from the
mailbox."
Is this correct?
setannotation "user.myuser"
"/vendor/cmu/cyrus-imapd/expire" ("value.shared" "0")
But is it possible to expunge a message immediately when it's deleted by
client and not with the next expire run?
> Downside is that I
> believe the annotation will be 'inherited' but subordinate mailboxes;
> which stinks for some use-cases.
>
>
> > su - cyrus -c "/usr/sbin/cyrus expire -E 1 -X 0 -D 0 -v -p
> > user.mailboxname"
>
> FYI, I believe with the very latest Cyrus the "su -" is unnecessary as
> it will automatically handle the context change when run as root.
Ciao!
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
delprune on a single mailbox
Hi, globally in cyrus.conf delprune is set to delprunecmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7" at=0501 For a single mailbox I don't want to keep deleted mails for 7 days, but expire them immediately or once a day per cron. How to do that? Ciao! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
Re: delprune on a single mailbox
Am Sonntag, den 01.11.2015, 13:35 +0100 schrieb Marcus Schopen via Info-cyrus: > Hi, > > globally in cyrus.conf delprune is set to > > delprune cmd="/usr/sbin/cyrus expire -E 1 -X 7 -D 7" at=0501 > > For a single mailbox I don't want to keep deleted mails for 7 days, but > expire them immediately or once a day per cron. How to do that? Forogt to say that delete_mode and expunge_mode is set to delayed. Via cron this should work for an immediate cleanup/expire: su - cyrus -c "/usr/sbin/cyrus expire -E 1 -X 0 -D 0 -v -p user.mailboxname" su - cyrus -c "/usr/sbin/cyrus expire -E 1 -X 0 -D 0 -v -p DELETED.user.mailboxname" Is it possible to set it by ACL on a single mailbox? Ciao! Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
