Re: Replication sieve scripts.
Op 7/19/2016 om 11:30 AM schreef Luescher Claude: > Hello, > > Following up on old thread: > > http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html > > I have the exact same issue with 2.2.10: > > ii dovecot-antispam 2.0+20130822-2 > amd64 Dovecot plugins for training spam filters > ii dovecot-core 1:2.2.10-1 > amd64 secure POP3/IMAP server - core files > ii dovecot-imapd 1:2.2.10-1 > amd64 secure POP3/IMAP server - IMAP daemon > ii dovecot-ldap 1:2.2.10-1 > amd64 secure POP3/IMAP server - LDAP support > ii dovecot-lmtpd 1:2.2.10-1 > amd64 secure POP3/IMAP server - LMTP server > ii dovecot-managesieved 1:2.2.10-1 > amd64 secure POP3/IMAP server - ManageSieve server > ii dovecot-mysql 1:2.2.10-1 > amd64 secure POP3/IMAP server - MySQL support > ii dovecot-pop3d 1:2.2.10-1 > amd64 secure POP3/IMAP server - POP3 daemon > ii dovecot-sieve 1:2.2.10-1 > amd64 secure POP3/IMAP server - Sieve filters support > > As even his version should already support sieve replication through > dsync mine would have to support it for sure but I would like to hear > a confirmation from the developers. > > The mail sync just works perfectly between the 2 nodes, it is only the > sieve scripts which don't get replicated. > The following bugs were fixed recently: https://github.com/dovecot/core/commit/b4adb461ce12bf578d2d70806b205cf3cbf1a51d https://github.com/dovecot/core/commit/27ccbb0f36e07141785db94557afb63a2aa9eeba I wonder whether this also applies to your problem. Regards, Stephan.
Re: Replication sieve scripts.
Hello, So basically this sieve replication support is still in some broken alfa stage and one should not build production system on top of it. I have found this older thread: http://www.dovecot.org/list/dovecot/2013-February/088640.html We came up with a few options for syncing: 1) rsync/unison ~/sieve every x seconds 2) move sieve dirs to separatate partition and sync with gluster or drbd 3) patch roundcube managesieve sieve plugin to upload to both managesieve servers (only roundcube can access managesieve in our setup) 4) store sieve scripts in mysql and patch roundcube managesieve plugin to directly access the db instead of using managesieve (we tried this, but apparently sieve scripts read from db via dovecot dict can't have more than one line? show stopper due to roundcube sieve plugin needing comments to parse the scripts) The database solution would be perfect since the nodes are using a db master-master cluster but at the time of that post this was not working. My original 2.2.10 dovecot is around that time so I would have to upgrade for that again and at the end it would might not work because as he said roundcube using multi-line sieve scripts. The problem with rsync is that rsync from where? If I rsync from machine A -> B and let's say the user already have sieves on both nodes and changes his holiday sieve on node B then the file will be overwritten. I will give unison a try otherwise I have to put the sieves on a shared filesystem like OCFS2 which imo is not reliable for production either. When nodes lose connectivity that can end in kernel crash, autoreboot on 1 node or both. On Sat, 23 Jul 2016 18:17:58 +0200, Michael Grimm wrote: Stephan Bosch wrote: Only the user's scripts in the ~/sieve directory and the active script marked by ~/.dovecot.sieve (symlink) are replicated. I do have those files in those directories: KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve KB> ls -al default -rw-r--r-- 1 root wheel 5909 Jul 21 19:39 default KB> doveadm sieve put -u mike -a default < default I do see traffic arriving at MW when listening on the defined port for synchronisation (tcpdump), but the new default file at KB doesn't become synchronized: MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve If I do remove that file at MW and repeat "doveadm sieve put" at KB, I do see synchronization succeed: MW> rm /var/mail/.homedirs/mike/.sieves/default.sieve MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve ls: /var/mail/.homedirs/mike/.sieves/default.sieve: No such file or directory lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve KB> doveadm sieve put -u mike -a default < default KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5909 Jul 21 19:40 /var/mail/.homedirs/mike/.sieves/default.sieve MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5909 Jan 1 1970 /var/mail/.homedirs/mike/.sieves/default.sieve The date of the file puzzles me, though. JFTR: This is a recent FreeBSD and ZFS. JFTR: The logfiles do show the very same messages as Claud has shown in his last mail. Regards, Michael Links: -- [1] mailto:step...@rename-it.nl
Re: Replication sieve scripts.
Stephan Boschwrote: > Only the user's scripts in the ~/sieve directory and the active script marked > by ~/.dovecot.sieve (symlink) are replicated. I do have those files in those directories: KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve KB> ls -al default -rw-r--r-- 1 root wheel 5909 Jul 21 19:39 default KB> doveadm sieve put -u mike -a default < default I do see traffic arriving at MW when listening on the defined port for synchronisation (tcpdump), but the new default file at KB doesn't become synchronized: MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5898 Jul 21 19:38 /var/mail/.homedirs/mike/.sieves/default.sieve If I do remove that file at MW and repeat "doveadm sieve put" at KB, I do see synchronization succeed: MW> rm /var/mail/.homedirs/mike/.sieves/default.sieve MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve ls: /var/mail/.homedirs/mike/.sieves/default.sieve: No such file or directory lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:38 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve KB> doveadm sieve put -u mike -a default < default KB> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5909 Jul 21 19:40 /var/mail/.homedirs/mike/.sieves/default.sieve MW> la /var/mail/.homedirs/mike/.dovecot.sieve /var/mail/.homedirs/mike/.sieves/default.sieve lrwx-- 1 vmail dovecot uarch 21 Jul 21 19:40 /var/mail/.homedirs/mike/.dovecot.sieve -> .sieves/default.sieve -rw--- 1 vmail dovecot uarch 5909 Jan 1 1970 /var/mail/.homedirs/mike/.sieves/default.sieve The date of the file puzzles me, though. JFTR: This is a recent FreeBSD and ZFS. JFTR: The logfiles do show the very same messages as Claud has shown in his last mail. Regards, Michael
Re: Replication sieve scripts.
Hello, Ok so I have set everything to ~/sieve dir on the 2 nodes: doveconf -n | grep siev managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave editheader sieve = ~/.dovecot.sieve sieve_after = ~/sieve sieve_before = ~/sieve sieve_dir = ~/sieve sieve_extensions = +editheader sieve_max_redirects = 20 protocols = " imap lmtp sieve pop3" service managesieve-login { inet_listener sieve { mail_plugins = " quota notify replication sieve" mail_plugins = " quota notify replication sieve" This way after I add a brand new user the sieve dir will be replicated but nothing after that. I have added a new user on node2 then enabled out of office message, the sieve files were created under the: /home/vmail/company.com/us...@company.com/sieve -rw--- 1 vmail vmail 1011 Jul 22 09:49 01_vacation.sieve -rw--- 1 vmail vmail 644 Jul 22 09:49 01_vacation.svbin directory but were never replicated not even after restarting both dove nodes. Node2 Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/us...@company.com//.dovecot.sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/us...@company.com//sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: permission lookup failed from /home/vmail/company.com/us...@company.com//sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: created storage directory /home/vmail/company.com/us...@company.com//sieve/tmp Jul 22 09:47:54 mailnode2 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/us...@company.com//.dovecot.sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/us...@company.com//sieve Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using permissions from /home/vmail/company.com/us...@company.com//sieve: mode=0700 gid=-1 Jul 22 09:47:54 mailnode2 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Node1 Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/us...@company.com//.dovecot.sieve Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/us...@company.com//sieve Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: permission lookup failed from /home/vmail/company.com/us...@company.com//sieve Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: created storage directory /home/vmail/company.com/us...@company.com//sieve/tmp Jul 22 09:50:16 mailnode1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 22 09:50:16 mailnode1 dovecot: dsync-local(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 22 09:50:16
Re: Replication sieve scripts.
Op 21-7-2016 om 15:50 schreef Luescher Claude: Hello, Thanks for the advice. I have looked for the libs and here is the difference: [...] First problem I notice is this /home/vmail/company.com/us...@company.com//.dovecot.sieve missing. Is this supposed to be a file or a directory? The second sieve_global_dir, I don't need global rules. The third is this minus gid=-1 value. The user/group/rights on that directory is not different from the mail dirs (where the replication works): ls -al /home/vmail/company.com/us...@company.com//sieve drwx--S--- 2 vmail vmail 4096 Jul 21 15:17 tmp The important directory for me is the siev_after because roundcube vacation plugin puts the sieves in there for example: -rw--- 1 vmail vmail 413 Jul 21 15:23 01_vacation.sieve -rw--- 1 vmail vmail 124 Jul 21 15:23 01_vacation.svbin -rw-r--r-- 1 vmail vmail 193 Feb 26 2014 antispam.sieve -rw-r--r-- 1 vmail vmail 272 Apr 13 2015 antispam.svbin Gets created after setting out of office message but will not get replicated to the other node. As I have showed in my earlier doveconf output is defined: sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_duplicate_period = 1h sieve_extensions = +editheader sieve_max_redirects = 20 And the sieve scripts work locally per server they are just not replicating as they should be. There is your problem. The sieve_after/sieve_before settings usually point to administrator-controlled global scripts. Therefore, these are not replicated by the doveadm_sieve plugin. Only the user's scripts in the ~/sieve directory and the active script marked by ~/.dovecot.sieve (symlink) are replicated. Regards, Stephan.
Re: Replication sieve scripts.
Hello, Thanks for the advice. I have looked for the libs and here is the difference: Dovecot production env 2.2.10: /usr/lib/dovecot/modules/doveadm rw-r--r-- 1 root root 18560 Jan 9 2014 lib10_doveadm_acl_plugin.so -rw-r--r-- 1 root root 14256 Jan 9 2014 lib10_doveadm_expire_plugin.so -rw-r--r-- 1 root root 10232 Jan 9 2014 lib10_doveadm_quota_plugin.so -rw-r--r-- 1 root root 51352 Jan 9 2014 lib10_doveadm_sieve_plugin.so -rw-r--r-- 1 root root 14344 Jan 9 2014 lib20_doveadm_fts_plugin.so Dovecot new test env: /usr/dovecot/lib/dovecot/doveadm -rw-r--r-- 1 root root 140574 Jul 20 12:53 lib10_doveadm_acl_plugin.a -rwxr-xr-x 1 root root 1066 Jul 20 12:53 lib10_doveadm_acl_plugin.la -rwxr-xr-x 1 root root 98910 Jul 20 12:53 lib10_doveadm_acl_plugin.so -rw-r--r-- 1 root root 56108 Jul 20 12:53 lib10_doveadm_expire_plugin.a -rwxr-xr-x 1 root root 1087 Jul 20 12:53 lib10_doveadm_expire_plugin.la -rwxr-xr-x 1 root root 43879 Jul 20 12:53 lib10_doveadm_expire_plugin.so -rw-r--r-- 1 root root 97212 Jul 20 12:53 lib10_doveadm_quota_plugin.a -rwxr-xr-x 1 root root 1080 Jul 20 12:53 lib10_doveadm_quota_plugin.la -rwxr-xr-x 1 root root 63227 Jul 20 12:53 lib10_doveadm_quota_plugin.so -rw-r--r-- 1 root root 338560 Jul 20 12:53 lib10_doveadm_sieve_plugin.a -rwxr-xr-x 1 root root 1286 Jul 20 12:53 lib10_doveadm_sieve_plugin.la -rwxr-xr-x 1 root root 177604 Jul 20 12:53 lib10_doveadm_sieve_plugin.so -rw-r--r-- 1 root root 139748 Jul 20 12:53 lib20_doveadm_fts_plugin.a -rwxr-xr-x 1 root root 1066 Jul 20 12:53 lib20_doveadm_fts_plugin.la -rwxr-xr-x 1 root root 94615 Jul 20 12:53 lib20_doveadm_fts_plugin.so Also I have set the mail_log to debug as you have suggested and I see the following for users in the logs (with my "old" version): Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/us...@company.com//.dovecot.sieve Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/us...@company.com//sieve Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: using permissions from /home/vmail/company.com/us...@company.com//sieve: mode=0700 gid=-1 Jul 21 11:52:27 server1 dovecot: dsync-server(us...@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/us...@company.com//.dovecot.sieve Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: using sieve script storage directory: /home/vmail/company.com/us...@company.com//sieve Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: using permissions from /home/vmail/company.com/us...@company.com//sieve: mode=0700 gid=-1 Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: relative path to sieve storage in active link: sieve/ Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: doveadm-sieve: Iterating Sieve mailbox attributes Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve: Pigeonhole version 0.4.2 initializing Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 21 15:17:21 server1 dovecot: dsync-local(us...@company.com): Debug: sieve-storage: using active sieve script path: /home/vmail/company.com/us...@company.com//.dovecot.sieve First problem I notice is this /home/vmail/company.com/us...@company.com//.dovecot.sieve missing. Is this supposed to be a file or a directory? The second sieve_global_dir, I don't need global rules. The third is this minus gid=-1 value. The user/group/rights on that directory is not different from the mail dirs (where the replication works): ls -al
Re: Replication sieve scripts.
Op 20-7-2016 om 14:51 schreef Luescher Claude: I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot: /usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766) /usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql and dovecot-2.2-pigeonhole-0.4.15. It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere. Otherwise I will have to look for other solutions like lsync, rsync to sync them between the nodes. Is the doveadm sieve plugin properly installed at both sides of the replication? On my system (Debian Jessie), it is located at /usr/lib/dovecot/modules/doveadm. The file name is lib10_doveadm_sieve_plugin.so. When mail_debug is enabled, the replication should log all kinds of doveadm-sieve debug messages. Any errors in the log? Regards, Stephan.
Re: Replication sieve scripts.
On 2016-07-20 14:51, Luescher Claude wrote: I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot: /usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766) /usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql and dovecot-2.2-pigeonhole-0.4.15. It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere. I did report a year ago [1] that I did observe mails being synced but no sieves. Although I did report at the end that replication started again, I do have to report today that it stopped again shortly afterwards (even on current version of dovecot and pigeonhole). The only way to get sieves synced in my case is: removing the corresponding files at one server, update the sieve files at the other server [2]. Only then, sieves become synced. On very rare occasions syncing happens without applying the first step just mentioned. This issue looks to me a very subtle one I am unable to reproduce reliably, sorry. HTH, Michael [1] http://dovecot.org/pipermail/dovecot/2015-May/100839.html [2] I can live with that workaround because I am the only user having sieves activated.
Re: Replication sieve scripts.
I did eventually was forced to do this and setup a 2 node test environment with the latest dovecot: /usr/dovecot/sbin/dovecot --version 2.2.25 (7be1766) /usr/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql and dovecot-2.2-pigeonhole-0.4.15. It unfortunately made no difference at all. The emails are still replicating while it is completely ignoring the sieve files so there must be a switch in the config for this somewhere. Otherwise I will have to look for other solutions like lsync, rsync to sync them between the nodes. Here is my complete doveconf -n output: # 2.2.25 (7be1766): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_cache_size = 1 M auth_mechanisms = plain login cram-md5 default_vsz_limit = 512 M dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = # hidden, use -P to show it doveadm_port = login_greeting = mail.company.com is ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_gid = vmail mail_location = maildir:~/Maildir mail_plugins = " quota notify replication" mail_uid = vmail maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext editheader namespace cyruscompat1 { alias_for = hidden = yes list = no location = prefix = INBOX/ separator = / type = private } namespace inbox { hidden = no inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = spool2dir antispam_spam = Spam;SPAM antispam_spool2dir_notspam = /home/vmail/spam_training/good/new/%%020lu-%u-%%05lu antispam_spool2dir_spam = /home/vmail/spam_training/bad/new/%%020lu-%u-%%05lu antispam_trash_pattern_ignorecase = trash;Deleted *;éléments supprimés;gelöschte *; supprim mail_replica = tcp:1.2.3.4 quota = dict:User quota::proxy::quota quota_grace = 10M quota_rule = *:storage=100M quota_rule2 = Trash:storage=+10%% quota_rule3 = Deleted Messages:storage=+10%% quota_status_nouser = DUNNO quota_status_success = DUNNO quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u replication_full_sync_interval = 1 hours sieve = ~/.dovecot.sieve sieve_after = ~/sieve_after sieve_dir = ~/sieve sieve_duplicate_period = 1h sieve_extensions = +editheader sieve_max_redirects = 20 } pop3c_host = pop3.e2advance.net postmaster_address = postmas...@company.com protocols = imap pop3 lmtp imap lmtp sieve pop3 service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } user = $default_internal_user } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = } } service imap-login { process_limit = 256 process_min_avail = 5 } service lmtp { unix_listener /var/spool/postfix/public/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 10020 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_ca = On 2016.07.19. 15:03, Luescher Claude wrote: Hello, Are you sure about that I would need so recent version? That
Re: Replication sieve scripts.
On 2016.07.19. 15:03, Luescher Claude wrote: Hello, Are you sure about that I would need so recent version? That is ok to have some bugfixes but you are talking about the latest cutting edge debian jessie backports package. Even the mainline debian jessie only have 2.2.13 in it and my system is a complex wheezy installation, the last thing I want to do is to move all my components into jessie just because of this or do you know 2.2.15 deb packages built for wheezy? My installation is a multi-node cluster already I would have to update all the nodes but if it would be only to install a couple of new debs on them I would not hesitate trying a newer dovecot version. Thank you. If you have a huge multi node system, then you need to have testing system also. It's easy to compile Dovecot by yourself and make deb packages, if needed. -- KSB
Re: Replication sieve scripts.
Hello, Are you sure about that I would need so recent version? That is ok to have some bugfixes but you are talking about the latest cutting edge debian jessie backports package. Even the mainline debian jessie only have 2.2.13 in it and my system is a complex wheezy installation, the last thing I want to do is to move all my components into jessie just because of this or do you know 2.2.15 deb packages built for wheezy? My installation is a multi-node cluster already I would have to update all the nodes but if it would be only to install a couple of new debs on them I would not hesitate trying a newer dovecot version. Thank you. On Tue, 19 Jul 2016 11:54:27 +0200, Stephan Bosch wrote: Op 19-7-2016 om 11:30 schreef Luescher Claude: Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html [1] I have the exact same issue with 2.2.10: It did not help. If there is one way I would like to get this work without upgrading dovecot. Many bugs were fixed since this version. That is likely Pigeonhole 0.4.2, which is very old (26-09-2013). The synchronization didn't work flawlessly until at least version 0.4.4, which goes with Dovecot 2.2.15. Regards, Stephan. Links: -- [1] http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html
Re: Replication sieve scripts.
Op 19-7-2016 om 11:30 schreef Luescher Claude: Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html I have the exact same issue with 2.2.10: It did not help. If there is one way I would like to get this work without upgrading dovecot. Many bugs were fixed since this version. That is likely Pigeonhole 0.4.2, which is very old (26-09-2013). The synchronization didn't work flawlessly until at least version 0.4.4, which goes with Dovecot 2.2.15. Regards, Stephan.
Replication sieve scripts.
Hello, Following up on old thread: http://www.dovecot.org/pipermail/dovecot/2014-December/099003.html I have the exact same issue with 2.2.10: ii dovecot-antispam 2.0+20130822-2amd64 Dovecot plugins for training spam filters ii dovecot-core 1:2.2.10-1amd64 secure POP3/IMAP server - core files ii dovecot-imapd 1:2.2.10-1amd64 secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 1:2.2.10-1amd64 secure POP3/IMAP server - LDAP support ii dovecot-lmtpd 1:2.2.10-1amd64 secure POP3/IMAP server - LMTP server ii dovecot-managesieved 1:2.2.10-1amd64 secure POP3/IMAP server - ManageSieve server ii dovecot-mysql 1:2.2.10-1amd64 secure POP3/IMAP server - MySQL support ii dovecot-pop3d 1:2.2.10-1amd64 secure POP3/IMAP server - POP3 daemon ii dovecot-sieve 1:2.2.10-1amd64 secure POP3/IMAP server - Sieve filters support As even his version should already support sieve replication through dsync mine would have to support it for sure but I would like to hear a confirmation from the developers. The mail sync just works perfectly between the 2 nodes, it is only the sieve scripts which don't get replicated. I have tried to add additional sieve related options to the replicator: service replicator { process_min_avail = 1 } plugin { mail_replica = tcp:1.2.3.4 replication_full_sync_interval = 1 hours #Added (makes no difference) sieve = ~/.dovecot.sieve sieve_dir = ~/sieve_after sieve_extensions = +vnd.dovecot.duplicate sieve_duplicate_period = 1h #Added (makes no difference) } service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } It did not help. If there is one way I would like to get this work without upgrading dovecot. Thank you.
Re: Replication sieve scripts.
Hi Am 15.12.2014 um 14:00 schrieb Hanns Mattes: After the following upgrade, the sieve-scripts are replicated - very good. I stand corrected. They are sometimes updated, sometimes not. I didn't have time for testing (and I don't see a system), but it's not reliably. I will report after some testing. Regards Hanns
Re: Replication sieve scripts.
Hi, Robert Schetterer r...@sys4.de schrieb: [...] so suse people may upgrade to sieve pigeonhole latest, contact them via mail etc [...] I sent a mail to one of the Maintainers (darix) and they updated only an hour later - very nice guy. After the following upgrade, the sieve-scripts are replicated - very good. Best regards Hanns Mattes
Re: Replication sieve scripts.
Am 15.12.2014 um 14:00 schrieb Hanns Mattes: Hi, Robert Schetterer r...@sys4.de schrieb: [...] so suse people may upgrade to sieve pigeonhole latest, contact them via mail etc [...] I sent a mail to one of the Maintainers (darix) and they updated only an hour later - very nice guy. After the following upgrade, the sieve-scripts are replicated - very good. Best regards Hanns Mattes cool ! Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: Replication sieve scripts.
I sent a mail to one of the Maintainers (darix) and they updated only an hour later - very nice guy. After the following upgrade, the sieve-scripts are replicated - very good. Best regards Hanns Mattes Wow, very quick ! I love those kind of guys. Regards, -- CHUNKZ.NET - script kiddie and computer technician Bertrand Caplet, Flers (FR) Feel free to send encrypted/signed messages Key ID: FF395BD9 GPG FP: DE10 73FD 17EB 5544 A491 B385 1EDA 35DC FF39 5BD9 signature.asc Description: OpenPGP digital signature
Re: Replication sieve scripts.
Hi, Bertrand Caplet bertrand.cap...@chunkz.net schrieb: [...] Maybe you could at least test to build it from source the last stable version ? I'll give it a try. I'm using the RPMs provided by https://build.opensuse.org/package/show/server:mail/dovecot22 and they are shipping dovecot with pigeonhole 0.4.3. I'm trying to build with 0.4.6 in Opensuse build service with no luck so far which is definitely caused by my complete lack of knowledge with building RPMs, modifying *.spec-files and the OBS. But this doesn't belong to this list (though I appreciate any hints and help). Regards Hanns
Re: Replication sieve scripts.
Am 14.12.2014 um 18:14 schrieb Hanns Mattes: Hi, Bertrand Caplet bertrand.cap...@chunkz.net schrieb: [...] Maybe you could at least test to build it from source the last stable version ? I'll give it a try. I'm using the RPMs provided by https://build.opensuse.org/package/show/server:mail/dovecot22 and they are shipping dovecot with pigeonhole 0.4.3. I'm trying to build with 0.4.6 in Opensuse build service with no luck so far which is definitely caused by my complete lack of knowledge with building RPMs, modifying *.spec-files and the OBS. But this doesn't belong to this list (though I appreciate any hints and help). Regards Hanns be sure to use latest revision https://build.opensuse.org/package/view_file/openSUSE:Factory/dovecot22/dovecot22.changes https://build.opensuse.org/package/show/server:mail/dovecot22?rev=49 dovecot-2.2-pigeonhole-0.4.3_adapt_to_2.2.15.patch latest dove sieve is http://pigeonhole.dovecot.org/download.html v0.4.6 so suse people may upgrade to sieve pigeonhole latest, contact them via mail etc dovecot itself seems up2date with latest revision Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: Replication sieve scripts.
On 12/4/2014 12:03 AM, Jan Wideł wrote: Hi, according to changelog 2.2.rc3, dsync should replicate sieve scripts. Do I need turn on or switch some option(s), for this to work? Replication of mailboxes works great, only sieve scripts not. What version of Pigeonhole is this? It doesn't look like the latest, since it doesn't include the version in the doveconf banner. Regards, Stephan.
Re: Replication sieve scripts.
What version of Pigeonhole is this? It doesn't look like the latest, since it doesn't include the version in the doveconf banner. Regards, Stephan. Yes, I think it could be that. I have dovecot with replication and without any configuration unless doveadm replication, it's working perfectly for sieve rules. Regards, -- CHUNKZ.NET - script kiddie and computer technician Bertrand Caplet, Flers (FR) Feel free to send encrypted/signed messages Key ID: FF395BD9 GPG FP: DE10 73FD 17EB 5544 A491 B385 1EDA 35DC FF39 5BD9 signature.asc Description: OpenPGP digital signature
Re: Replication sieve scripts.
Hi, Am 13.12.2014 um 22:26 schrieb Stephan Bosch: On 12/4/2014 12:03 AM, Jan Wideł wrote: Hi, according to changelog 2.2.rc3, dsync should replicate sieve scripts. Do I need turn on or switch some option(s), for this to work? Replication of mailboxes works great, only sieve scripts not. What version of Pigeonhole is this? It doesn't look like the latest, since it doesn't include the version in the doveconf banner. (Sorry for opening a second thread - I didn't notice Jan's original post to the same topic) I'm using the RPMs provided by https://build.opensuse.org/package/show/server:mail/dovecot22 Not being an RPM-expert, they seem to use an older pigeonhole-version. Regards Hanns
Re: Replication sieve scripts.
(Sorry for opening a second thread - I didn't notice Jan's original post to the same topic) Well that's not exactly the same problem so don't worry Not being an RPM-expert, they seem to use an older pigeonhole-version. Maybe you could at least test to build it from source the last stable version ? Regards, -- CHUNKZ.NET - script kiddie and computer technician Bertrand Caplet, Flers (FR) Feel free to send encrypted/signed messages Key ID: FF395BD9 GPG FP: DE10 73FD 17EB 5544 A491 B385 1EDA 35DC FF39 5BD9 signature.asc Description: OpenPGP digital signature
Re: Replication sieve scripts.
On 12/13/2014 10:26 PM, Stephan Bosch wrote: On 12/4/2014 12:03 AM, Jan Wideł wrote: Hi, according to changelog 2.2.rc3, dsync should replicate sieve scripts. Do I need turn on or switch some option(s), for this to work? Replication of mailboxes works great, only sieve scripts not. What version of Pigeonhole is this? It doesn't look like the latest, since it doesn't include the version in the doveconf banner. Truly, I don't remember. I was thinking about sync this files by software like https://www.csync.org/, but after while (and some updates) replication sieve starts working... magically. It bugs me why, but I have no idea. Debs are from: deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.2 main My current config is: # 2.2.15 (6dd190bd6dcb): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 # OS: Linux 3.2.0-4-grsec-amd64 x86_64 Debian 7.7 ext4 doveadm_password = x doveadm_port = 10900 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environ ment mailbox date index ihave duplicate vnd.dovecot.duplicate plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:mail2.xx:10900 sieve = file:~/sieve;active=~/dovecot.sieve sieve_dir = ~/sieve sieve_duplicate_period = 6h sieve_extensions = +vnd.dovecot.duplicate sieve_global_dir = /etc/dovecot/sieve/global/ sieve_global_path = /etc/dovecot/sieve/default.sieve } replication_max_conns = 5 service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service managesieve { process_limit = 1024 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0660 user = dovecot } } ssl_cert = /etc/dovecot/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_key = /etc/dovecot/private/dovecot.pem protocol lmtp { mail_plugins = notify replication mail_log quota sieve postmaster_address = postmas...@networkers.pl } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_line_length = 65536 } -- Jan Wideł Senior System Administrator e-mail: jan.wi...@networkers.pl mobile: +48 797 004 946 www: http://www.networkers.pl GPG: http://networkers.pl/GPG/2E7359CD.asc
Replication sieve scripts.
Hi, according to changelog 2.2.rc3, dsync should replicate sieve scripts. Do I need turn on or switch some option(s), for this to work? Replication of mailboxes works great, only sieve scripts not. root@mail-1-proidea ~ # dpkg -l dovecot* | grep ^ii ii dovecot-core 2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - core files ii dovecot-imapd2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - IMAP daemon ii dovecot-ldap 2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - LDAP support ii dovecot-lmtpd2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - LMTP server ii dovecot-managesieved 2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - ManageSieve server ii dovecot-pop3d2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - POP3 daemon ii dovecot-sieve2:2.2.15-1~auto+0 amd64secure POP3/IMAP server - Sieve filters support My configuration (doveconf-n): # 2.2.15: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-grsec-amd64 x86_64 Debian 7.7 ext4 auth_mechanisms = plain login auth_verbose = yes default_client_limit = 5000 default_process_limit = 500 default_vsz_limit = 768 M doveadm_password = x doveadm_port = 10900 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_gid = vmail mail_location = maildir:/srv/mail/virtual/%d/%u mail_plugins = notify replication mail_log mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate imapflags notify namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Sent Messages { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:mail2.xxx:10900 sieve = file:~/sieve;active=~/dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_global_dir = /etc/dovecot/sieve/global/ sieve_global_path = /etc/dovecot/sieve/default.sieve } postmaster_address = postmas...@networkers.pl protocols = imap lmtp sieve pop3 replication_max_conns = 5 service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-client { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { address = x.x.x.x port = 10900 ssl = yes } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 500 process_min_avail = 2 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0660 user = dovecot } } ssl = required ssl_cert = /etc/dovecot/dovecot.pem ssl_client_ca_dir = /etc/ssl/certs ssl_key = /etc/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = home=/srv/mail/virtual/%d/%u driver = ldap } protocol lmtp { mail_plugins = notify replication mail_log quota sieve postmaster_address = postmas...@networkers.pl } protocol lda { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = notify replication mail_log sieve sieve autocreate } protocol imap { mail_max_userip_connections = 20 } protocol sieve { managesieve_implementation_string = Dovecot Pigeonhole managesieve_max_line_length = 65536 } -- Jan Wideł Senior System Administrator e-mail: jan.wi...@networkers.pl mobile: +48 797 004 946 www: http://www.networkers.pl GPG: http://networkers.pl/GPG/2E7359CD.asc