Re: replicator: User listing returned failure
Wow. Sorry, maybe it works from morning or maybe somethings changes at the evening, but I may say that replication works fine. But I can confirm it now only with "less on mdbox file". But replication works!!! Thanks Gerald. On 2018-05-08 23:17, Alexey wrote: I don't know if it makes a difference, I don't have quotes on my mail_plugins: I don't have quotes too (It's difference between config file and dovecot -n output) Did you check permissons on the replication fifos? For what? I think that problems on slave. How it should work in automatic mode? I repeat that from slave manually all works fine: As I can see nothing happens automatically. But mx2:~# doveadm -D sync -u ab...@example.com -d -N -l 30 -U successfully executed and getting mails from mx1. prw-rw 1 dovecot mail 0 May 8 06:29 replication-notify-fifo Does it work if you issue: doveadm replicator replicate '*' mx1:~# doveadm replicator replicate '*' 2 users updated doveadm replicator status '*' I already posted this output. Do you have different hostnames for your servers? Sure. You may see in listing that short hostnames are mx1 and mx2.
Re: replicator: User listing returned failure
I don't know if it makes a difference, I don't have quotes on my mail_plugins: I don't have quotes too (It's difference between config file and dovecot -n output) Did you check permissons on the replication fifos? For what? I think that problems on slave. How it should work in automatic mode? I repeat that from slave manually all works fine: As I can see nothing happens automatically. But mx2:~# doveadm -D sync -u ab...@example.com -d -N -l 30 -U successfully executed and getting mails from mx1. prw-rw 1 dovecot mail 0 May 8 06:29 replication-notify-fifo Does it work if you issue: doveadm replicator replicate '*' mx1:~# doveadm replicator replicate '*' 2 users updated doveadm replicator status '*' I already posted this output. Do you have different hostnames for your servers? Sure. You may see in listing that short hostnames are mx1 and mx2.
Re: replicator: User listing returned failure
> What about automatic replication without manually running of doveadm sync? >>> As I can see nothing happens automatically. >>> But mx2:~# doveadm -D sync -u ab...@example.com -d -N -l 30 -U >>> successfully executed and getting mails from mx1. I don't know if it makes a difference, I don't have quotes on my mail_plugins: mail_plugins = $mail_plugins notify replication Did you check permissons on the replication fifos? Does it work if you issue: doveadm replicator replicate '*' doveadm replicator status '*' Do you have different hostnames for your servers? Best regards Gerald
lmtp panic with many recipients
Hi, I had an email with 58 recipients in the "To" and 13 in the "CC" Delivering it from exim to dovecot lmtp panics (see below) Panic: file smtp-address.c: line 533 (smtp_address_write): assertion failed: (smtp_char_is_qpair(*p)) # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.devel (61b47828) # OS: Linux 2.6.32-696.23.1.el6.x86_64 x86_64 CentOS release 6.9 (Final) Regards, Olaf May 8 10:01:52 irams2 dovecot: lmtp(17557): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= May 8 10:01:52 irams2 dovecot: lmtp(17557): Connect from local May 8 10:01:52 irams2 dovecot: lmtp(ms2t...@irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Loading modules from directory: /usr/lib64/dovecot May 8 10:01:52 irams2 dovecot: lmtp(ms2t...@irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Module loaded: /usr/lib64/dovecot/lib90_sieve_plugin.so May 8 10:01:52 irams2 dovecot: lmtp(ms2t...@irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: auth USER input: ms2test system_groups_user=ms2test uid=10852 gid=1602 home=/home/atis/ms2test May 8 10:01:52 irams2 dovecot: lmtp(ms2t...@irams2.ira.uka.de)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: changed username to ms2test May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: Effective uid=10852, gid=1602, home=/home/atis/ms2test May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: Namespace inbox: type=private, prefix=INBOX., sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: maildir++: root=/home/atis/ms2test/Maildir, index=, indexpvt=, control=, inbox=/home/atis/ms2test/Maildir, alt= May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=yes, list=no, subscriptions=no location=fail::LAYOUT=none May 8 10:01:52 irams2 dovecot: lmtp(17557, ms2test): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Pigeonhole version 0.5.devel (61b47828) initializing May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using active Sieve script path: /home/atis/ms2test/.dovecot.sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using script storage path: /home/atis/ms2test/sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Relative path to sieve storage in active link: sieve/ May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using Sieve script path: /home/atis/ms2test/.dovecot.sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file script: Opened script `ingo' from `/home/atis/ms2test/.dovecot.sieve' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file storage: Using Sieve script path: /etc/dovecot/sieve-master May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: file script: Opened script `sieve-master' from `/etc/dovecot/sieve-master' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Executed before user's personal Sieve script(1): /etc/dovecot/sieve-master May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Using the following location for user's Sieve script: /home/atis/ms2test/.dovecot.sieve May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Mailbox : Opened mail UID=1 because: header Message-ID (Cache file is unusable) May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Opening script 1 of 2 from `/etc/dovecot/sieve-master' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Loading script /etc/dovecot/sieve-master May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Script binary /etc/dovecot/sieve-master.svbin successfully loaded May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: binary save: not saving binary /etc/dovecot/sieve-master.svbin, because it is already stored May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: sieve: Executing script from `/etc/dovecot/sieve-master.svbin' May 8 10:01:52 irams2 dovecot: lmtp(ms2test)<17557><30+oHXBZ8VqVRAAApw0JKA>: Debug: Mailbox : Opened mail UID=1 because: header List-Id (Cache file is unusable) May 8 10:01:52 irams2 dovecot:
Re: [sieve][regex] Matching multiple strings in the "Received" header
Op 08/05/2018 om 08:42 schreef Adi Pircalabu: On 08-05-2018 16:20, Gerald Galster wrote: Hello Adi, did you try: " from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com) " If you need to specify the posix character class: [[:blank:]] means space and tab. With pcre it would be like [ \t] [[:space:]] includes space, tab, newline, linefeed, formfeed, vertical tab (in pcre like [ \t\n\r\f\v]) "[[:blank:]]from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)[[:blank:]]" Thanks Gerald, none of your solutions worked, but I've just figured it out now. In the expression the space should only be added at the end, *not* at the beginning! In the Received header the first character isn't [[:blank:]], but "f", so I've been chasing the wild goose all this time because I started with the wrong assumption :) Sorry for the noise, all good now. Also keep in mind that "\" is first used for escaping the string itself, meaning that these need to be duplicated. https://tools.ietf.org/html/rfc5228#section-2.4.2 https://tools.ietf.org/html/draft-murchison-sieve-regex-08#section-3 Regards, Stephan.
Re: imapsieve: script not triggered
* Andreas Krischer2018.05.07 19:58: > sieve_global_extensions = +vnd.dovecot.pipe Hi, my working configuration looks like this: sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute Regards Thomas signature.asc Description: PGP signature
Re: imapsieve: script not triggered
> Check that sieve-test actually shows it would do what you expect it to do. sieve-test just shows: learn-spam: error: the imapsieve extension cannot be used outside IMAP. Andreas > Am 07.05.2018 um 20:56 schrieb Aki Tuomi: > > > > On 2018-05-07 20:58, Andreas Krischer wrote: >> Hi, folks! >> >> My problem: >> I set up dovecot with imapsieve to execute a script whenever a user copies a >> mail to his Spam folder (spam autolearning). Unfortunately this script is >> never executed regardless what I’m configuring for imapsieve. It looks like >> dovecot doesn’t really load the plugin… >> >> The "dovecot -n" output is attached. >> >> I already tried setting imapsieve_mailbox1_name to *, but even this does not >> execute the script… Also I tried a mailbox without mail_crypt enabled, no >> success as well. >> >> In the attached mail.log (mail_debug=yes) you can see, that >> imap_sieve_plugin is loaded, but it isn’t really executed. >> >> The normal (LMTP) sieve is working - if a mail is incoming the sieve module >> shows it work in mail.log: >> >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: file storage: Storage path >> `/var/vmail/myhost.name/postmaster/sieve' not found >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: file storage: Storage path >> `/var/vmail/myhost.name/postmaster/.dovecot.sieve' not found >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: storage: No default script location configured >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: User has no personal script >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: file storage: Using script storage path: >> /etc/dovecot/sieve/after/ >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: file script: Opened script `spam' from >> `/etc/dovecot/sieve/after/spam.sieve' >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: executed after user's Sieve script(1): >> /etc/dovecot/sieve/after/spam.sieve >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> Mailbox : Opened mail UID=1 because: header Message-ID (Cache >> file is unusable) >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: Opening script 1 of 1 from >> `/etc/dovecot/sieve/after/spam.sieve' >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: Loading script >> /etc/dovecot/sieve/after/spam.sieve >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: Script binary >> /etc/dovecot/sieve/after/spam.svbin successfully loaded >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: binary save: not saving binary >> /etc/dovecot/sieve/after/spam.svbin, because it is already stored >> May 7 17:54:04 dovecot dovecot: lmtp(postmas...@myhost.name): Debug: >> YEm9ILyS8FqSaAAAujt/SA: sieve: Executing script from >> `/etc/dovecot/sieve/after/spam.svbin' >> >> >> I already spent hours on this problem so please let me know if you have any >> idea what the problem could be… :( >> >> Thanks in advance! >> >> Andreas >> >> >> > > Check that sieve-test actually shows it would do what you expect it to do. > > Aki
Re: [sieve][regex] Matching multiple strings in the "Received" header
On 08-05-2018 16:20, Gerald Galster wrote: Hello Adi, did you try: " from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com) " If you need to specify the posix character class: [[:blank:]] means space and tab. With pcre it would be like [ \t] [[:space:]] includes space, tab, newline, linefeed, formfeed, vertical tab (in pcre like [ \t\n\r\f\v]) "[[:blank:]]from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)[[:blank:]]" Thanks Gerald, none of your solutions worked, but I've just figured it out now. In the expression the space should only be added at the end, *not* at the beginning! In the Received header the first character isn't [[:blank:]], but "f", so I've been chasing the wild goose all this time because I started with the wrong assumption :) Sorry for the noise, all good now. Cheers, --- Adi Pircalabu
Re: replicator: User listing returned failure
Hello Gerald, Besides you have configured mysql userdb twice: userdb { args = /etc/dovecot/dovecot-sql-master.conf.ext driver = sql } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } Great!!! This was the root cause. Second userdb with dovecot-sql-master.conf rewrite the first one, but sql-master.conf didn't include iterate_query. I removed the wrong second declaration. Thanks. What about automatic replication without manually running of doveadm sync? As I can see nothing happens automatically. But mx2:~# doveadm -D sync -u ab...@example.com -d -N -l 30 -U successfully executed and getting mails from mx1. Regards, Alexey
Re: [sieve][regex] Matching multiple strings in the "Received" header
Hello Adi, did you try: " from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com) " If you need to specify the posix character class: [[:blank:]] means space and tab. With pcre it would be like [ \t] [[:space:]] includes space, tab, newline, linefeed, formfeed, vertical tab (in pcre like [ \t\n\r\f\v]) "[[:blank:]]from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)[[:blank:]]" Best regards, Gerald > Am 08.05.2018 um 03:38 schrieb Adi Pircalabu: > > On 08-05-2018 2:43, Benny Pedersen wrote: >> Adi Pircalabu skrev den 2018-05-07 05:10: >>> How should I write it to also match the space character at both the >>> beginning and end of the expression? >> use \ before space char > > Tks. Just tried these two, unsuccessfully: > "\.from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)\." > "\ from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)\ > " > > However, this expression always matches: > "from.*(outbound.protection.outlook.com|.google.com|.yahoo.com|mx.aol.com)" > > What am I missing? > > --- > Adi Pircalabu
Re: replicator: User listing returned failure
Hello Alexey, > mx1:~# dovecot --version > 2.2.27 (c0f36b0) > > From dovecot.log: > May 07 19:27:41 auth-worker(34348): Warning: mysql: Query failed, retrying: > Unknown column 'username' in 'field list' > May 07 19:27:41 auth-worker(34348): Debug: sql(*): SELECT username, domain > FROM users > May 07 19:27:41 auth-worker(34348): Error: sql: Iterate query failed: Unknown > column 'username' in 'field list' (using built-in default iterate_query: > SELECT username, domain FROM users) > May 07 19:27:41 auth-worker(34348): Debug: sql(*): SELECT id AS username, > domain FROM users > May 07 19:27:41 replicator: Error: User listing returned failure > May 07 19:27:41 replicator: Error: listing users failed, can't replicate > existing data > dovecot-sql.conf.ext: > driver = mysql > ... > iterate_query = SELECT id AS username, domain FROM users you defined a custom iterate_query but the debug message says it uses the built-in default: > using built-in default iterate_query: SELECT username, domain FROM users Please check if your config is included: conf.d/10-auth.conf: !include auth-sql.conf.ext conf.d/auth-sql.conf.ext: userdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext } Maybe your dovecot-sql.conf.ext is somewhere dovecot does not look for it. Besides you have configured mysql userdb twice: > userdb { > args = /etc/dovecot/dovecot-sql-master.conf.ext > driver = sql > } > userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } Best regards, Gerald > > mx1:~# doveadm replicator status > Queued 'sync' requests0 > Queued 'high' requests0 > Queued 'low' requests 0 > Queued 'failed' requests 0 > Queued 'full resync' requests 0 > Waiting 'failed' requests 2 > Total number of known users 2 > > > mx1:~# doveadm replicator status '*' > username > priority fast sync full sync success sync failed > ab...@example.com > none 00:03:01 01:06:36 -y > ad...@exmaple.com > none 00:03:41 01:31:49 -y > > > > From slave: > > mx2:~# cat /etc/dovecot/conf.d/90-replication.conf > # use tcp:hostname as the dsync target > plugin { > mail_replica = tcp:mx1 # use doveadm_port > } > > > As I can see nothing happens automatically. > But mx2:~# doveadm -D sync -u ab...@example.com -d -N -l 30 -U successfully > executed and getting mails from mx1. > > > > > dovecot-sql.conf.ext: > driver = mysql > connect = host=localhost dbname=vmail user=sqlmail > password=sqL_hidden033|TGPAS > default_pass_scheme = SHA512-CRYPT > password_query = SELECT id AS username, password, domain FROM users WHERE id > = '%n' AND domain = '%d' AND active = 'Y' > user_query = SELECT id AS username, uid, gid, home, concat('*:storage=', > quota, 'M' ) as quota_rule FROM users WHERE id = '%n' AND domain = '%d' > iterate_query = SELECT id AS username, domain FROM users > > Regards, > Alexey > > > mx1:~# dovecot -n > # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.16 (fed8554) > # OS: Linux 4.9.0-6-amd64 x86_64 Debian 9.4 > auth_master_user_separator = * > auth_verbose = yes > auth_verbose_passwords = sha1 > default_vsz_limit = 512 M > doveadm_password = # hidden, use -P to show it > doveadm_port = 994 > hostname = mx1.example.com > imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags > imap_idle_notify_interval = 12 mins > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > lmtp_save_to_detail_mailbox = yes > log_path = /var/log/dovecot/dovecot.log > mail_access_groups = mail > mail_location = mdbox:~/mdbox:UTF-8 > mail_plugins = " notify replication" > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character > vacation subaddress comparator-i;ascii-numeric relational regex imap4flags > copy include variables body enotify environment mailbox date index ihave > duplicate mime foreverypart extracttext > namespace inbox { > inbox = yes > location = > mailbox Drafts { >auto = subscribe >special_use = \Drafts > } > mailbox Junk { >auto = subscribe >special_use = \Junk > } > mailbox Sent { >auto = subscribe >special_use = \Sent > } > mailbox Trash { >auto = subscribe >special_use = \Trash > } > prefix = > } > passdb { > args = /etc/dovecot/dovecot-sql-master.conf.ext > driver = sql > master = yes > pass = yes > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > quota = dict:User quota::proxy::quota > quota_rule = *:storage=100G > quota_rule2 = Trash:storage=+10G > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve_before =