Re: [EXT] Re: Dovecot v2.3.17 released

2021-10-28 Thread Aki Tuomi


> On 28/10/2021 21:24 Frank Elsner  wrote:
> 
>  
> On Thu, 28 Oct 2021 19:17:40 +0300 (EEST) Aki Tuomi wrote:
> > 
> > > On 28/10/2021 19:10 Frank Elsner  wrote:
> > > 
> > >  
> > > On Thu, 28 Oct 2021 17:32:28 +0300 (EEST) Aki Tuomi wrote:
> > > 
> > >   [ ... ]
> > > 
> > > > I guess you are using systemd-coredump, so try
> > > > 
> > > > coredumpctl dump -o core /usr/libexec/dovecot/imap 
> 
>   { ... ]
> 
> > In the directory you ran that command, there will be now a file called 
> > 'core'. Can you now do:
> > 
> > gdb /usr/libexec/dovecot/imap core
> > bt full
> 
> Hi Aki,
> 
> I hope to deliver the requested info with this output


> By for today, Frank

Perfect, thank you very much.

Aki


Re: [EXT] Re: Dovecot v2.3.17 released

2021-10-28 Thread Frank Elsner
On Thu, 28 Oct 2021 19:17:40 +0300 (EEST) Aki Tuomi wrote:
> 
> > On 28/10/2021 19:10 Frank Elsner  wrote:
> > 
> >  
> > On Thu, 28 Oct 2021 17:32:28 +0300 (EEST) Aki Tuomi wrote:
> > 
> >   [ ... ]
> > 
> > > I guess you are using systemd-coredump, so try
> > > 
> > > coredumpctl dump -o core /usr/libexec/dovecot/imap 

  { ... ]

> In the directory you ran that command, there will be now a file called 
> 'core'. Can you now do:
> 
> gdb /usr/libexec/dovecot/imap core
> bt full

Hi Aki,

I hope to deliver the requested info with this output

(gdb) bt full
#0  fts_user_autoindex_exclude (box=, box@entry=0x559500fdd138) 
at fts-user.c:347
fuser = 
#1  0x7f50d8f623b6 in fts_mailbox_allocated (box=0x559500fdd138) at 
fts-storage.c:806
flist = 
v = 0x559500fdd7b8
fbox = 0x559500fdd918
#2  0x7f50d942bd1c in hook_mailbox_allocated (box=box@entry=0x559500fdd138) 
at mail-storage-hooks.c:256
_data_stack_cur_id = 5
_foreach_end = 0x559500fce758
_foreach_ptr = 0x559500fce740
hooks = 0x7f50d8f90b40 
ctx = 0x559500fde558
#3  0x7f50d9426f11 in mailbox_alloc (list=0x559500fd40c8, 
vname=vname@entry=0x559500f87620 "Administration", 
flags=flags@entry=0) at mail-storage.c:885
_data_stack_cur_id = 4
new_list = 0x559500fd40c8
storage = 0x559500fd48d8
box = 0x559500fdd138
open_error = MAIL_ERROR_NONE
errstr = 0x0
__func__ = "mailbox_alloc"
#4  0x5594fffcaae2 in cmd_create (cmd=0x559500fd38b8) at cmd-create.c:40
ns = 0x559500fd3390
--Type  for more, q to quit, c to continue without paging--
mailbox = 0x559500f87620 "Administration"
orig_mailbox = 
box = 0x559500fdc5a0
directory = 
len = 14
#5  0x5594fffd8494 in command_exec (cmd=cmd@entry=0x559500fd38b8) at 
imap-commands.c:201
hook = 0x559500f93580
finished = 
__func__ = "command_exec"
#6  0x5594fffd64c2 in client_command_input (cmd=) at 
imap-client.c:1230
client = 0x559500fd2788
command = 
tag = 0x7f50d9331f3a  
"]A\\A]\303\061\300\303ff.\017\037\204"
name = 0x5594fffea726 "CREATE"
ret = 
__func__ = "client_command_input"
#7  0x5594fffd6574 in client_command_input (cmd=) at 
imap-client.c:1297
client = 0x559500fd2788
command = 
tag = 0x559500f9e0e8 "161"
name = 0x559500f9e0f0 "CREATE"
ret = 
__func__ = "client_command_input"
--Type  for more, q to quit, c to continue without paging--
#8  0x5594fffd697d in client_handle_next_command (remove_io_r=, client=0x559500fd2788)
at imap-client.c:1339
No locals.
#9  client_handle_input (client=0x559500fd2788) at imap-client.c:1353
_data_stack_cur_id = 3
ret = 
remove_io = false
handled_commands = false
__func__ = "client_handle_input"
#10 0x5594fffd6f30 in client_input (client=0x559500fd2788) at 
imap-client.c:1397
cmd = 0x559500f9b6a8
output = 0x559500fd31f0
bytes = 27
__func__ = "client_input"
#11 0x7f50d9314249 in io_loop_call_io (io=0x559500fd3330) at ioloop.c:737
ioloop = 0x559500f8fe70
t_id = 2
__func__ = "io_loop_call_io"
#12 0x7f50d93158f2 in io_loop_handler_run_internal 
(ioloop=ioloop@entry=0x559500f8fe70) at ioloop-epoll.c:222
ctx = 0x559500f925d0
events = 
event = 0x559500f92650
list = 0x559500f9b690
--Type  for more, q to quit, c to continue without paging--
io = 
tv = {tv_sec = 1799, tv_usec = 999057}
events_count = 
msecs = 
ret = 1
i = 0
j = 0
call = 
__func__ = "io_loop_handler_run_internal"
#13 0x7f50d93142f0 in io_loop_handler_run (ioloop=0x559500f8fe70) at 
ioloop.c:789
__func__ = "io_loop_handler_run"
#14 0x7f50d93144b0 in io_loop_run (ioloop=0x559500f8fe70) at ioloop.c:762
__func__ = "io_loop_run"
#15 0x7f50d92884f3 in master_service_run (service=0x559500f8fcd0, 
callback=callback@entry=0x5594fffe4c90 ) at 
master-service.c:863
No locals.
#16 0x5594fffc7f85 in main (argc=, argv=) at 
main.c:564
set_roots = {0x559498e0 , 
0x55949600 , 0x0}
login_set = {auth_socket_path = 0x559500f877f8 "\003", 
postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, 
  callback = 0x5594fffe5640 , failure_callback 
= 0x5594fffe4d60 , 
  request_auth_token = true}
service_flags = 
storage_service_flags = (MAIL_STORAGE_SERVICE_FLAG_NO_NAMESPACES | 
MAIL_STORAGE_SERVICE_FLAG_NO_SSL_CA)
--Type  for more, q to quit, c to continue without paging--
username = 
auth_socket_path = 
c = 
error = 0x7fffe05085a8 ""
(gdb) 


By for today, Frank



Re: [Dovecot-news] Dovecot v2.3.17 released

2021-10-28 Thread Daniel J. Luke
On Oct 28, 2021, at 5:12 AM, Aki Tuomi  wrote:
> We are pleased to release v2.3.17 of Dovecot.

This patch is still needed to build on newer MacOS -

--- src/lib/ioloop-notify-kqueue.c.orig 2021-06-14 07:56:46.0 -0400
+++ src/lib/ioloop-notify-kqueue.c  2021-06-21 12:10:16.0 -0400
@@ -11,6 +11,7 @@
 
 #include "ioloop-private.h"
 #include "llist.h"
+#include "time-util.h"
 #include 
 #include 
 #include 

see also https://dovecot.org/pipermail/dovecot/2020-September/119963.html and 
https://dovecot.org/list/dovecot/2021-June/122375.html
-- 
Daniel J. Luke



Re: Pigeonhole v0.5.17 released

2021-10-28 Thread Aki Tuomi


> On 28/10/2021 19:43 Gedalya  wrote:
> 
>  
> I've noticed 27ab897 is not in the new release, may I ask why?

It didn't make it.

Aki


Re: Dovecot v2.3.17 released

2021-10-28 Thread Frank Elsner
On Thu, 28 Oct 2021 17:32:28 +0300 (EEST) Aki Tuomi wrote:

  [ ... ]

> I guess you are using systemd-coredump, so try
> 
> coredumpctl dump -o core /usr/libexec/dovecot/imap 

That worked so I can provide

# coredumpctl dump -o core /usr/local/dovecot/libexec/dovecot/imap
   PID: 309338 (imap)
   UID: 1953 (frank)
   GID: 12203 (elsner)
Signal: 11 (SEGV)
 Timestamp: Thu 2021-10-28 15:48:43 CEST (2h 19min ago)
  Command Line: dovecot/imap [frank 127.0.0.1 STATUS]
Executable: /usr/local/dovecot/libexec/dovecot/imap
 Control Group: /system.slice/dovecot.service
  Unit: dovecot.service
 Slice: system.slice
   Boot ID: c97b6e2b6f464589b7a50fbf4009d2bb
Machine ID: 5367c967725543b39d46d1b5b5b90fa8
  Hostname: christo
   Storage: 
/var/lib/systemd/coredump/core.imap.1953.c97b6e2b6f464589b7a50fbf4009d2bb.309338.163542892300.zst
 (present)
 Disk Size: 252.6K
   Message: Process 309338 (imap) of user 1953 dumped core.

Stack trace of thread 309338:
#0  0x7ff2193782cb fts_user_autoindex_exclude 
(lib20_fts_plugin.so + 0xa2cb)
#1  0x7ff2193813b6 fts_mailbox_allocated 
(lib20_fts_plugin.so + 0x133b6)
#2  0x7ff21984ad1c hook_mailbox_allocated 
(libdovecot-storage.so.0 + 0x62d1c)
#3  0x7ff219845f11 mailbox_alloc (libdovecot-storage.so.0 + 
0x5df11)
#4  0x55d49e2003e5 imap_status_get (imap + 0x2b3e5)
#5  0x55d49e1f0c6e cmd_status (imap + 0x1bc6e)
#6  0x55d49e1f7494 command_exec (imap + 0x22494)
#7  0x55d49e1f54c2 client_command_input (imap + 0x204c2)
#8  0x55d49e1f5574 client_command_input (imap + 0x20574)
#9  0x55d49e1f597d client_handle_next_command (imap + 
0x2097d)
#10 0x55d49e1f5f30 client_input (imap + 0x20f30)
#11 0x7ff219733249 io_loop_call_io (libdovecot.so.0 + 
0x116249)
#12 0x7ff2197348f2 io_loop_handler_run_internal 
(libdovecot.so.0 + 0x1178f2)
#13 0x7ff2197332f0 io_loop_handler_run (libdovecot.so.0 + 
0x1162f0)
#14 0x7ff2197334b0 io_loop_run (libdovecot.so.0 + 0x1164b0)
#15 0x7ff2196a74f3 master_service_run (libdovecot.so.0 + 
0x8a4f3)
#16 0x55d49e1e6f85 main (imap + 0x11f85)
#17 0x7ff219453b75 __libc_start_main (libc.so.6 + 0x27b75)
#18 0x55d49e1e704e _start (imap + 0x1204e)

HTH, Frank


Re: Dovecot does not start on MacOS 12.01

2021-10-28 Thread Don Feliciano
Same issue in 2.3.17

dovecot-2.3.17.tar.gz:
./configure --prefix=/usr/local/Cellar/dovecot/2.3.17 
--libexecdir=/usr/local/Cellar/dovecot/2.3.17/libexec 
--sysconfdir=/usr/local/etc --localstatedir=/usr
make install

dovecot-2.3-pigeonhole-0.5.17.tar.gz:
 ./configure --with-dovecot=/usr/local/Cellar/dovecot/2.3.17/lib/dovecot 
--prefix=/usr/local/Cellar/dovecot/2.3.17
make
make install

sudo /usr/local/opt/dovecot/sbin/dovecot -F
Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 
1000), because of default_client_limit
Oct 28 12:08:48 service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): Invalid 
argument

> On Oct 27, 2021, at 11:08 AM, Don Feliciano  wrote:
>
> I've been happily running Dovecot on my Mac for many years (installed via 
> Homebrew). After upgrading to Monterey (MacOS 12.01), it no longer starts:
>
> $ sw_vers
> ProductName: macOS
> ProductVersion: 12.0.1
> BuildVersion: 21A559
>
> $ uname -a
> Darwin dfelicia-mac 21.1.0 Darwin Kernel Version 21.1.0: Wed Oct 13 17:33:23 
> PDT 2021; root:xnu-8019.41.5~1/RELEASE_X86_64 x86_64
>
> $ sudo dovecot -F
> Oct 27 10:11:18 service(log): Fatal: setrlimit(RLIMIT_DATA, 268435456): 
> Invalid argument
>
> $ mount
> /dev/disk1s2s1 on / (apfs, sealed, local, read-only, journaled)
> devfs on /dev (devfs, local, nobrowse)
> /dev/disk1s5 on /System/Volumes/VM (apfs, local, noexec, journaled, noatime, 
> nobrowse)
> /dev/disk1s3 on /System/Volumes/Preboot (apfs, local, journaled, nobrowse)
> /dev/disk1s6 on /System/Volumes/Update (apfs, local, jour naled, nobrowse)
> /dev/disk1s1 on /System/Volumes/Data (apfs, local, journaled, nobrowse)
>
> $ sudo dovecot -n
> # 2.3.16 (7e2e900c1a): /usr/local/etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.16 (09c29328)
> # OS: Darwin 21.1.0 x86_64 apfs
> # Hostname: dfelicia-mac
> default_internal_group = mail
> default_internal_user = _dovecot
> default_login_user = _dovenull
> first_valid_uid = 100
> listen = 127.0.0.1
> log_path = /var/log/dovecot.log
> mail_location = maildir:/usr/local/var/mail/%u
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope encoded-character 
> vacation subaddress comparator-i;ascii-numeric relational regex imap4flags 
> copy include variables body enotify environment mailbox date index ihave 
> duplicate mime foreverypart extracttext
> namespace inbox {
> inbox = yes
> location =
> mailbox Drafts {
> special_use = \Drafts
> }
> mailbox Junk {
> special _use = \Junk
> }
> mailbox Sent {
> special_use = \Sent
> }
> mailbox "Sent Messages" {
> special_use = \Sent
> }
> mailbox Trash {
> special_use = \Trash
> }
> prefix =
> }
> passdb {
> args = login
> driver = pam
> }
> protocols = imap
> service imap-login {
> inet_listener imap {
> address = 127.0.0.1
> port = 143
> }
> inet_listener imaps {
> address = 127.0.0.1
> ssl = no
> }
> }
> ssl = no
> userdb {
> driver = passwd
> }
>
> $ ulimit -Ha
> core file size (blocks, -c) unlimited
> data seg size (kbytes, -d) unlimited
> file size (blocks, -f) unlimited
> max locked memory (kbytes, -l) unlimited
> max memory size (kbytes, -m) unlimited
> open files (-n) unlimited
> pipe size (512 bytes, -p) 1
> stack size (kbytes, -s) 65532
> cpu time (seconds, -t) unlimited
> max user processes (-u) 2784
> virtual memory (kbytes, -v) unlimited
>
>
> 



Re: Dovecot version 2.3.17 - mail replication stopped working

2021-10-28 Thread Daniel Botting

Hi Aki,

Thanks for your quick response, much appreciated.

We've removed -T from our config and replication is now working again.

I'll keep an eye on the mailing list to see when it is fixed in master.

Best regards

Daniel

On 28/10/2021 16:11, Aki Tuomi wrote:

Appears option T got removed by accident. We'll fix it latest for 2.3.18, and 
we'll let you know once it's fixed in master. You can either rollback to 2.3.16 
or remove -T from sync options in config.


--
Daniel Botting
Systems Administrator
Codethink Ltd.
3rd Floor Dale House,
35 Dale Street,
Manchester, M1 2HF
United Kingdom

http://www.codethink.co.uk/
We respect your privacy. See https://www.codethink.co.uk/privacy.html



Re: Dovecot version 2.3.17 - mail replication stopped working

2021-10-28 Thread Aki Tuomi
Appears option T got removed by accident. We'll fix it latest for 2.3.18, and 
we'll let you know once it's fixed in master. You can either rollback to 2.3.16 
or remove -T from sync options in config.

Aki

> On 28/10/2021 17:33 Daniel Botting  wrote:
> 
>  
> Hi,
> 
> We upgraded to Dovecot 2.3.17 today and mail replication has stopped 
> working as expected, the error in mail.err is:
> 
> Oct 28 10:27:44 hostname dovecot: imap-login: Error: 
> net_connect_unix(/var/run/dovecot/stats-writer) failed: No such file or 
> directory
> 
> Oct 28 10:28:47 hostname dovecot: doveadm: Error: sync: invalid option 
> -- 'T'
> 
> This is constantly repeated afterwards
> 
> All the files in /var/run/dovecot have last modified time of 10:27
> 
> The permissions on both hosts for this file are:
> 
> srw-rw  1 root    dovecot    0 Oct 28 10:27 stats-writer=
> 
> OS: Debian 10.11
> 
> Doveconf -n output below:
> 
> # 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
> # Pigeonhole version 0.5.17 (054dddfa)
> # OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11
> # Hostname: mailserver fqdn
> auth_verbose = yes
> default_vsz_limit = 0
> doveadm_password = # hidden, use -P to show it
> first_valid_gid = 8
> first_valid_uid = 8
> last_valid_gid = 8
> last_valid_uid = 8
> lda_mailbox_autocreate = yes
> lda_mailbox_autosubscribe = yes
> mail_gid = 8
> mail_location = maildir:~/Maildir
> mail_plugins = " notify replication"
> mail_uid = 8
> managesieve_notify_capability = mailto
> managesieve_sieve_capability = fileinto reject envelope 
> encoded-character vacation subaddress comparator-i;ascii-numeric 
> relational regex imap4flags copy include variables body enotify 
> environment mailbox date index ihave duplicate mime foreverypart 
> extracttext editheader imapflags
> namespace inbox {
>    inbox = yes
>    location =
>    mailbox Drafts {
>      special_use = \Drafts
>    }
>    mailbox Junk {
>      special_use = \Junk
>    }
>    mailbox Sent {
>      special_use = \Sent
>    }
>    mailbox "Sent Messages" {
>      special_use = \Sent
>    }
>    mailbox Trash {
>      special_use = \Trash
>    }
>    prefix =
> }
> passdb {
>    args = /etc/dovecot/dovecot-ldap.conf.ext
>    driver = ldap
> }
> plugin {
>    mail_replica = tcps:fqdn:port number
>    sieve = ~/.dovecot.sieve
>    sieve_dir = ~/sieve
>    sieve_extensions = +editheader +imapflags
> }
> postmaster_address = postmaster@ourdomain
> protocols = " imap sieve pop3"
> replication_dsync_parameters = -d -N -l 15 -U -T 10
> replication_max_conns = 24
> service aggregator {
>    fifo_listener replication-notify-fifo {
>      user = mail
>    }
>    unix_listener replication-notify {
>      user = mail
>    }
> }
> service auth {
>    unix_listener /var/run/dovecot-exim-bridge {
>      mode = 0660
>      user = Debian-exim
>    }
> }
> service doveadm {
>    inet_listener {
>      port = 12345
>      ssl = yes
>    }
> }
> service imap-login {
>    inet_listener imap {
>      port = 143
>    }
>    inet_listener imaps {
>      port = 993
>      ssl = yes
>    }
>    process_limit = 512
>    process_min_avail = 4
>    service_count = 1
> }
> service imap {
>    process_limit = 1024
> }
> service managesieve-login {
>    inet_listener sieve {
>      port = 4190
>    }
>    process_min_avail = 1
>    service_count = 8
>    vsz_limit = 256 M
> }
> service managesieve {
>    process_limit = 1024
> }
> service replicator {
>    process_min_avail = 1
>    unix_listener replicator-doveadm {
>      mode = 0666
>    }
> }
> ssl = required
> ssl_cert =  ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL
> ssl_client_ca_dir = /etc/ssl/certs
> ssl_key = # hidden, use -P to show it
> userdb {
>    args = /etc/dovecot/dovecot-ldap.conf.ext
>    driver = ldap
>    override_fields = home=/var/mail/%u uid=mail gid=mail
> }
> protocol lda {
>    mail_plugins = " notify replication sieve"
> }
> protocol imap {
>    mail_max_userip_connections = 32
>    mail_plugins = " notify replication"
> }
> protocol sieve {
>    mail_max_userip_connections = 10
>    managesieve_logout_format = bytes=%i/%o
>    managesieve_max_line_length = 64 k
> }
> 
> Best regards
> 
> Daniel


Dovecot version 2.3.17 - mail replication stopped working

2021-10-28 Thread Daniel Botting

Hi,

We upgraded to Dovecot 2.3.17 today and mail replication has stopped 
working as expected, the error in mail.err is:


Oct 28 10:27:44 hostname dovecot: imap-login: Error: 
net_connect_unix(/var/run/dovecot/stats-writer) failed: No such file or 
directory


Oct 28 10:28:47 hostname dovecot: doveadm: Error: sync: invalid option 
-- 'T'


This is constantly repeated afterwards

All the files in /var/run/dovecot have last modified time of 10:27

The permissions on both hosts for this file are:

srw-rw  1 root    dovecot    0 Oct 28 10:27 stats-writer=

OS: Debian 10.11

Doveconf -n output below:

# 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.17 (054dddfa)
# OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11
# Hostname: mailserver fqdn
auth_verbose = yes
default_vsz_limit = 0
doveadm_password = # hidden, use -P to show it
first_valid_gid = 8
first_valid_uid = 8
last_valid_gid = 8
last_valid_uid = 8
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_gid = 8
mail_location = maildir:~/Maildir
mail_plugins = " notify replication"
mail_uid = 8
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext editheader imapflags

namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  mail_replica = tcps:fqdn:port number
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +editheader +imapflags
}
postmaster_address = postmaster@ourdomain
protocols = " imap sieve pop3"
replication_dsync_parameters = -d -N -l 15 -U -T 10
replication_max_conns = 24
service aggregator {
  fifo_listener replication-notify-fifo {
    user = mail
  }
  unix_listener replication-notify {
    user = mail
  }
}
service auth {
  unix_listener /var/run/dovecot-exim-bridge {
    mode = 0660
    user = Debian-exim
  }
}
service doveadm {
  inet_listener {
    port = 12345
    ssl = yes
  }
}
service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_limit = 512
  process_min_avail = 4
  service_count = 1
}
service imap {
  process_limit = 1024
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  process_min_avail = 1
  service_count = 8
  vsz_limit = 256 M
}
service managesieve {
  process_limit = 1024
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0666
  }
}
ssl = required
ssl_cert = 

Re: Dovecot v2.3.17 released

2021-10-28 Thread Aki Tuomi


> On 28/10/2021 15:39 Frank Elsner  wrote:
> 
>  
> On Thu, 28 Oct 2021 11:39:42 + Aki Tuomi wrote:
> > Hi!
> > 
> > Can you provide gdb bt full output for this crash?
> 
> I fear this is far beyond my knowledge :-( but I will try.
> 
> > Install debug symbols (if necessary) and
> > 
> > gdb /usr/libexec/dovecot/imap /path/to/core
> 
> What core?
> 
> 
> --Frank

I guess you are using systemd-coredump, so try

coredumpctl dump -o core /usr/libexec/dovecot/imap 

If this does not work, or you are not using systemd-coredump, please see 
https://www.dovecot.org/bugreport-mail/

Aki


Re: Solr FTS - message deletes not working as expected

2021-10-28 Thread Shawn Heisey

On 10/26/21 12:18 PM, Shawn Heisey wrote:
But if I use shift-delete in Thunderbird, which deletes the message 
immediately without going through Trash, things are different. 



Also, when I send a message with Thunderbird, which deletes the message 
in Drafts and adds one to Sent, I am not seeing a delete request in the 
Solr log.  I do see the add.  So this isn't limited to just the 
shift-delete workflow.


Thanks,
Shawn




Re: Dovecot v2.3.17 released

2021-10-28 Thread Frank Elsner
On Thu, 28 Oct 2021 11:39:42 + Aki Tuomi wrote:
> Hi!
> 
> Can you provide gdb bt full output for this crash?

I fear this is far beyond my knowledge :-( but I will try.

> Install debug symbols (if necessary) and
> 
> gdb /usr/libexec/dovecot/imap /path/to/core

What core?


--Frank


Re: Dovecot v2.3.17 released

2021-10-28 Thread Aki Tuomi
Hi!

Can you provide gdb bt full output for this crash?

Install debug symbols (if necessary) and

gdb /usr/libexec/dovecot/imap /path/to/core
bt full

Aki

On 28 October 2021 11.27.07 UTC, Frank Elsner  wrote:
>On Thu, 28 Oct 2021 12:12:53 +0300 (EEST) Aki Tuomi wrote:
>> We are pleased to release v2.3.17 of Dovecot.
>
>Hello,
>
>on my private fully updated Fedora 34 system I've replaced version 2.3.16 by
>version 2.3.17 without touching the configuration.
>
>Whereas version 2.3.16 worked the new 2.3.17 says
>Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, 
>TLS
>Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): ID sent: 
>name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, 
>support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +, 
>side=host2
>Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: 
>service(imap): child 216943 killed with signal 11 (core dumped)
>Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, 
>TLS
>Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: 
>service(imap): child 216952 killed with signal 11 (core dumped)
>Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, 
>TLS
>Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: 
>service(imap): child 216964 killed with signal 11 (core dumped)
>
>
>For comparison the old version
>
>Oct 28 11:36:58 christo dovecot[279524]: imap-login: Login: frank, 127.0.0.1, 
>TLS
>Oct 28 11:36:58 christo dovecot[279524]: IMAP(frank,127.0.0.1): ID sent: 
>name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, 
>support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +, 
>side=host2
>Oct 28 11:37:37 christo dovecot[279524]: IMAP(frank,127.0.0.1): Disconnected: 
>Logged out [79451/32883563]
>
>
>If required here is the configuration which is rather old but worked until 
>2.3.16
>
># 2.3.16 (7e2e900c1a): /usr/local/dovecot/etc/dovecot/dovecot.conf
># OS: Linux 5.14.13-200.fc34.x86_64 x86_64 Fedora release 34 (Thirty Four) ext3
># Hostname: christo
>auth_mechanisms = plain login
>default_client_limit = 1024
>default_process_limit = 256
>default_vsz_limit = 512 M
>first_valid_uid = 200
>last_valid_uid = 65534
>listen = *
>lmtp_save_to_detail_mailbox = yes
>login_greeting = m28a.ddns.net - IMAPs Service (dovecot) ready.
>login_log_format_elements = %u %r %c
>mail_location = maildir:/var/spool/mail/%u:LAYOUT=fs
>mail_log_prefix = "%Us(%u,%r): "
>mail_plugin_dir = /usr/dovecot/lib/dovecot/
>mail_plugins = notify quota fts fts_squat acl
>namespace inbox {
>  inbox = yes
>  list = yes
>  location = 
>  mailbox Drafts {
>special_use = \Drafts
>  }
>  mailbox Gesendet {
>special_use = \Sent
>  }
>  mailbox SPAM {
>special_use = \Junk
>  }
>  mailbox Sent {
>special_use = \Sent
>  }
>  mailbox Trash {
>special_use = \Trash
>  }
>  prefix = 
>  subscriptions = yes
>  type = private
>}
>passdb {
>  args = dovecot
>  driver = pam
>}
>plugin {
>  fts = squat
>  fts_squat = partial=4 full=10
>  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename 
> flag_change append
>  mail_log_fields = uid box from subject msgid size flags
>  mail_log_group_events = yes
>}
>postmaster_address = mailona...@mailbox.org
>protocols = imap
>service anvil {
>  client_limit = 1024
>}
>service auth {
>  unix_listener auth-client {
>group = exim
>mode = 0660
>user = exim
>  }
>}
>service imap-login {
>  inet_listener imap {
>port = 143
>  }
>  process_limit = 512
>  process_min_avail = 3
>}
>service imap-postlogin {
>  executable = script-login /usr/local/sbin/dovecot-imap-post-login
>}
>service imap {
>  executable = imap
>  process_limit = 128
>}
>service pop3-login {
>  inet_listener pop3 {
>port = 110
>  }
>  inet_listener pop3s {
>port = 995
>ssl = yes
>  }
>}
>service pop3 {
>  process_limit = 128
>}
>ssl_cert = ssl_dh = # hidden, use -P to show it
>ssl_key = # hidden, use -P to show it
>userdb {
>  driver = passwd
>}
>verbose_proctitle = yes
>protocol lmtp {
>  mail_plugins = notify quota fts fts_squat acl
>}
>protocol lda {
>  mail_plugins = notify quota fts fts_squat acl
>}
>protocol imap {
>  imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags
>  imap_id_log = *
>  imap_logout_format = [%i/%o]
>  mail_max_userip_connections = 0
>  mail_plugins = notify quota fts fts_squat acl quota imap_quota listescape 
> fts fts_squat
>}
>
>
>What's going on? Any pointer to solution welcome.
>
>
>Kind regards, Frank 


Re: Dovecot v2.3.17 released

2021-10-28 Thread Frank Elsner
On Thu, 28 Oct 2021 12:12:53 +0300 (EEST) Aki Tuomi wrote:
> We are pleased to release v2.3.17 of Dovecot.

Hello,

on my private fully updated Fedora 34 system I've replaced version 2.3.16 by
version 2.3.17 without touching the configuration.

Whereas version 2.3.16 worked the new 2.3.17 says
Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, 
TLS
Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): ID sent: 
name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, 
support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +, 
side=host2
Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: 
service(imap): child 216943 killed with signal 11 (core dumped)
Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, 
TLS
Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: 
service(imap): child 216952 killed with signal 11 (core dumped)
Oct 28 11:29:27 christo dovecot[216609]: imap-login: Login: frank, 127.0.0.1, 
TLS
Oct 28 11:29:27 christo dovecot[216609]: IMAP(frank,127.0.0.1): Fatal: master: 
service(imap): child 216964 killed with signal 11 (core dumped)


For comparison the old version

Oct 28 11:36:58 christo dovecot[279524]: imap-login: Login: frank, 127.0.0.1, 
TLS
Oct 28 11:36:58 christo dovecot[279524]: IMAP(frank,127.0.0.1): ID sent: 
name=imapsync, version=1.977, os=linux, vendor=Gilles LAMIRAL, 
support-url=https://imapsync.lamiral.info/, date=23-Dec-2019 20:18:02 +, 
side=host2
Oct 28 11:37:37 christo dovecot[279524]: IMAP(frank,127.0.0.1): Disconnected: 
Logged out [79451/32883563]


If required here is the configuration which is rather old but worked until 
2.3.16

# 2.3.16 (7e2e900c1a): /usr/local/dovecot/etc/dovecot/dovecot.conf
# OS: Linux 5.14.13-200.fc34.x86_64 x86_64 Fedora release 34 (Thirty Four) ext3
# Hostname: christo
auth_mechanisms = plain login
default_client_limit = 1024
default_process_limit = 256
default_vsz_limit = 512 M
first_valid_uid = 200
last_valid_uid = 65534
listen = *
lmtp_save_to_detail_mailbox = yes
login_greeting = m28a.ddns.net - IMAPs Service (dovecot) ready.
login_log_format_elements = %u %r %c
mail_location = maildir:/var/spool/mail/%u:LAYOUT=fs
mail_log_prefix = "%Us(%u,%r): "
mail_plugin_dir = /usr/dovecot/lib/dovecot/
mail_plugins = notify quota fts fts_squat acl
namespace inbox {
  inbox = yes
  list = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Gesendet {
special_use = \Sent
  }
  mailbox SPAM {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
  subscriptions = yes
  type = private
}
passdb {
  args = dovecot
  driver = pam
}
plugin {
  fts = squat
  fts_squat = partial=4 full=10
  mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename 
flag_change append
  mail_log_fields = uid box from subject msgid size flags
  mail_log_group_events = yes
}
postmaster_address = mailona...@mailbox.org
protocols = imap
service anvil {
  client_limit = 1024
}
service auth {
  unix_listener auth-client {
group = exim
mode = 0660
user = exim
  }
}
service imap-login {
  inet_listener imap {
port = 143
  }
  process_limit = 512
  process_min_avail = 3
}
service imap-postlogin {
  executable = script-login /usr/local/sbin/dovecot-imap-post-login
}
service imap {
  executable = imap
  process_limit = 128
}
service pop3-login {
  inet_listener pop3 {
port = 110
  }
  inet_listener pop3s {
port = 995
ssl = yes
  }
}
service pop3 {
  process_limit = 128
}
ssl_cert = 

Re: Pigeonhole v0.5.17 released

2021-10-28 Thread Aki Tuomi
And by this, we of course mean an actual release for v0.5.17... Apologies for 
the confusion.

Aki Tuomi

> On 28/10/2021 12:12 Aki Tuomi  wrote:
> 
>  
> We are pleased to release first release candidate for v0.5.14. We have done 
> changes to packaging so please give us any feedback on how it works.
> 
> https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz
> https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz.sig
> Binary packages in https://repo.dovecot.org/
> Docker images in https://hub.docker.com/r/dovecot/dovecot
> 
> Kind regards,
> Aki Tuomi
> Open-Xchange oy
> 
> ---
> 
> - duplicate: The Sieve duplicate test is prone to false negatives when
>   the user receives many e-mails concurrently, meaning that duplicate
>   deliveries can still occur.
> - fileinto: v2.3.16 regression: Sieve delivery crashes if mail is
>   delivered to non-existing and existing folder.
> - imap-filter-sieve: v2.3.15 regression: The CPU limits on Sieve
>   execution are too easily exceeded in IMAP context (the IMAPSieve and
>   FILTER=SIEVE capabilities). Changed the default to unlimited CPU time
>   for IMAP context, since similar excessive resource usage can be caused
>   by other means as well. The CPU limits on Sieve scripts executed at
>   LDA/LMTP delivery are still enforced by default.
> - redirect:  The Sieve redirect action has protections against users
>   triggering mail loops. Unfortunately, the detection of a redirect mail
>   loop sometimes causes the message to get lost if no other Sieve action
>   is applied that delivers the message somewhere else.
> - redirect: v2.3.16 regression: With certain Sieve scripts if redirect
>   fails due to temporary failure, the lmtp process may crash after the
>   delivery. Fixes:
>   Panic: file mail-user.c: line 229 (mail_user_deinit):
>   assertion failed: ((*user)->refcount == 1).


Re: [Dovecot-news] Pigeonhole v0.5.17 released

2021-10-28 Thread Aki Tuomi
And by this, we of course mean an actual release for v0.5.17... Apologies for 
the confusion.

Aki Tuomi

> On 28/10/2021 12:12 Aki Tuomi  wrote:
> 
>  
> We are pleased to release first release candidate for v0.5.14. We have done 
> changes to packaging so please give us any feedback on how it works.
> 
> https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz
> https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz.sig
> Binary packages in https://repo.dovecot.org/
> Docker images in https://hub.docker.com/r/dovecot/dovecot
> 
> Kind regards,
> Aki Tuomi
> Open-Xchange oy
> 
> ---
> 
> - duplicate: The Sieve duplicate test is prone to false negatives when
>   the user receives many e-mails concurrently, meaning that duplicate
>   deliveries can still occur.
> - fileinto: v2.3.16 regression: Sieve delivery crashes if mail is
>   delivered to non-existing and existing folder.
> - imap-filter-sieve: v2.3.15 regression: The CPU limits on Sieve
>   execution are too easily exceeded in IMAP context (the IMAPSieve and
>   FILTER=SIEVE capabilities). Changed the default to unlimited CPU time
>   for IMAP context, since similar excessive resource usage can be caused
>   by other means as well. The CPU limits on Sieve scripts executed at
>   LDA/LMTP delivery are still enforced by default.
> - redirect:  The Sieve redirect action has protections against users
>   triggering mail loops. Unfortunately, the detection of a redirect mail
>   loop sometimes causes the message to get lost if no other Sieve action
>   is applied that delivers the message somewhere else.
> - redirect: v2.3.16 regression: With certain Sieve scripts if redirect
>   fails due to temporary failure, the lmtp process may crash after the
>   delivery. Fixes:
>   Panic: file mail-user.c: line 229 (mail_user_deinit):
>   assertion failed: ((*user)->refcount == 1).
___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news


Pigeonhole v0.5.17 released

2021-10-28 Thread Aki Tuomi
We are pleased to release first release candidate for v0.5.14. We have done 
changes to packaging so please give us any feedback on how it works.

https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz
https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz.sig
Binary packages in https://repo.dovecot.org/
Docker images in https://hub.docker.com/r/dovecot/dovecot

Kind regards,
Aki Tuomi
Open-Xchange oy

---

- duplicate: The Sieve duplicate test is prone to false negatives when
  the user receives many e-mails concurrently, meaning that duplicate
  deliveries can still occur.
- fileinto: v2.3.16 regression: Sieve delivery crashes if mail is
  delivered to non-existing and existing folder.
- imap-filter-sieve: v2.3.15 regression: The CPU limits on Sieve
  execution are too easily exceeded in IMAP context (the IMAPSieve and
  FILTER=SIEVE capabilities). Changed the default to unlimited CPU time
  for IMAP context, since similar excessive resource usage can be caused
  by other means as well. The CPU limits on Sieve scripts executed at
  LDA/LMTP delivery are still enforced by default.
- redirect:  The Sieve redirect action has protections against users
  triggering mail loops. Unfortunately, the detection of a redirect mail
  loop sometimes causes the message to get lost if no other Sieve action
  is applied that delivers the message somewhere else.
- redirect: v2.3.16 regression: With certain Sieve scripts if redirect
  fails due to temporary failure, the lmtp process may crash after the
  delivery. Fixes:
  Panic: file mail-user.c: line 229 (mail_user_deinit):
  assertion failed: ((*user)->refcount == 1).


signature.asc
Description: PGP signature


Dovecot v2.3.17 released

2021-10-28 Thread Aki Tuomi
We are pleased to release v2.3.17 of Dovecot.

Please note that 2.3.17 release will be the last one to support Debian/Stretch 
since Bullseye is now supported.

https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz.sig
Binary packages in https://repo.dovecot.org/
Docker images in https://hub.docker.com/r/dovecot/dovecot

Regards
Aki Tuomi
Open-Xchange oy

---

* Dovecot now logs a warning if time seems to jump forward at least
  100 milliseconds.
* dict: Lines logged by the dict process now contain the dict name as
  the prefix.
* lib-index: mail_cache_fields, mail_always_cache_fields and
  mail_never_cache_fields now verifies that the listed header names are
  valid. Especially the UTF8 "–" character has sometimes been wrongly
  used instead of the ASCII "-".
+ *-login: Added login_proxy_rawlog_dir setting to capture
  rawlogs between proxy and backend.
+ dict: The server process now keeps the last 10 idle dict backends
  cached for maximum of 30 seconds. Practically this acts as a
  connection pool for dict-redis and dict-ldap. Note that this doesn't
  affect dict-sql, because it already had its own internal cache.
+ doveadm: New stats add/remove commands added to support changing the
  metrics configuration on runtime.
+ lazy_expunge: Added lazy_expunge_exclude settings to disable
  lazy_expunge for specific folders. \Special-use flags can be used as
  folder names.
+ lib-lua: Added a new helper function dovecot.restrict_global_variables()
  to disable or enable defining new global variables.
- LAYOUT=index List index rebuild was missing.
- LAYOUT=index: Duplicate GUIDs were not detected.
- acl: When using acl_ignore_namespace Dovecot attempted to access or
  create dovecot-acl-list even when the namespace should have been
  ignored. For virtual namespaces this could have yielded errors about
  "Read-only file system" or "Permission denied".
- auth: Setting the "master" passdb field to empty value would
  cause proxying to fail with an authentication error.
  Now an empty "master" field is ignored.
- doveadm-server: Duplicate error lines were sent for failed commands.
  This didn't normally cause visible problems, except when using
  wildcards in usernames or -A parameter to go through multiple users.
- doveadm-server: Logs written by doveadm-server were often missing log
  prefixes, especially mail_log_prefix for mail commands. Logs sent to
  doveadm TCP client were also missing log prefixes.
- doveadm: v2.3 regression: batch command always crashes.
- doveadm: v2.3.11 regression: Commands failed if ssl_cert or
  ssl_key files weren't readable by the user running doveadm, even
  though doveadm didn't actually use these settings
- imap-hibernate: Process may crash at deinit:
  Panic: file ioloop.c: line 928 (io_loop_destroy): assertion failed:
  (ioloop->cur_ctx == NULL).
- imap: Using imap_fetch_failure=no-after can cause assert-crash
  with some IMAP commands if reading the mail fails (e.g. wrong cached
  mail size). Fixes:
  Panic: file index-mail-headers.c: line 198 (index_mail_parse_header_init):
  assertion failed: (!mail->data.header_parser_initialized)
- imap: v2.3.10 regression: When using INDEXPVT to enable private
  \Seen flags (for shared or public namespaces) the STORE command did
  not send untagged replies for the \Seen flag changes.
- imap: v2.3.15 regression: If PREVIEW/SNIPPET is not the final FETCH
  option in the command, the IMAP FETCH response is broken.
- imap: v2.3.15 regression: MOVE command leaks mailbox if it can't be
  opened and crashes at deinit:
  Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed:
  ((*user)->refcount == 1).
- imapc: Copying nonexistent mail via imapc could have crashed. Fixes:
  Panic: file mail-storage.c: line 2385 
(mailbox_transaction_commit_get_changes):
  assertion failed: (ret < 0 || seq_range_count(_r->saved_uids) == 
save_count ||
  array_count(_r->saved_uids) == 0).
- indexer: v2.3.15 regression: Process crashes if indexer-client
  disconnects while it's waiting for command reply. This happened for
  example if IMAP SEARCH triggered long fts indexing and the IMAP
  client disconnected while waiting for the reply.
- indexer: v2.3.15 regression: Process may have crashed in some situations.
- indexer: v2.3.15 regression: indexer-worker processes may not have
  reached the process_limit in some situations, possibly even using just
  one indexer-worker process even though there were many indexing
  requests queued.
- lib-compression: Reading lz4 compressed mdbox mails may crash. Fixes:
  Panic: file istream.c: line 345 (i_stream_read_memarea):
  assertion failed: (!stream->blocking).
- lib-compression: bench-compress crashes due to xz being read-only.
- lib-lua: Fix linking libdict_lua for non-GNU linkers when Lua support
  is disabled.
- lib-mail: There was no limit on how large an email header name could be.
  Processable header names are now limited to 1000 bytes.
- 

[Dovecot-news] Pigeonhole v0.5.17 released

2021-10-28 Thread Aki Tuomi
We are pleased to release first release candidate for v0.5.14. We have done 
changes to packaging so please give us any feedback on how it works.

https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz
https://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.17.tar.gz.sig
Binary packages in https://repo.dovecot.org/
Docker images in https://hub.docker.com/r/dovecot/dovecot

Kind regards,
Aki Tuomi
Open-Xchange oy

---

- duplicate: The Sieve duplicate test is prone to false negatives when
  the user receives many e-mails concurrently, meaning that duplicate
  deliveries can still occur.
- fileinto: v2.3.16 regression: Sieve delivery crashes if mail is
  delivered to non-existing and existing folder.
- imap-filter-sieve: v2.3.15 regression: The CPU limits on Sieve
  execution are too easily exceeded in IMAP context (the IMAPSieve and
  FILTER=SIEVE capabilities). Changed the default to unlimited CPU time
  for IMAP context, since similar excessive resource usage can be caused
  by other means as well. The CPU limits on Sieve scripts executed at
  LDA/LMTP delivery are still enforced by default.
- redirect:  The Sieve redirect action has protections against users
  triggering mail loops. Unfortunately, the detection of a redirect mail
  loop sometimes causes the message to get lost if no other Sieve action
  is applied that delivers the message somewhere else.
- redirect: v2.3.16 regression: With certain Sieve scripts if redirect
  fails due to temporary failure, the lmtp process may crash after the
  delivery. Fixes:
  Panic: file mail-user.c: line 229 (mail_user_deinit):
  assertion failed: ((*user)->refcount == 1).


signature.asc
Description: PGP signature
___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news


[Dovecot-news] Dovecot v2.3.17 released

2021-10-28 Thread Aki Tuomi
We are pleased to release v2.3.17 of Dovecot.

Please note that 2.3.17 release will be the last one to support Debian/Stretch 
since Bullseye is now supported.

https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.3.17.tar.gz.sig
Binary packages in https://repo.dovecot.org/
Docker images in https://hub.docker.com/r/dovecot/dovecot

Regards
Aki Tuomi
Open-Xchange oy

---

* Dovecot now logs a warning if time seems to jump forward at least
  100 milliseconds.
* dict: Lines logged by the dict process now contain the dict name as
  the prefix.
* lib-index: mail_cache_fields, mail_always_cache_fields and
  mail_never_cache_fields now verifies that the listed header names are
  valid. Especially the UTF8 "–" character has sometimes been wrongly
  used instead of the ASCII "-".
+ *-login: Added login_proxy_rawlog_dir setting to capture
  rawlogs between proxy and backend.
+ dict: The server process now keeps the last 10 idle dict backends
  cached for maximum of 30 seconds. Practically this acts as a
  connection pool for dict-redis and dict-ldap. Note that this doesn't
  affect dict-sql, because it already had its own internal cache.
+ doveadm: New stats add/remove commands added to support changing the
  metrics configuration on runtime.
+ lazy_expunge: Added lazy_expunge_exclude settings to disable
  lazy_expunge for specific folders. \Special-use flags can be used as
  folder names.
+ lib-lua: Added a new helper function dovecot.restrict_global_variables()
  to disable or enable defining new global variables.
- LAYOUT=index List index rebuild was missing.
- LAYOUT=index: Duplicate GUIDs were not detected.
- acl: When using acl_ignore_namespace Dovecot attempted to access or
  create dovecot-acl-list even when the namespace should have been
  ignored. For virtual namespaces this could have yielded errors about
  "Read-only file system" or "Permission denied".
- auth: Setting the "master" passdb field to empty value would
  cause proxying to fail with an authentication error.
  Now an empty "master" field is ignored.
- doveadm-server: Duplicate error lines were sent for failed commands.
  This didn't normally cause visible problems, except when using
  wildcards in usernames or -A parameter to go through multiple users.
- doveadm-server: Logs written by doveadm-server were often missing log
  prefixes, especially mail_log_prefix for mail commands. Logs sent to
  doveadm TCP client were also missing log prefixes.
- doveadm: v2.3 regression: batch command always crashes.
- doveadm: v2.3.11 regression: Commands failed if ssl_cert or
  ssl_key files weren't readable by the user running doveadm, even
  though doveadm didn't actually use these settings
- imap-hibernate: Process may crash at deinit:
  Panic: file ioloop.c: line 928 (io_loop_destroy): assertion failed:
  (ioloop->cur_ctx == NULL).
- imap: Using imap_fetch_failure=no-after can cause assert-crash
  with some IMAP commands if reading the mail fails (e.g. wrong cached
  mail size). Fixes:
  Panic: file index-mail-headers.c: line 198 (index_mail_parse_header_init):
  assertion failed: (!mail->data.header_parser_initialized)
- imap: v2.3.10 regression: When using INDEXPVT to enable private
  \Seen flags (for shared or public namespaces) the STORE command did
  not send untagged replies for the \Seen flag changes.
- imap: v2.3.15 regression: If PREVIEW/SNIPPET is not the final FETCH
  option in the command, the IMAP FETCH response is broken.
- imap: v2.3.15 regression: MOVE command leaks mailbox if it can't be
  opened and crashes at deinit:
  Panic: file mail-user.c: line 229 (mail_user_deinit): assertion failed:
  ((*user)->refcount == 1).
- imapc: Copying nonexistent mail via imapc could have crashed. Fixes:
  Panic: file mail-storage.c: line 2385 
(mailbox_transaction_commit_get_changes):
  assertion failed: (ret < 0 || seq_range_count(_r->saved_uids) == 
save_count ||
  array_count(_r->saved_uids) == 0).
- indexer: v2.3.15 regression: Process crashes if indexer-client
  disconnects while it's waiting for command reply. This happened for
  example if IMAP SEARCH triggered long fts indexing and the IMAP
  client disconnected while waiting for the reply.
- indexer: v2.3.15 regression: Process may have crashed in some situations.
- indexer: v2.3.15 regression: indexer-worker processes may not have
  reached the process_limit in some situations, possibly even using just
  one indexer-worker process even though there were many indexing
  requests queued.
- lib-compression: Reading lz4 compressed mdbox mails may crash. Fixes:
  Panic: file istream.c: line 345 (i_stream_read_memarea):
  assertion failed: (!stream->blocking).
- lib-compression: bench-compress crashes due to xz being read-only.
- lib-lua: Fix linking libdict_lua for non-GNU linkers when Lua support
  is disabled.
- lib-mail: There was no limit on how large an email header name could be.
  Processable header names are now limited to 1000 bytes.
- 

Re: Design Check

2021-10-28 Thread Felix Ingram
On Thu, 28 Oct 2021 at 00:15,  wrote:

> > I think your approach would work, however, if I set
> > up aliases similar to:
> >
> > @barbaz.mydomain.com -> bar...@mydomain.com.
> >
> > I believe I can do that in postfix with some regex magic.
>
> Yes, that would work perfectly without any regex.
> You just point the catchall alias to the "user".
> @barbaz.mydomain.com -> bar...@mydomain.com
>
>
I've managed to get this working in postfix - I needed the regex rather
than
a static map, as I need to extract the unknown subdomain portion but it
seems
to be working. I have been able to get postfix to save it to a file as well
and it seems
to work as I expected.


> [..]
> > The purpose of the system is that users can create disposable/temporary
> > email addresses for various testing jobs.
>
> Are you aware of postfix recipient_delimiter? It allows for disposable /
> wild card addresses. If enabled in postfix, you setup a mailbox user
> like bar...@mydomain.com and any address with that user and the
> delimiter would still get delivered to that user.
>
> bar...@mydomain.com -> bar...@mydomain.com
> barbaz+randomt...@mydomain.com -> bar...@mydomain.com
> barbaz+te...@mydomain.com -> bar...@mydomain.com
>
> You can change the + to any symbol you want postfix to look out for.
>

We were using this approach on a different domain but our issue was that we
have
multiple people on the same piece of work and so they needed to share
access to
all of the mails. We decided on the approach I'm describing as we also
wanted to
have control at the DNS level to do this such as expiring addresses.


>
> > I think my "creating users" was me wanting to make sure that when
> > postfix
> > passes an email for "bar...@mydomain.com" to Dovecot, then Dovecot will
> > store it and wait for
> > someone to come along and impersonate barbaz. i.e. "barbaz" doesn't
> > have to exist as a user
> > already before Dovecot will store the mail.
>
> If you are using LMTP dovecot will only accept emails from postfix that
> it can lookup the /directory/path to from one of the userdb{} or
> passdb{} sections. If dovecot can not find a match in any of the
> userdb{} or passdb{} it will reject the email as user unknown causing
> postfix to send a undeliverable notice email back to the envelope sender
> address, also known as back-scatter. I am not aware of a way to use
> wildcard addresses in dovecot userdb{}, i don't think its possible but i
> don't know what i don't know.
>

So I think this will be the main issue now - there's no way of knowing the
addresses ahead of time, so it sounds like I'll need to add them to
userdb{} when they
hit postfix and before they get passed to dovecot.

For my sins I'm building this on Kubernetes so dovecot is on a separate
"machine" at the
moment. The userdb will be in postgres, as I'm using that for other things,
so I guess I'll need
to update that in postfix somewhere. This is a fairly low volume system, so
I can probably take
the hit of a DB query per email.

Currently postfix doesn't even seem to be attempting to talk to dovecot but
that's one for the
postfix list.

Thanks again for the help.

Cheers,

Felix


Re: Design Check

2021-10-28 Thread Felix Ingram
On Thu, 28 Oct 2021 at 07:40, Bernardo Reino  wrote:

>
> > [...]
>
> Further to the responses you have received already, I'd like to note that
> if you
> want to receive mail at {alias}@{user}.mydomain.com then, at the time of
> *sending* the e-mail there needs to be an MX record for user, as otherwise
> the
> sender won't be able to connect to your (postfix) server.
>
> That means that the users will have to exist *before* postfix receives the
> message, and thus clearly before dovecot receives it.. so you may have to
> reconsider your requirement of adding users of on-the-fly.
>

Excellent point. So we are also using a DNS server with this (CoreDNS with
some custom plugins).
This means we are able to handle the dynamic MX records (plus also allowing
us to expire domains if
we need to).

Cheers,

Felix


Re: download from pigeonhole.dovecot.org not possible via IPv6 http

2021-10-28 Thread Aki Tuomi


> On 19/10/2021 12:57 Wolfgang Zenker  wrote:
> 
>  
> Hello,
> 
> I tried to download the dovecot-2.3-pigeonhole-0.5.15.tar.gz source
> archive from
> http://pigeonhole.dovecot.org/releases/2.3/dovecot-2.3-pigeonhole-0.5.15.tar.gz
>  on an IPv6-only server, but the download fails.
> The server pigeonhole.dovecot.org has an IPv6 address published in DNS
> that is actually pingable, but connection on the http-port is not possible.
> Connecting on https does work, as does http AND https on IPv4.
> This appears to be a simple configuration problem and should be easily
> fixed; the correct behaviour would be to redirect from http to https on
> IPv6 the same way it is done for IPv4 already.
> 
> Greetings,
> Wolfgang

Hi!

This has been now fixed, thanks for reporting.

Aki


Re: Design Check

2021-10-28 Thread Bernardo Reino

On Wed, 27 Oct 2021, Felix Ingram wrote:


[...]

People would be able to send email to addresses that match the following
format:

us...@foobar.mydomain.com
us...@foobar.mydomain.com

us...@barbaz.mydomain.com
us...@barbaz.mydomain.com

[...]

I will be creating a web interface for users to get/set their credentials,
so can add those users on an adhoc basis, but I will need to have the
"foobar", "barbaz", etc users created whenever an email arrives (we won't
know ahead of time).

[...]


Further to the responses you have received already, I'd like to note that if you 
want to receive mail at {alias}@{user}.mydomain.com then, at the time of 
*sending* the e-mail there needs to be an MX record for user, as otherwise the 
sender won't be able to connect to your (postfix) server.


That means that the users will have to exist *before* postfix receives the 
message, and thus clearly before dovecot receives it.. so you may have to 
reconsider your requirement of adding users of on-the-fly.


Of course, you could use a wildcard MX, but my understanding is that this can 
cause problems (but I'd have to check in RFC1912 and RFC4592).


Cheers.