Purges now happen 5 times faster

[Steve Litt via dovecot]

Hi all,

I use Claws-Mail to look inside my local (same metal) Dovecot server.
All incoming mail is dumped into Dovecot folders via procmail.

All of a sudden today, mass purges of "marked for delete" happen about
5 times faster. The process used to be a significant productivity
destroyer, but now is easy to work with. In other words, it's a good
thing, but until I know why this happened, I'll have that little voice
inside telling me there's some evil side effect somewhere. 

Does anybody know any reason this should have happened? I'm using
Dovecot 2.3.5 (513208660) and Claws-Mail 3.17.3.

Thanks,

SteveT

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Jason Lewis via dovecot]

Hi Hendrik,

Hendrik Boom via dovecot wrote on 29/3/19 4:03 am:
> On Wed, Mar 27, 2019 at 10:25:02AM +1100, Jason Lewis via dovecot wrote:
>> Hi Aki,
>>
>> debian jessie backports has been moved to archive.debian.org and
>> initially I was unable to install dovecot-dbg because of that. But I've
>> managed to resolve that issue now.
> 
> Just curious -- what deb line did you use in /etc/apt/sources.lst to 
> refer to the archived repositories? 
> 
> -- hendrik
> 


my sources.list:

deb http://deb.debian.org/debian jessie main contrib non-free
deb http://archive.debian.org/debian jessie-backports main contrib non-free
deb  http://security.debian.org jessie/updates main contrib non-free

deb-src http://deb.debian.org/debian jessie   main contrib non-free
deb-src http://archive.debian.org/debian jessie-backports main contrib
non-free
deb-src http://security.debian.org jessie/updates main contrib non-free


Jason


-- 
Jason Lewis
http://emacstragic.net

Apparmor problem

[Ervin Hegedüs via dovecot]

Hi there,

I know this isn't a Dovecot issue, but hope that somebody can helps me.

I've successfully installed and configured Dovecot to a Debian 9 server.
Looks like everything works as well, I just see a line in the log when I
send a mail:

Mar 28 22:21:47 mailng kernel: [3150146.825007] audit: type=1400
audit(1553808107.757:286204): apparmor="DENIED" operation="open"
profile="/usr/lib/dovecot/dovecot-lda"
name="/usr/share/dovecot/protocols.d/" pid=26197 comm="doveconf"
requested_mask="r" denied_mask="r" fsuid=5000 ouid=0

The /usr/share/dovecot/protocols.d/ directory and its content
(.../protocols.d/**) set up in usr.lib.dovecot.dovecot-lda apparmor profile
with mask "r", but this line every time comes, when I send a mail through
webmail - think when the client copies the message to the Sent folder
(anyway, the sent messages are in Sent folder after sending, so everything
works).

Do you have any idea?


Thanks,

a.

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Robert Kudyba via dovecot]

 Set
 
 ssl_client_ca_file=/path/to/cacert.pem to validate the certificate 
>>> 
>>> Can this be the Lets Encrypt cert that we already have? In other words we 
>>> have:
>>> ssl_cert = >> ssl_key = >> 
>>> Can those be used?
>> 
>> Set it to *CA* cert. You can also use
>> 
>> ssl_client_ca_file=/etc/pki/tls/ca-bundle crt (on centos) 

OK did that.

>> ssl_client_ca_dir=/etc/ssl/certs (on debian based)
 Are you using haproxy or something in front of dovecot?
>>> 
>>> No. Just Squirrelmail webmail with sendmail.
>>> 
>> Maybe squirrelmail supports forwarding original client ip with ID command. 
>> Otherwise dovecot cannot know it. Or you could configure squirrelmail to use 
>> weakforced ?

I see some options in http://squirrelmail.org/docs/admin/admin-5.html#ss5.3 
. Would it be a plugin?

> Also check that auth_policy_request_attributes use %{rip} and not 
> %{real_rip}. You can see this with 
> 
> `doveconf auth_policy_request_attributes`

Yes I’ve confirmed it matches. Still getting the URL or IP of the webmail 
address as well as errors like SSL handshake to ex.ter.na.lip:8084 failed: 
Connection closed

Mar 28 16:13:36 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Timeout (now: 2019-03-28 16:13:36.300)
Mar 28 16:13:36 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Absolute timeout expired for request [Req10: POST 
https://ourdomain:8084/?command=allow] (Request queued 2.002 secs ago, not yet 
sent, 0.000 in other ioloops)
Mar 28 16:13:36 auth: Debug: http-client[1]: request [Req10: POST 
https://ourdomain:8084/?command=allow]: Error: 9008 Absolute request timeout 
expired (Request queued 2.002 secs ago, not yet sent, 0.000 in other ioloops)
Mar 28 16:13:36 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Dropping request [Req10: POST https://ourdomain:8084/?command=allow]
Mar 28 16:13:36 auth: Error: policy(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): Policy 
server HTTP error: Absolute request timeout expired (Request queued 2.002 secs 
ago, not yet sent, 0.000 in other ioloops)
Mar 28 16:13:36 auth: Debug: http-client[1]: request [Req10: POST 
https://ourdomain:8084/?command=allow]: Destroy (requests left=1)
Mar 28 16:13:36 auth: Debug: http-client[1]: request [Req10: POST 
https://ourdomain:8084/?command=allow]: Free (requests left=0)
Mar 28 16:13:36 auth-worker(32249): Debug: 
pam(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): lookup service=dovecot
Mar 28 16:13:36 auth-worker(32249): Debug: 
pam(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): #1/1 style=1 msg=Password: 
Mar 28 16:13:38 auth-worker(32249): Info: 
pam(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): unknown user
Mar 28 16:13:38 auth: Debug: policy(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): Policy 
request https://ourdomain:8084/?command=report
Mar 28 16:13:38 auth: Debug: policy(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): Policy 
server request JSON: 
{"device_id":"","login":"abc","protocol":"imap","pwhash":"00","remote":"127.0.0.1","success":false,"policy_reject":false,"tls":false}
Mar 28 16:13:38 auth: Debug: http-client[1]: queue https://ourdomain:8084: Set 
request timeout to 2019-03-28 16:13:40.625 (now: 2019-03-28 16:13:38.625)
Mar 28 16:13:38 auth: Debug: http-client: peer ex.ter.na.lip:8084 (shared): 
Peer reused
Mar 28 16:13:38 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Setting up connection to ex.ter.na.lip:8084 (SSL=ourdomain) (1 requests pending)
Mar 28 16:13:38 auth: Debug: http-client[1]: request [Req11: POST 
https://ourdomain:8084/?command=report]: Submitted (requests left=1)
Mar 28 16:13:38 auth: Debug: http-client[1]: peer ex.ter.na.lip:8084: Creating 
1 new connections to handle requests (already 0 usable, connecting to 0, 
closing 0)
Mar 28 16:13:40 auth: Debug: client passdb out: FAIL1   user=abc
Mar 28 16:13:40 imap-login: Info: Aborted login (auth failed, 1 attempts in 6 
secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 
session=<5aBSMC2FROF/AAAB>
Mar 28 16:13:40 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Timeout (now: 2019-03-28 16:13:40.626)
Mar 28 16:13:40 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Absolute timeout expired for request [Req11: POST 
https://ourdomain:8084/?command=report] (Request queued 2.000 secs ago, not yet 
sent, 0.000 in other ioloops)
Mar 28 16:13:40 auth: Debug: http-client[1]: request [Req11: POST 
https://ourdomain:8084/?command=report]: Error: 9008 Absolute request timeout 
expired (Request queued 2.000 secs ago, not yet sent, 0.000 in other ioloops)
Mar 28 16:13:40 auth: Debug: http-client[1]: queue https://ourdomain:8084: 
Dropping request [Req11: POST https://ourdomain:8084/?command=report]
Mar 28 16:13:40 auth: Error: policy(abc,127.0.0.1,<5aBSMC2FROF/AAAB>): Policy 
server HTTP error: Absolute request timeout expired (Request queued 2.000 secs 
ago, not yet sent, 0.000 in other ioloops)
Mar 28 16:13:40 auth: Debug: http-client[1]: request [Req11: POST 

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 22:02 Aki Tuomi via dovecot  wrote:
   
   

   
   

   
   

   
   

 On 28 March 2019 21:52 Robert Kudyba  wrote:


 


 


 
  
   
Set
   
   

   
   
ssl_client_ca_file=/path/to/cacert.pem to validate the certificate 
   
  
 
 
  
 
 
  Can this be the Lets Encrypt cert that we already have? In other words we have:
 
 
  
   ssl_cert = 
  
  
   ssl_key = 
  
 
 
  
 
 
  Can those be used?
 

   
   

   
   
Set it to *CA* cert. You can also use
   
   

   
   
ssl_client_ca_file=/etc/pki/tls/ca-bundle crt (on centos) 
   
   

   
   
or
   
   

   
   
ssl_client_ca_dir=/etc/ssl/certs (on debian based)
   
   

 
  
   
Are you using haproxy or something in front of dovecot?
   
  
 
 


 No. Just Squirrelmail webmail with sendmail.


   
   
Maybe squirrelmail supports forwarding original client ip with ID command. Otherwise dovecot cannot know it. Or you could configure squirrelmail to use weakforced ?
   
   
---
Aki Tuomi
   
  
  
   Also check that auth_policy_request_attributes use %{rip} and not %{real_rip}. You can see this with 
  
  
   
  
  
   `doveconf auth_policy_request_attributes`
  
  
   ---
Aki Tuomi
   
 

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 21:52 Robert Kudyba  wrote:
   
   

   
   

   
   

 
  
   Set
  
  
   
  
  
   ssl_client_ca_file=/path/to/cacert.pem to validate the certificate 
  
 


 


 Can this be the Lets Encrypt cert that we already have? In other words we have:


 
  ssl_cert = 
 
 
  ssl_key = 
 


 


 Can those be used?

   
  
  
   
  
  
   Set it to *CA* cert. You can also use 
  
  
   
  
  
   ssl_client_ca_file=/etc/pki/tls/ca-bundle crt (on centos) 
  
  
   
  
  
   or
  
  
   
  
  
   ssl_client_ca_dir=/etc/ssl/certs (on debian based)
  
  
   

 
  
   Are you using haproxy or something in front of dovecot?
  
 


   
   
No. Just Squirrelmail webmail with sendmail.
   
   
  
  
   Maybe squirrelmail supports forwarding original client ip with ID command. Otherwise dovecot cannot know it. Or you could configure squirrelmail to use weakforced ?
  
  
   ---
Aki Tuomi
   
 

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Robert Kudyba via dovecot]

> Set
> 
> ssl_client_ca_file=/path/to/cacert.pem to validate the certificate 

Can this be the Lets Encrypt cert that we already have? In other words we have:
ssl_cert =  Are you using haproxy or something in front of dovecot?

No. Just Squirrelmail webmail with sendmail.

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 21:31 Robert Kudyba  wrote:
   
   

   
   

   
   

 
  On Mar 28, 2019, at 10:29 AM, Aki Tuomi via dovecot <
  dovecot@dovecot.org> wrote:
 
 
  
   

   
   

 On 28 March 2019 16:08 Robert Kudyba via dovecot <
 dovecot@dovecot.org> wrote:


 


 


 
  dovecot-2.3.3-1.fc29.x86_64
 


 


 
  Mar 28 10:04:47 auth: Panic: file http-client-request.c: line 283 (http_client_request_unref): assertion failed: (req->refcount > 0)
 
 
  Mar 28 10:04:47 auth: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xe34fb) [0x7fe76e0834fb] -> /usr/lib64/dovecot/libdovecot.so.0(+0xe3597) [0x7fe76e083597] -> /usr/lib64/dovecot/libdovecot.so.0(+0x51207) [0x7fe76dff1207] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4972b) [0x7fe76dfe972b] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_destroy+0x107) [0x7fe76e02cf87] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_deinit+0x4c) [0x7fe76e03b9ec] -> dovecot/auth(auth_policy_deinit+0x1e) [0x55facfdb350e] -> dovecot/auth(main+0x3e1) [0x55facfdae3c1] -> /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fe76dd93413] -> dovecot/auth(_start+0x2e) [0x55facfdae57e]
 
 
  Mar 28 10:04:47 auth: Fatal: master: service(auth): child 31162 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)
 
 
  Mar 28 10:04:48 master: Info: Dovecot v2.3.3 (dcead646b) starting up for imap, pop3
 


 

   
   
Hi,
   
   

   
   
this is a known issue as DOV-3019 and we are fixing this. It happens during auth process shutdown if there are pending requests.
   
  
 

   
   

   Another issue is that the dovecot logs always report the offending URL or IP as what’s in 
   /etc/dovecot/conf.d/95-auth.conf
    in our case:
   
auth_policy_server_url = 
https://ourdomain:8084/

 
  
 
 
  These are HTTP errors in the logs:
 
 
  
 
 
  Mar 28 09:58:04 auth: Debug: client in: AUTH 
  1 
  PLAIN 
  service=imap 
  secured 
  session=lmNw8SeFoMl/AAAB 
  lip=127.0.0.1 
  rip=127.0.0.1 
  lport=143 
  rport=51616 
  resp=
 
 
  Mar 28 09:58:04 auth: Debug: policy(unclroot,127.0.0.1,): Policy request https://ourdomain:8084/?command=allow
 
 
  Mar 28 09:58:04 auth: Debug: policy(unclroot,127.0.0.1,): Policy server request JSON: {"device_id":"","login":"unclroot","protocol":"imap","pwhash":"68","remote":"127.0.0.1","tls":false}
 
 
  Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST https://ourdomain:8084/?command=allow]: Error: 9003 Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings)
 
 
  Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST https://ourdomain:8084/?command=allow]: Submitted (requests left=3)
 
 
  Mar 28 09:58:04 auth: Error: policy(unclroot,127.0.0.1,): Policy server HTTP error: Couldn't initialize SSL context: Can't verify remote server certs without trusted CAs (ssl_client_ca_* settings)
 
 
  Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST https://ourdomain:8084/?command=allow]: Destroy (requests left=3)
 
 
  Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST https://ourdomain:8084/?command=allow]: Free (requests left=2)
 


 

   
   

   
   
So wforce is always recording the “bad” IP as 127.0.0.1 or the FQDN, and not the actual user IP. Is there another place to set this?
   
   

   
   
Perhaps I have to set this in wforce.conf?
   
   
webserver("0.0.0.0:8084", “ourpassword")
   
  
  
   
  
  
   Set
  
  
   
  
  
   ssl_client_ca_file=/path/to/cacert.pem to validate the certificate 
  
  
   
  
  
   Are you using haproxy or something in front of dovecot?
  
  
   ---
Aki Tuomi
   
 

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Robert Kudyba via dovecot]

> On Mar 28, 2019, at 10:29 AM, Aki Tuomi via dovecot  
> wrote:
> 
>> On 28 March 2019 16:08 Robert Kudyba via dovecot  wrote:
>> 
>> 
>> dovecot-2.3.3-1.fc29.x86_64
>> 
>> Mar 28 10:04:47 auth: Panic: file http-client-request.c: line 283 
>> (http_client_request_unref): assertion failed: (req->refcount > 0)
>> Mar 28 10:04:47 auth: Error: Raw backtrace: 
>> /usr/lib64/dovecot/libdovecot.so.0(+0xe34fb) [0x7fe76e0834fb] -> 
>> /usr/lib64/dovecot/libdovecot.so.0(+0xe3597) [0x7fe76e083597] -> 
>> /usr/lib64/dovecot/libdovecot.so.0(+0x51207) [0x7fe76dff1207] -> 
>> /usr/lib64/dovecot/libdovecot.so.0(+0x4972b) [0x7fe76dfe972b] -> 
>> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_destroy+0x107) 
>> [0x7fe76e02cf87] -> 
>> /usr/lib64/dovecot/libdovecot.so.0(http_client_deinit+0x4c) [0x7fe76e03b9ec] 
>> -> dovecot/auth(auth_policy_deinit+0x1e) [0x55facfdb350e] -> 
>> dovecot/auth(main+0x3e1) [0x55facfdae3c1] -> 
>> /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fe76dd93413] -> 
>> dovecot/auth(_start+0x2e) [0x55facfdae57e]
>> Mar 28 10:04:47 auth: Fatal: master: service(auth): child 31162 killed with 
>> signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps 
>> 
>>  - set /proc/sys/fs/suid_dumpable to 2)
>> Mar 28 10:04:48 master: Info: Dovecot v2.3.3 (dcead646b) starting up for 
>> imap, pop3
>> 
> Hi,
> 
> this is a known issue as DOV-3019 and we are fixing this. It happens during 
> auth process shutdown if there are pending requests.


Another issue is that the dovecot logs always report the offending URL or IP as 
what’s in /etc/dovecot/conf.d/95-auth.conf in our case:
auth_policy_server_url = https://ourdomain:8084/ 


These are HTTP errors in the logs:

Mar 28 09:58:04 auth: Debug: client in: AUTH1   PLAIN   service=imap
secured session=lmNw8SeFoMl/AAABlip=127.0.0.1   rip=127.0.0.1   
lport=143   rport=51616 resp=
Mar 28 09:58:04 auth: Debug: policy(unclroot,127.0.0.1,): 
Policy request https://ourdomain:8084/?command=allow 

Mar 28 09:58:04 auth: Debug: policy(unclroot,127.0.0.1,): 
Policy server request JSON: 
{"device_id":"","login":"unclroot","protocol":"imap","pwhash":"68","remote":"127.0.0.1","tls":false}
Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST 
https://ourdomain:8084/?command=allow]: 
 Error: 9003 Couldn't 
initialize SSL context: Can't verify remote server certs without trusted CAs 
(ssl_client_ca_* settings)
Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST 
https://ourdomain:8084/?command=allow]: 
 Submitted (requests 
left=3)
Mar 28 09:58:04 auth: Error: policy(unclroot,127.0.0.1,): 
Policy server HTTP error: Couldn't initialize SSL context: Can't verify remote 
server certs without trusted CAs (ssl_client_ca_* settings)
Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST 
https://ourdomain:8084/?command=allow]: 
 Destroy (requests left=3)
Mar 28 09:58:04 auth: Debug: http-client[1]: request [Req11: POST 
https://ourdomain:8084/?command=allow]: 
 Free (requests left=2)


So wforce is always recording the “bad” IP as 127.0.0.1 or the FQDN, and not 
the actual user IP. Is there another place to set this?

Perhaps I have to set this in wforce.conf?
webserver("0.0.0.0:8084", “ourpassword")

Re: Hibernation and proxy

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 20:12 
azu...@pobox.sk wrote:
   
   

   
   

   
   
Citát Aki Tuomi <
aki.tu...@open-xchange.com>:
   
   

   
   
>> On 28 March 2019 17:46 azurit--- via dovecot < 
dovecot@dovecot.org>
   
   
>> wrote:
   
   
>>
   
   
>>
   
   
>>
   
   
>>
   
   
>> Hi,
   
   
>>
   
   
>>
   
   
>> does hibernation works well with proxy? Are proxy connections
   
   
>> hibernated or not?
   
   
>>
   
   
>>
   
   
>> azur
   
   
>
   
   

 hibernation only happens on backends.


 --- Aki Tuomi

   
   

   
   

   
   

   
   
And if i have mixed server? Some users are local, some are proxied away.
   
  
  
   Local ones can be hibernated. Hibernation works by moving the connection into hibernation process and freeing the backend process until such time it is needed.
  
  
   ---
Aki Tuomi
   
 

Re: Hibernation and proxy

[azurit--- via dovecot]

Citát Aki Tuomi :


On 28 March 2019 17:46 azurit--- via dovecot < dovecot@dovecot.org>
wrote:




Hi,


does hibernation works well with proxy? Are proxy connections
hibernated or not?


azur


   hibernation only happens on backends.
   --- Aki Tuomi





And if i have mixed server? Some users are local, some are proxied away.

Re: Regression ACL & namespace prefix

[Michal Hlavinka via dovecot]

Hi,

were you able to reproduce this problem? Do you need more information to 
reproduce this?

Cheers,
Michal Hlavinka

On 3/12/19 3:29 PM, Michal Hlavinka wrote:

Hi,

thanks for the answer. I think your environment was not set up correctly
to reproduce this bug. I've retested with 2.3.5 and I can still
reproduce it. I've attached a script that will configure everything for
testing and if you have a virtual machine available, you can use it
directly (it expects linux with systemd for dovecot restart).

relevant section from config:
namespace {
   hidden = no
   list = yes
   location = maildir:/var/mail/pub
   prefix = pub/
   separator = /
   type = public
}

this expects maildir directly in pub:
/var/mail/pub/cur
/var/mail/pub/new
/var/mail/pub/tmp

as it uses '/' separator and there could be subfolders, it should look
for .DEFAULT file in global acls directory which it does not in your
debug output

doveadm(testuser): Info: Mailbox '' is in namespace 'pub/'
doveadm(testuser): Info: All message flags are shared across users in 
mailbox
doveadm(testuser): Debug: acl vfile: file 
/etc/dovecot/global-acls//.DEFAULT

not found
doveadm(testuser): Debug: acl vfile: file /var/mail/pub/dovecot-acl not 
found

doveadm(testuser): Info: User testuser has no rights for mailbox
doveadm(testuser): Error: User testuser is missing 'lookup' right
doveadm(testuser): Info: Mailbox pub is NOT visible in LIST

in this output see that it checks this location:
acl vfile: file /etc/dovecot/global-acls//.DEFAULT not found

instead of

/etc/dovecot/global-acls/pub/.DEFAULT

this is caused by line in
src/plugins/acl/acl-backend-vfile.c: acl_backend_vfile_object_init(...)

vname = *name == '\0' ? "" :
  mailbox_list_get_vname(_backend->list, name);

and because name is empty, it will not use the "pub" prefix in the path.
If I'd test acl for "pub/subfolder" that condition would have different
result and bug would not trigger:

doveadm(testuser): Debug: acl vfile: reading file
/etc/dovecot/global-acls/pub/subfolder/.DEFAULT


For testing I use this acl configuration:
cat /etc/dovecot/global-acls/pub/.DEFAULT
user=testuser l

but as this acl file location is not found by dovecot, content should
not matter.


Cheers,
Michal Hlavinka


On 3/7/19 7:00 PM, Aki Tuomi via dovecot wrote:

I tested with release 2.3.5, and

doveadm -Dv acl debug -u testuser pub doveadm(testuser): Debug: acl
vfile: file /etc/dovecot/global-acls/pub/INBOX not found 
doveadm(testuser): Debug: acl vfile: file

/home/vmail/pub/Mail/mailboxes/INBOX/dbox-Mails/dovecot-acl not
found doveadm(testuser): Debug: acl vfile: file
/etc/dovecot/global-acls/ not found doveadm(testuser): Debug: acl
vfile: file /home/vmail/pub/Mail/mailboxes/dovecot-acl not found

so our advice is to upgrade into 2.3.5, as 2.2.36 is no longer in
development.

Aki


On 7 March 2019 19:47 Aki Tuomi via dovecot 
wrote:


Sorry, we have not yet been able to look into this..

It's now in our internal system as DOP-966

Aki


On 7 March 2019 17:31 Michal Hlavinka via dovecot
 wrote:


Hi, any progress with this issue? Do you need more information to
debug and fix this?

Cheers Michal Hlavinka

On 9/18/18 4:10 PM, Michal Hlavinka wrote:

Hi

tl;dr: Seems that for Global ACL directory, namespace prefix is
not part of the path, when looking for acl file.

Long version:

We're planning to update dovecot in next os update to 2.2.36
and while going through regression testing, we found a problem
with ACL configuration combined with namespace.

Test uses "Global ACL directory" configuration.

Relevant configuration part: mail_location = maildir:~/Maildir

namespace inbox { hidden = no inbox = yes list = yes location
= prefix = separator = / } namespace { hidden = no list = yes 
location = maildir:/var/mail/pub prefix = pub/ separator = / type = 
public }


mail_plugins = acl

protocol imap { mail_plugins = $mail_plugins acl imap_acl } plugin 
{ acl = vfile:/etc/dovecot/global-acls }


ACL config file is stored at: /etc/dovecot/global-acls/pub/.DEFAULT

when trying to examine "pub", it is denied: fetchmail: IMAP>
A0005 EXAMINE "pub" fetchmail: IMAP< A0005 NO Mailbox doesn't
exist: pub (0.001 + 0.000 secs).

# doveadm acl debug -u d2 pub doveadm(d2): Info: Mailbox '' is
in namespace 'pub/' doveadm(d2): Info: Mailbox path:
/var/mail/pub doveadm(d2): Info: All message flags are shared
across users in mailbox doveadm(d2): Info: User d2 has no
rights for mailbox doveadm(d2): Error: User d2 is missing
'lookup' right doveadm(d2): Info: Mailbox pub is NOT visible in
LIST

because it did not find acl file: imap(d2): Debug: Namespace :
type=public, prefix=pub/, sep=/, inbox=no, hidden=no, list=yes,
subscriptions=yes location=maildir:/var/mail/pub imap(d2):
Debug: maildir++: root=/var/mail/pub, index=, indexpvt=, control=, 
inbox=, alt= imap(d2): Debug: acl: initializing

backend with data: vfile:/etc/dovecot/global-acls imap(d2):
Debug: acl: acl username = d2 imap(d2): Debug: acl: owner = 0 
imap(d2): Debug: acl 

Re: pigeonhole tests crashing in deleteheader.svtest

[Aki Tuomi via dovecot]

> On 28 March 2019 19:40 Michal Hlavinka via dovecot  
> wrote:
> 
>  
> Hi,
> 
> when trying to build dovecot 2.3.5.1 pigeonhole testsuite crashes in
> 

Which version of pigeonhole are you using?

Aki

pigeonhole tests crashing in deleteheader.svtest

[Michal Hlavinka via dovecot]

Hi,

when trying to build dovecot 2.3.5.1 pigeonhole testsuite crashes in

Test case: ./tests/extensions/editheader/deleteheader.svtest:

 1: Test 'Deleteheader - nonexistent' SUCCEEDED
 2: Test 'Deleteheader - nonexistent (match)' SUCCEEDED
 3: Test 'Deleteheader - one' SUCCEEDED
 4: Test 'Deleteheader - two (first)' SUCCEEDED
 5: Test 'Deleteheader - two (last)' SUCCEEDED
 6: Test 'Deleteheader - :index' SUCCEEDED
 7: Test 'Deleteheader - :index :last' SUCCEEDED
 8: Test 'Deleteheader - implicit keep' SUCCEEDED
make: *** [Makefile:1190: 
tests/extensions/editheader/deleteheader.svtest] Segmentation fault 
(core dumped)


Let me know if you need more debug information.
Cheers
Michal Hlavinka

#0  0x7f9924faea8a in __strlen_sse2 () from /lib64/libc.so.6
#1  0x7f992514fdd2 in smtp_address_clone (pool=0x7f992528dce0 
, src=src@entry=0x7ffd30eace10) at smtp-address.c:632
#2  0x55f7330916b3 in testsuite_message_set_data 
(mail=0x55f73486f848) at testsuite-message.c:96
#3  0x55f7330918b9 in testsuite_message_set_file 
(renv=renv@entry=0x55f734856ac0, file_path=) at 
testsuite-message.c:139
#4  0x55f7330931cc in testsuite_smtp_get 
(renv=renv@entry=0x55f734856ac0, index=) at 
testsuite-smtp.c:171
#5  0x55f73309566c in cmd_test_message_smtp_operation_execute 
(renv=0x55f734856ac0, address=) at cmd-test-message.c:404
#6  0x7f992543f371 in sieve_interpreter_operation_execute 
(interp=0x55f734856a68) at sieve-interpreter.c:911
#7  sieve_interpreter_continue (interp=0x55f734856a68, interrupted=0x0) 
at sieve-interpreter.c:949
#8  0x7f992543f5d1 in sieve_interpreter_run (interp=0x55f734856a68, 
result=) at sieve-interpreter.c:999
#9  0x55f73308f670 in testsuite_run (msgdata=, 
ehandler=, senv=0x7ffd30eacfa0, sbin=) at 
testsuite.c:73

#10 main (argc=, argv=) at testsuite.c:203

(gdb) up
#1  0x7f992514fdd2 in smtp_address_clone (pool=0x7f992528dce0 
, src=src@entry=0x7ffd30eace10) at smtp-address.c:632

632 dsize = strlen(src->domain) + 1;

(gdb) p *src
$4 = {localpart = 0x55f73480cdc8 
"/tmp/dsieve-testsuite.1553794316.30960/smtp/0.eml", domain = 0x1 
}

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Hendrik Boom via dovecot]

On Wed, Mar 27, 2019 at 10:25:02AM +1100, Jason Lewis via dovecot wrote:
> Hi Aki,
> 
> debian jessie backports has been moved to archive.debian.org and
> initially I was unable to install dovecot-dbg because of that. But I've
> managed to resolve that issue now.

Just curious -- what deb line did you use in /etc/apt/sources.lst to 
refer to the archived repositories? 

-- hendrik

Re: CVE-2019-7524 backport patch for 2.2.33.2

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 17:11 Gerald Galster via dovecot <
dovecot@dovecot.org> wrote:
   
   

   
   

   
   
Hello Aki,
   
   

   
   
I'm currently stuck with 2.2.33.2 as 2.2.36 still duplicates mails after pop3 deletion on a two node dsync cluster.
   
   

   
   
Therefore I've created a small patch and it seems only these two files are affected:
   
   

   
   
dovecot-2.2.36.3/src/lib-storage/index/index-pop3-uidl.c
   
   
dovecot-2.2.36.3/src/plugins/fts/fts-api.c
   
   

   
   
Please correct me if I have missed something.
   
   

   
   
Best regards
   
   
Gerald
   
  
  
   
  
  
   Seems to be correct, yes.
  
  
   ---
Aki Tuomi
   
 

Re: Hibernation and proxy

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 17:46 azurit--- via dovecot <
dovecot@dovecot.org> wrote:
   
   

   
   

   
   
Hi,
   
   

   
   
does hibernation works well with proxy? Are proxy connections
   
   
hibernated or not?
   
   

   
   
azur
   
  
  
   hibernation only happens on backends.
  
  
   ---
Aki Tuomi
   
 

Hibernation and proxy

[azurit--- via dovecot]

Hi,

does hibernation works well with proxy? Are proxy connections  
hibernated or not?


azur

CVE-2019-7524 backport patch for 2.2.33.2

[Gerald Galster via dovecot]

Hello Aki,

I'm currently stuck with 2.2.33.2 as 2.2.36 still duplicates mails after pop3 
deletion on a two node dsync cluster.

Therefore I've created a small patch and it seems only these two files are 
affected:

dovecot-2.2.36.3/src/lib-storage/index/index-pop3-uidl.c
dovecot-2.2.36.3/src/plugins/fts/fts-api.c

Please correct me if I have missed something.

Best regards
Gerald



dovecot-CVE-2019-7524-2.2.36-1-3.patch
Description: Binary data

Re: Mitigation / disable FTS and pop3-uidl plugin was Re: CVE-2019-7524: Buffer overflow when reading extension header from dovecot index files

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 16:44 Kevin A. McGrail via dovecot <
dovecot@dovecot.org> wrote:
   
   

   
   

   
   
On 3/28/2019 10:40 AM, Aki Tuomi wrote:
   
   
>
   
   

 check for fts in mail_plugins. pop3-uidl is used by pop3_migration


 plugin.

   
   
Sorry if I'm dense but can you be more specific?  Are you talking about
   
   
checking conf files or binary files? 
   
   

   
   
For example, does the existence of
   
   
/usr/local/lib/dovecot/lib20_fts_plugin.so imply an exploitable situation? 
   
   

   
   
Are their settings in a conf file that disable those plugins? 
   
   

   
   
Regards,
   
   

   
   
KAM
   
  
  
   
  
  
   Plugin needs to be explicitly loaded in configuration.
  
  
   ---
Aki Tuomi
   
 

Re: Mitigation / disable FTS and pop3-uidl plugin was Re: CVE-2019-7524: Buffer overflow when reading extension header from dovecot index files

[Kevin A. McGrail via dovecot]

On 3/28/2019 10:40 AM, Aki Tuomi wrote:
>
> check for fts in mail_plugins. pop3-uidl is used by pop3_migration
> plugin.

Sorry if I'm dense but can you be more specific?  Are you talking about
checking conf files or binary files? 

For example, does the existence of
/usr/local/lib/dovecot/lib20_fts_plugin.so imply an exploitable situation? 

Are their settings in a conf file that disable those plugins? 

Regards,

KAM

Re: Mitigation / disable FTS and pop3-uidl plugin was Re: CVE-2019-7524: Buffer overflow when reading extension header from dovecot index files

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 16:37 Kevin A. McGrail via dovecot <
dovecot@dovecot.org> wrote:
   
   

   
   

   
   
On 3/28/2019 7:42 AM, Aki Tuomi via dovecot wrote:
   
   

 olution:


 Operators should update to the latest Patch Release. The only workaround


 is to disable FTS and pop3-uidl plugin.

   
   
Hi Aki, thanks for the CVE.  For quick mitigation, can you confirm how
   
   
to disable these plugins and what they provide?  We'd like to assess if
   
   
we are using them while we rollout the fix.
   
   

   
   
Regards,
   
   

   
   
KAM
   
  
  
   
  
  
   check for fts in mail_plugins. pop3-uidl is used by pop3_migration plugin.
  
  
   ---
Aki Tuomi
   
 

Mitigation / disable FTS and pop3-uidl plugin was Re: CVE-2019-7524: Buffer overflow when reading extension header from dovecot index files

[Kevin A. McGrail via dovecot]

On 3/28/2019 7:42 AM, Aki Tuomi via dovecot wrote:
> olution:
> Operators should update to the latest Patch Release. The only workaround
> is to disable FTS and pop3-uidl plugin.

Hi Aki, thanks for the CVE.  For quick mitigation, can you confirm how
to disable these plugins and what they provide?  We'd like to assess if
we are using them while we rollout the fix.

Regards,

KAM

Re: configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Aki Tuomi via dovecot]

 
 
  
   
  
  
   
On 28 March 2019 16:08 Robert Kudyba via dovecot  wrote:
   
   

   
   

   
   

 dovecot-2.3.3-1.fc29.x86_64

   
   

   
   

 Mar 28 10:04:47 auth: Panic: file http-client-request.c: line 283 (http_client_request_unref): assertion failed: (req->refcount > 0)


 Mar 28 10:04:47 auth: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xe34fb) [0x7fe76e0834fb] -> /usr/lib64/dovecot/libdovecot.so.0(+0xe3597) [0x7fe76e083597] -> /usr/lib64/dovecot/libdovecot.so.0(+0x51207) [0x7fe76dff1207] -> /usr/lib64/dovecot/libdovecot.so.0(+0x4972b) [0x7fe76dfe972b] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_request_destroy+0x107) [0x7fe76e02cf87] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_deinit+0x4c) [0x7fe76e03b9ec] -> dovecot/auth(auth_policy_deinit+0x1e) [0x55facfdb350e] -> dovecot/auth(main+0x3e1) [0x55facfdae3c1] -> /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fe76dd93413] -> dovecot/auth(_start+0x2e) [0x55facfdae57e]


 Mar 28 10:04:47 auth: Fatal: master: service(auth): child 31162 killed with signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)


 Mar 28 10:04:48 master: Info: Dovecot v2.3.3 (dcead646b) starting up for imap, pop3

   
   

   
  
  
   Hi,
  
  
   
  
  
   this is a known issue as DOV-3019 and we are fixing this. It happens during auth process shutdown if there are pending requests.
  
  
   ---
Aki Tuomi
   
 

configuring Dovecot with wforced and auth_policy_server_url with https results in assertion failed

[Robert Kudyba via dovecot]

dovecot-2.3.3-1.fc29.x86_64

Mar 28 10:04:47 auth: Panic: file http-client-request.c: line 283 
(http_client_request_unref): assertion failed: (req->refcount > 0)
Mar 28 10:04:47 auth: Error: Raw backtrace: 
/usr/lib64/dovecot/libdovecot.so.0(+0xe34fb) [0x7fe76e0834fb] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0xe3597) [0x7fe76e083597] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0x51207) [0x7fe76dff1207] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0x4972b) [0x7fe76dfe972b] -> 
/usr/lib64/dovecot/libdovecot.so.0(http_client_request_destroy+0x107) 
[0x7fe76e02cf87] -> /usr/lib64/dovecot/libdovecot.so.0(http_client_deinit+0x4c) 
[0x7fe76e03b9ec] -> dovecot/auth(auth_policy_deinit+0x1e) [0x55facfdb350e] -> 
dovecot/auth(main+0x3e1) [0x55facfdae3c1] -> 
/lib64/libc.so.6(__libc_start_main+0xf3) [0x7fe76dd93413] -> 
dovecot/auth(_start+0x2e) [0x55facfdae57e]
Mar 28 10:04:47 auth: Fatal: master: service(auth): child 31162 killed with 
signal 6 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set 
/proc/sys/fs/suid_dumpable to 2)
Mar 28 10:04:48 master: Info: Dovecot v2.3.3 (dcead646b) starting up for imap, 
pop3

Re: v2.2.36.3 released

[Aki Tuomi via dovecot]

On 28.3.2019 13.41, Aki Tuomi via dovecot wrote:
> https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz
> https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz.sig
>
>     * CVE-2019-7524: Missing input buffer size validation leads into
>   arbitrary buffer overflow when reading fts or pop3 uidl header
>   from Dovecot index. Exploiting this requires direct write access to
>   the index files.
>
> ---
> Aki Tuomi
> Open-Xchange oy
>
Small mistake in the URLs, please use these.

https://dovecot.org/releases/2.2/dovecot-2.2.36.3.tar.gz
https://dovecot.org/releases/2.2/dovecot-2.2.36.3.tar.gz.sig

Aki





signature.asc
Description: OpenPGP digital signature

Re: [Dovecot-news] v2.2.36.3 released

[Aki Tuomi via Dovecot-news]

On 28.3.2019 13.41, Aki Tuomi via dovecot wrote:
> https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz
> https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz.sig
>
>     * CVE-2019-7524: Missing input buffer size validation leads into
>   arbitrary buffer overflow when reading fts or pop3 uidl header
>   from Dovecot index. Exploiting this requires direct write access to
>   the index files.
>
> ---
> Aki Tuomi
> Open-Xchange oy
>
Small mistake in the URLs, please use these.

https://dovecot.org/releases/2.2/dovecot-2.2.36.3.tar.gz
https://dovecot.org/releases/2.2/dovecot-2.2.36.3.tar.gz.sig

Aki





signature.asc
Description: OpenPGP digital signature
___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news

Re: dovecot mailing list stopped delivering mail

[Christian Anthon via dovecot]

Ahh,

I had made some changes to our own server around the time, but couldn't 
for the life of me understand why it would selectively refuse to talk to 
the dovecot mailing list server.


Thanks for clearing it up so promptly.

Cheers, Christian.

On 28/03/2019 13.21, Aki Tuomi wrote:

Fixed, we had smtp_security_level=verify, which I forgot to remove when
we removed the relay server we used. Now it should work.

Aki

On 28.3.2019 13.59, Christian Anthon via dovecot wrote:

Strangest thing. Since some time Marts 25. I'm no longer receiving
mails from the dovecot mailing list server. Mails from other sources
are behaving fine.

Instead I'm just seing

Mar 26 08:27:54 dna01 postfix/smtpd[107746]: Anonymous TLS connection
established from talvi.dovecot.org[94.237.25.159]: TLSv1.2 with cipher
ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Mar 26 08:27:54 dna01 postfix/smtpd[107746]: disconnect from
talvi.dovecot.org[94.237.25.159] ehlo=1 starttls=1 quit=1 commands=3

And then no more.

I tried turning up the postfix verbosity, but I'm none the wiser. I
realize that this is probably not a dovecot issue, but anybody with
any ideas?

Cheers, Christian (and please send an off-list reply also)

Verbose log:

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: connection established
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: master_notify: status 0
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: name_mask: resource
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: name_mask: software
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: connect from
talvi.dovecot.org[94.237.25.159]
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
talvi.dovecot.org: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
94.237.25.159: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
talvi.dovecot.org: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
94.237.25.159: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: smtp_stream_setup:
maxtime=300 enable_deadline=0
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostname:
smtpd_client_event_limit_exceptions: talvi.dovecot.org ~? 192.168.3.0/24
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostaddr:
smtpd_client_event_limit_exceptions: 94.237.25.159 ~? 192.168.3.0/24
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostname:
smtpd_client_event_limit_exceptions: talvi.dovecot.org ~? 127.0.0.0/8
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostaddr:
smtpd_client_event_limit_exceptions: 94.237.25.159 ~? 127.0.0.0/8
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
talvi.dovecot.org: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
94.237.25.159: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: send attr request = connect
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: send attr ident =
smtp:94.237.25.159
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
attribute: status
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: status
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 0
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
attribute: count
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: count
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 1
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
attribute: rate
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: rate
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 1
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
attribute: (list terminator)
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: (end)
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: report connect to all milters
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: "j"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
result "genome.ku.dk"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
"{daemon_name}"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
result "genome.ku.dk"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
"{daemon_addr}"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
result "192.168.3.112"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: "v"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
result "Postfix 3.3.3"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
non-protocol events for protocol version 6:
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
transport=inet endpoint=localhost:11332
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: trying... [127.0.0.1]
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: vstream_tweak_tcp:
TCP_MAXSEG 32741
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: fd=11: stream buffer size
old=0 new=65482
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
my_version=0x6
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:

Re: dovecot mailing list stopped delivering mail

[Aki Tuomi via dovecot]

Fixed, we had smtp_security_level=verify, which I forgot to remove when
we removed the relay server we used. Now it should work.

Aki

On 28.3.2019 13.59, Christian Anthon via dovecot wrote:
> Strangest thing. Since some time Marts 25. I'm no longer receiving
> mails from the dovecot mailing list server. Mails from other sources
> are behaving fine.
>
> Instead I'm just seing
>
> Mar 26 08:27:54 dna01 postfix/smtpd[107746]: Anonymous TLS connection
> established from talvi.dovecot.org[94.237.25.159]: TLSv1.2 with cipher
> ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
> Mar 26 08:27:54 dna01 postfix/smtpd[107746]: disconnect from
> talvi.dovecot.org[94.237.25.159] ehlo=1 starttls=1 quit=1 commands=3
>
> And then no more.
>
> I tried turning up the postfix verbosity, but I'm none the wiser. I
> realize that this is probably not a dovecot issue, but anybody with
> any ideas?
>
> Cheers, Christian (and please send an off-list reply also)
>
> Verbose log:
>
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: connection established
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: master_notify: status 0
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: name_mask: resource
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: name_mask: software
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: connect from
> talvi.dovecot.org[94.237.25.159]
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
> talvi.dovecot.org: no match
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
> 94.237.25.159: no match
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
> talvi.dovecot.org: no match
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
> 94.237.25.159: no match
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: smtp_stream_setup:
> maxtime=300 enable_deadline=0
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostname:
> smtpd_client_event_limit_exceptions: talvi.dovecot.org ~? 192.168.3.0/24
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostaddr:
> smtpd_client_event_limit_exceptions: 94.237.25.159 ~? 192.168.3.0/24
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostname:
> smtpd_client_event_limit_exceptions: talvi.dovecot.org ~? 127.0.0.0/8
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostaddr:
> smtpd_client_event_limit_exceptions: 94.237.25.159 ~? 127.0.0.0/8
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
> talvi.dovecot.org: no match
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match:
> 94.237.25.159: no match
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: send attr request = connect
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: send attr ident =
> smtp:94.237.25.159
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
> attribute: status
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: status
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 0
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
> attribute: count
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: count
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 1
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
> attribute: rate
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: rate
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 1
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted
> attribute: (list terminator)
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: (end)
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: report connect to all milters
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: "j"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
> result "genome.ku.dk"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
> "{daemon_name}"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
> result "genome.ku.dk"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
> "{daemon_addr}"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
> result "192.168.3.112"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: "v"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup:
> result "Postfix 3.3.3"
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
> non-protocol events for protocol version 6:
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
> transport=inet endpoint=localhost:11332
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: trying... [127.0.0.1]
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: vstream_tweak_tcp:
> TCP_MAXSEG 32741
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: fd=11: stream buffer size
> old=0 new=65482
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
> my_version=0x6
> Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect:
> my_actions=0x1ff SMFIF_ADDHDRS SMFIF_CHGBODY SMFIF_ADDRCPT
> SMFIF_DELRCPT SMFIF_CHGHDRS 

CVE-2019-7524: Buffer overflow when reading extension header from dovecot index files

[Aki Tuomi via dovecot]

Product: Dovecot
Vendor: OX Software GmbH
Internal reference: DOV-2964 (Bug ID)
Vulnerability type: CWE-120
Vulnerable version: 2.0.14 - 2.3.5
Vulnerable component: fts, pop3-uidl-plugin
Report confidence: Confirmed
Researcher credits: Found in internal testing
Solution status: Fixed by Vendor
Fixed version: 2.3.5.1, 2.2.36.3
Vendor notification: 2019-02-05
Solution date: 2019-03-21
Public disclosure: 2019-03-28
CVE reference: CVE-2019-7524
CVSS: 3.0/AV:L/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H/E:P/RL:O/RC:C (8.8)
 
Vulnerability Details:
When reading FTS or POP3-UIDL header from dovecot index, the input
buffer size is not bound, and data is copied to target structure causing
stack overflow.

Risk:
This can be used for local root privilege escalation or executing
arbitrary code in dovecot process context. This requires ability to
directly modify dovecot indexes.
Steps to reproduce:
Produce dovecot.index.log entry that creates an FTS header which has
more than 12 bytes of data.
Trigger dovecot indexer-worker or run doveadm index.
Dovecot will crash.

Mitigations:
Since 2.3.0 dovecot has been compiled with stack smash protection, ASLR,
read-only GOT tables and other techniques that make exploiting this bug
much harder.

Solution:
Operators should update to the latest Patch Release. The only workaround
is to disable FTS and pop3-uidl plugin.

--
Aki Tuomi
Open-Xchange Oy



signature.asc
Description: OpenPGP digital signature

v2.3.5.1 released

[Aki Tuomi via dovecot]

https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz.sig
Binary packages in https://repo.dovecot.org/

    * CVE-2019-7524: Missing input buffer size validation leads into
  arbitrary buffer overflow when reading fts or pop3 uidl header
  from Dovecot index. Exploiting this requires direct write access to
  the index files.

---
Aki Tuomi
Open-Xchange oy



signature.asc
Description: OpenPGP digital signature

doveadm backup doesn't transfer mail from root INBOX

[Francis via dovecot]

Hi,

I'm trying to migrate IMAP mails from dovecot 1.1.20apple0.5 (osx) to
dovecot 2.2.33.2 (ubuntu). I'm using "doveadm backup" to migrate my data.
It works fine for all subfolders, but the root INBOX stay empty on the new
server. I suspect a problem related with hierarchy separator ("." on
previous server, "/" on new) or with the namespace prefix (set to "INBOX/"
on new server). Any hint?

Thanks.


show there is no messages in INBOX
# doveadm mailbox status -u (myuser) all INBOX
INBOX messages=0 recent=0 uidnext=1 uidvalidity=1553630060 unseen=0
highestmodseq=1 vsize=0 guid=d54cd42849739b5cb579ed87e144
firstsaved=18446744073709551615

show there is messages in subdirectory
# doveadm mailbox status -u (myuser) all INBOX/Sent
INBOX/Sent messages=253 recent=0 uidnext=8721 uidvalidity=1263328366
unseen=0 highestmodseq=256 vsize=16335358
guid=bfb2e03fdce327671e82bf173b1ccb8b firstsaved=1553634631

The command I'm runnning on the new server to migrate data:
# doveadm -o mail_fsync=never backup -R -u (myuser) imapc:

# My config on the new server:
# doveadm -n
# 2.2.33.2 (d6601f4ec): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.21 (92477967)
# OS: Linux 4.15.0-46-generic x86_64 Ubuntu 18.04.2 LTS
auth_default_realm = example.com
doveadm_password =  # hidden, use -P to show it
doveadm_port = 53683
first_valid_uid = 200
imapc_features = rfc822.size fetch-headers
imapc_host = oldmail.example.com
imapc_master_user = (removed)
imapc_password =  # hidden, use -P to show it
imapc_user = %n
last_valid_uid = 200
mail_location = mbox:~/mail:INBOX=/var/mail/%u
mail_plugins = " notify replication zlib quota"
mail_prefetch_count = 20
mail_privileged_group = mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date index ihave
duplicate mime foreverypart extracttext
mdbox_rotate_size = 10 M
namespace inbox {
  inbox = yes
  location =
  mailbox "Éléments envoyés" {
special_use = \Sent
  }
  mailbox "Éléments supprimés" {
special_use = \Trash
  }
  mailbox Brouillon {
special_use = \Drafts
  }
  mailbox "Courrier indésirable" {
special_use = \Junk
  }
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix = INBOX/
  separator = /
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  mail_replica = tcps:mail2.example.com:53683
  quota = count:User quota
  quota_grace = 10%%
  quota_rule = *:storage=10G
  quota_rule2 = INBOX/Trash:storage=+100M
  quota_vsizes = yes
  quota_warning = storage=100%% quota-warning 100 %u
  quota_warning2 = storage=95%% quota-warning 95 %u
  quota_warning3 = storage=90%% quota-warning 90 %u
  quota_warning4 = storage=85%% quota-warning 85 %u
  quota_warning5 = storage=75%% quota-warning 75 %u
  quota_warning6 = -storage=80%% quota-warning '-80' %u
  quota_warning7 = -storage=100%% quota-warning '-100' %u
  sieve = /var/vmail/domains/%Ld/%Ln/.dovecot.sieve
  sieve_before = /var/vmail/sieve/before.sieve
  sieve_default = /var/vmail/sieve/default.sieve
  sieve_global = /var/vmail/sieve/global
  zlib_save = lz4
}
protocols = " imap lmtp sieve"
service aggregator {
  fifo_listener replication-notify-fifo {
user = vmail
  }
  unix_listener replication-notify {
user = vmail
  }
}
service auth {
  inet_listener {
address = (removed)
port = 55123
  }
  unix_listener /var/spool/postfix/private/auth {
mode = 0666
  }
}
service doveadm {
  inet_listener {
port = 53683
ssl = yes
  }
}
service imap-login {
  inet_listener imap {
port = 0
  }
  process_min_avail = 10
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
user = vmail
  }
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
mode = 0600
user = vmail
  }
}
ssl = required
ssl_cert = 

Maildir permissions issue with Postfix

[Randall R. Sargent via dovecot]

Hi all,

We have a Postfix server that serves as an alias MTA to route to other mail 
systems. I've recently installed Dovecot on it because we have three service 
accounts that need simple POP3 mailboxes. I have the accounts set up on the 
system and mail does get delivered to their ~/Maildir/ locations by Postfix, 
however, every time a message is dropped in the Maildir for the user, the 
permissions for the message are set to the user's uid:gid and Dovecot can't 
access the mail message. Doing a chmod fixes it temporarily until the next 
message drops.  The vmail group has access to the user's home folder structure.

Any help is appreciated!!

Dovecot: 2.2.36

/etc/dovecot/dovecot.conf:

auth_debug = yes
disable_plaintext_auth = no
first_valid_uid = 1002
log_path = /var/log/dovecot.log
mail_debug = yes
mail_home = /srv/mail/%Lu
mail_location = maildir:~/Maildir
mbox_very_dirty_syncs = yes

namespace {
  inbox = yes
  location =
  prefix =
  separator = /
}
passdb {
  driver = pam
}
service auth-worker {
  user = root
}
userdb {
  args = blocking=no
  driver = passwd
  override_fields = uid=vmail gid=vmail
}

Thanks,
Randy

Re: Panic: file mail-transaction-log-file.c: line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

[Timo Sirainen via dovecot]

On 28 Mar 2019, at 10.15, Arkadiusz Miśkiewicz  wrote:
> 
>  error = 0x55e3e2b40ac0 "Fixed index file
> /var/mail/piast_efaktury/dovecot.index: log_file_seq 13 -> 15",
>  nodiskspace = true,

This was one of the things I was first wondering, but I'm not sure why it's not 
logging an error. Anyway, you're using filesystem quota? And this index is 
large enough that trying to rewrite it brings the user over quota?

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Sami Ketola via dovecot]

> On 28 Mar 2019, at 1.08, Jason Lewis via dovecot  wrote:
> 
> Thanks Timo.
> 
> Given the age of these dovecot packages, and this being on debian
> oldstable, what should we do next? I'm inclined to just delete the email
> in question and move on.
> 

https://repo.dovecot.org/ 

Sami

Re: Maildir permissions issue with Postfix

[Sami Ketola via dovecot]

> On 27 Mar 2019, at 17.03, Randall R. Sargent via dovecot 
>  wrote:
> 
> Hi all,
>  
> We have a Postfix server that serves as an alias MTA to route to other mail 
> systems. I’ve recently installed Dovecot on it because we have three service 
> accounts that need simple POP3 mailboxes. I have the accounts set up on the 
> system and mail does get delivered to their ~/Maildir/ locations by Postfix, 
> however, every time a message is dropped in the Maildir for the user, the 
> permissions for the message are set to the user’s uid:gid and Dovecot can’t 
> access the mail message. Doing a chmod fixes it temporarily until the next 
> message drops.  The vmail group has access to the user’s home folder 
> structure.
>  
> Any help is appreciated!!

use dovecot-lda to deliver the emails. https://wiki.dovecot.org/LDA/Postfix

Sami

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Jason Lewis via dovecot]

After some investigation, it turns out it is non trivial to install
dovecot-dbg on debian jessie.

Sorry I can't investigate further.

Jason

Aki Tuomi wrote on 25/3/19 6:12 pm:
> Can you install dovecot-dbg and try gdb again?
> 
> Aki
> 
> On 25.3.2019 3.20, Jason Lewis via dovecot wrote:
>> Hi,
>>
>> I've been having an issue with the indexer giving me errors on mailbox
>> in dovecot.
>>
>> I managed to narrow it down to a specific email in that mailbox.
>>
>> Various dovecot functions have issues with this email.
>>
>> The email itself is just spam. I can email it to you if you want to
>> analyse it. I did run it through mbox-anonymize but its not clear to me
>> that that would be of any use. Happy to email the suspect email
>> privately to anyone who wants it.
>>
>> /home is mounted nfs4 and is zfs on the nfs server.
>>
>>
>> Dovecot is installed from Debian Jessie.
>> $ /usr/sbin/dovecot --version
>> 2.2.27 (c0f36b0)
>>
>> dovecot-core:
>>   Installed: 1:2.2.27-3+deb9u2~bpo8+1
>>   Candidate: 1:2.2.27-3+deb9u2~bpo8+1
>>   Version table:
>>  *** 1:2.2.27-3+deb9u2~bpo8+1 0
>> 100 /var/lib/dpkg/status
>>  1:2.2.13-12~deb8u5 0
>> 400 http://security.debian.org/ jessie/updates/main amd64 Packages
>>  1:2.2.13-12~deb8u4 0
>> 400 http://deb.debian.org/debian/ jessie/main amd64 Packages
>>
>>
>> ~# dovecot -n
>> # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
>> # Pigeonhole version 0.4.16 (fed8554)
>> # OS: Linux 4.9.0-0.bpo.6-amd64 x86_64 Debian 8.10
>> imap_hibernate_timeout = 5 secs
>> mail_location = maildir:~/Maildir
>> mail_plugins = fts fts_solr
>> mailbox_list_index = yes
>> namespace inbox {
>>   inbox = yes
>>   location =
>>   mailbox Drafts {
>> special_use = \Drafts
>>   }
>>   mailbox Junk {
>> special_use = \Junk
>>   }
>>   mailbox Sent {
>> special_use = \Sent
>>   }
>>   mailbox "Sent Messages" {
>> special_use = \Sent
>>   }
>>   mailbox Trash {
>> special_use = \Trash
>>   }
>>   prefix =
>> }
>> passdb {
>>   driver = pam
>> }
>> plugin {
>>   fts = solr
>>   fts_autoindex = yes
>>   fts_enforced = yes
>>   fts_solr = url=http://10.0.2.19:8080/solr/
>>   imapsieve_mailbox1_before = file:/etc/dovecot/train-as-spam.sieve
>>   imapsieve_mailbox1_causes = COPY
>>   imapsieve_mailbox1_name = Junk
>>   imapsieve_mailbox2_before = file:/etc/dovecot/train-as-ham.sieve
>>   imapsieve_mailbox2_causes = COPY
>>   imapsieve_mailbox2_from = Junk
>>   imapsieve_mailbox2_name = *
>>   sieve = file:~/sieve;active=~/.dovecot.sieve
>>   sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment
>>   sieve_pipe_bin_dir = /usr/bin
>>   sieve_plugins = sieve_imapsieve sieve_extprograms
>> }
>> protocols = " imap"
>> service anvil {
>>   client_limit = 1127
>> }
>> service auth {
>>   client_limit = 2200
>>   unix_listener /var/spool/postfix/private/auth {
>> mode = 0666
>>   }
>> }
>> service imap-hibernate {
>>   unix_listener imap-hibernate {
>> group = dovecot
>> mode = 0660
>>   }
>> }
>> service imap-login {
>>   process_limit = 1024
>>   process_min_avail = 12
>>   service_count = 0
>>   vsz_limit = 1 G
>> }
>> service imap {
>>   extra_groups = dovecot
>>   unix_listener imap-master {
>> user = dovecot
>>   }
>> }
>> ssl_cert = > ssl_key =  # hidden, use -P to show it
>> userdb {
>>   driver = passwd
>> }
>> protocol imap {
>>   mail_max_userip_connections = 20
>>   mail_plugins = fts fts_solr imap_sieve
>> }
>>
>>
>>
>>
>>
>> jason@debian:~$ doveadm -D -f flow fetch imap.envelope mailbox
>> crm-spam.2008.g
>> Debug: Loading modules from directory: /usr/lib/dovecot/modules
>> Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so
>> Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so
>> Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
>> Debug: Skipping module doveadm_acl_plugin, because dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined
>> symbol: acl_user_module (this is usually intentional, so just ignore
>> this message)
>> Debug: Skipping module doveadm_expire_plugin, because dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so:
>> undefined symbol: expire_set_deinit (this is usually intentional, so
>> just ignore this message)
>> Debug: Skipping module doveadm_quota_plugin, because dlopen() failed:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so:
>> undefined symbol: quota_user_module (this is usually intentional, so
>> just ignore this message)
>> Debug: Module loaded:
>> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so
>> Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen()
>> failed:
>> /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so:
>> undefined symbol: lucene_index_iter_deinit (this is usually intentional,
>> so just ignore this message)
>> Debug: Module loaded:
>> /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Aki Tuomi via dovecot]

What's non-trivial about apt-get install dovecot-dbg?

Aki

On 26.3.2019 5.38, Jason Lewis via dovecot wrote:
> After some investigation, it turns out it is non trivial to install
> dovecot-dbg on debian jessie.
>
> Sorry I can't investigate further.
>
> Jason
>
> Aki Tuomi wrote on 25/3/19 6:12 pm:
>> Can you install dovecot-dbg and try gdb again?
>>
>> Aki
>>
>> On 25.3.2019 3.20, Jason Lewis via dovecot wrote:
>>

Re: dovecot 2.3.5 - tests fail: http payload echo (ssl)

[Helmut K. C. Tessarek via dovecot]

On 2019-03-18 19:04, Marcus Rueckert via dovecot wrote:
> Just a guess ... maybe it is lack of entropy on the build machine?
> Try running haveged.

Nope, the entropy is just fine. But thanks for the idea.

Cheers,
  K. C.


-- 
regards Helmut K. C. Tessarek  KeyID 0x172380A011EF4944
Key fingerprint = 8A55 70C1 BD85 D34E ADBC 386C 1723 80A0 11EF 4944

/*
   Thou shalt not follow the NULL pointer for chaos and madness
   await thee at its end.
*/



signature.asc
Description: OpenPGP digital signature

Re: IMAP coredumps for one user

[Aki Tuomi via dovecot]

This seems to be a problem with xapian fts plugin, perhaps you should
open issue there?

Aki

On 26.3.2019 10.15, Odhiambo Washington via dovecot wrote:
> FreeBSD-12
> Dovecot-2.3.5
>
> I am having problems with one use
>
> Mar 25 21:30:12 imap(gau@crownkenya.com
> )<91364>: Fatal:
> master: service(imap): child 91364 killed with signal 6 (core dumped)
> Mar 25 21:30:14 imap(gau@crownkenya.com
> )<2381>: Fatal:
> master: service(imap): child 2381 killed with signal 6 (core dumped)
> Mar 26 06:29:26 imap(gau@crownkenya.com
> )<7644><0aVP7faEI/GaTGQE>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:26 imap(gau@crownkenya.com
> )<8806>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:26 imap(gau@crownkenya.com
> )<8493>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:26 imap(gau@crownkenya.com
> )<9412><2qtP7faEIfGaTGQE>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:28 imap(gau@crownkenya.com
> )<7644><0aVP7faEI/GaTGQE>: Fatal:
> master: service(imap): child 7644 killed with signal 6 (core dumped)
> Mar 26 06:29:29 imap(gau@crownkenya.com
> )<10764><7sW+7faEJvGaTGQE>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:31 imap(gau@crownkenya.com
> )<8806>: Fatal:
> master: service(imap): child 8806 killed with signal 6 (core dumped)
> Mar 26 06:29:33 imap(gau@crownkenya.com
> )<8493>: Fatal:
> master: service(imap): child 8493 killed with signal 6 (core dumped)
> Mar 26 06:29:36 imap(gau@crownkenya.com
> )<9412><2qtP7faEIfGaTGQE>: Fatal:
> master: service(imap): child 9412 killed with signal 6 (core dumped)
> Mar 26 06:29:38 imap(gau@crownkenya.com
> )<10764><7sW+7faEJvGaTGQE>: Fatal:
> master: service(imap): child 10764 killed with signal 6 (core dumped)
> Mar 26 06:29:57 imap(gau@crownkenya.com
> )<13285>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:57 imap(gau@crownkenya.com
> )<19736>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:57 imap(gau@crownkenya.com
> )<22060>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:29:57 imap(gau@crownkenya.com
> )<21367>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:30:00 imap(gau@crownkenya.com
> )<13285>: Fatal:
> master: service(imap): child 13285 killed with signal 6 (core dumped)
> Mar 26 06:30:00 imap(gau@crownkenya.com
> )<49026>: Panic: file
> mempool-system.c: line 137 (pool_system_realloc): assertion failed:
> (old_size == (size_t)-1 || mem == NULL || old_size <=
> malloc_usable_size(mem))
> Mar 26 06:30:02 imap(gau@crownkenya.com
> )<19736>: Fatal:
> master: service(imap): child 19736 killed with signal 6 (core dumped)
> Mar 26 06:30:05 imap(gau@crownkenya.com
> )<22060>: Fatal:
> master: service(imap): child 22060 killed with signal 6 (core dumped)
> Mar 26 06:30:07 imap(gau@crownkenya.com
> )<21367>: Fatal:
> master: service(imap): child 21367 killed with signal 6 (core dumped)
> Mar 26 06:30:10 imap(gau@crownkenya.com
> )<49026>: Fatal:
> master: service(imap): child 49026 killed with signal 6 (core dumped)
> Mar 26 07:59:40 imap(gau@crownkenya.com
> )<27221><06UEMPiESPKaRh6E>: Panic: file
> 

dovecot mailing list stopped delivering mail

[Christian Anthon via dovecot]

Strangest thing. Since some time Marts 25. I'm no longer receiving mails 
from the dovecot mailing list server. Mails from other sources are 
behaving fine.


Instead I'm just seing

Mar 26 08:27:54 dna01 postfix/smtpd[107746]: Anonymous TLS connection 
established from talvi.dovecot.org[94.237.25.159]: TLSv1.2 with cipher 
ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Mar 26 08:27:54 dna01 postfix/smtpd[107746]: disconnect from 
talvi.dovecot.org[94.237.25.159] ehlo=1 starttls=1 quit=1 commands=3


And then no more.

I tried turning up the postfix verbosity, but I'm none the wiser. I 
realize that this is probably not a dovecot issue, but anybody with any 
ideas?


Cheers, Christian (and please send an off-list reply also)

Verbose log:

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: connection established
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: master_notify: status 0
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: name_mask: resource
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: name_mask: software
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: connect from 
talvi.dovecot.org[94.237.25.159]
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match: 
talvi.dovecot.org: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match: 
94.237.25.159: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match: 
talvi.dovecot.org: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match: 
94.237.25.159: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: smtp_stream_setup: 
maxtime=300 enable_deadline=0
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostname: 
smtpd_client_event_limit_exceptions: talvi.dovecot.org ~? 192.168.3.0/24
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostaddr: 
smtpd_client_event_limit_exceptions: 94.237.25.159 ~? 192.168.3.0/24
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostname: 
smtpd_client_event_limit_exceptions: talvi.dovecot.org ~? 127.0.0.0/8
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_hostaddr: 
smtpd_client_event_limit_exceptions: 94.237.25.159 ~? 127.0.0.0/8
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match: 
talvi.dovecot.org: no match
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: match_list_match: 
94.237.25.159: no match

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: send attr request = connect
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: send attr ident = 
smtp:94.237.25.159
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted 
attribute: status

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: status
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 0
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted 
attribute: count

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: count
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 1
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted 
attribute: rate

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: rate
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute value: 1
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: private/anvil: wanted 
attribute: (list terminator)

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: input attribute name: (end)
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: report connect to all milters
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: "j"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: result 
"genome.ku.dk"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: 
"{daemon_name}"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: result 
"genome.ku.dk"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: 
"{daemon_addr}"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: result 
"192.168.3.112"

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: "v"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter_macro_lookup: result 
"Postfix 3.3.3"
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect: 
non-protocol events for protocol version 6:
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect: 
transport=inet endpoint=localhost:11332

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: trying... [127.0.0.1]
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: vstream_tweak_tcp: 
TCP_MAXSEG 32741
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: fd=11: stream buffer size 
old=0 new=65482

Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect: my_version=0x6
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect: 
my_actions=0x1ff SMFIF_ADDHDRS SMFIF_CHGBODY SMFIF_ADDRCPT SMFIF_DELRCPT 
SMFIF_CHGHDRS SMFIF_QUARANTINE SMFIF_CHGFROM SMFIF_ADDRCPT_PAR 
SMFIF_SETSYMLIST
Mar 28 10:57:42 dna01 postfix/smtpd[66648]: milter8_connect: 
my_events=0x1f SMFIP_NOCONNECT SMFIP_NOHELO SMFIP_NOMAIL 
SMFIP_NORCPT SMFIP_NOBODY SMFIP_NOHDRS SMFIP_NOEOH SMFIP_NR_HDR 
SMFIP_NOUNKNOWN SMFIP_NODATA SMFIP_SKIP SMFIP_RCPT_REJ SMFIP_NR_CONN 
SMFIP_NR_HELO 

Re: index problems after update

[Hajo Locke via dovecot]

Hello,


Am 21.02.2019 um 23:06 schrieb Adi Pircalabu via dovecot:

On 2019-02-21 22:18, Sami Ketola via dovecot wrote:

On 21 Feb 2019, at 12.23, Hajo Locke via dovecot
 wrote:
I think mbox+procmail is a classic setup and wide used and good
solution for many usecases. Same setup we use many years.
We run ~2 mio mailboxes. our automated systems depends on this
setup. creating mailboxes, managing mailboxes, creating automated
filterrules, backupsystem to tell something of them. we can not
switch our whole mailsetup to work around this bug.
How to get a dump if dovecot not crashing but has wrong behaviour? I
would like to help and provide useful info, but it depends on kind
of problem.
I think if a classic setup is not working in dovecot any more, this
is a serious problem.


In you first email to this thread it says:


Feb  8 08:45:37 hostname dovecot[14882]: imap(myuser): Fatal:
master: service(imap): child 14135 killed with signal 6 (core dumped)


So imap is crashing and even dumping a core.

Also I must disagree with your mbox+procmail statement. mbox has
always been very unoptimised mailbox format and everyone should be
emphasised not to use it.
Also that combination has always had problems with indexing and file
locking. I would not use it on high volume mailservers. Or even medium
volume mailservers.


Not directly affected by this issue since I'm not using mbox for any
production system nor have I for many years. And it'd take a lot of
effort to convince me to use mbox for anything someone would even dare
to classify, even remotely, as "production". But if I understand OP's
point of view correctly, he's not arguing necessarily for or against a
specific mailbox format. Instead, he's flagging a regression and
people will be very reluctant to upgrade or even adopt a certain
feature in a new release of a product if regressions are seen as
acceptable. Something that previously worked in an otherwise unchanged
environment stopped working after an upgrade and this is a regression.
Trying to convince people to move away from mbox is a very sensible
approach, I'm all for it, but in cases like this not practical.


yes, thanks for your support. I think this is the case here.

As conclusion:
We believe mdbox is better storage format, but we cant switch 2mio
mboxes + peripherie adhoc.
We did a downgrade to 2.2.26.0 a few weeks ago. 2.2.26.0 is first
version usable with openssl 1.1.0 and solves this issue. so this dovecot
problem must introduced after 2.2.26.0 and leads to a problem where
dovecot is not able to fulfill its basic purpose.
until this downgrade we had a lot of problems and complaints and felt
forsaken.
Too bad that there is no really interest to fix mbox related problems.

Thanks,
Hajo

Re: Panic: file mail-transaction-log-file.c: line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

[Arkadiusz Miśkiewicz via dovecot]

On 28/03/2019 09:36, Timo Sirainen wrote:
> On 28 Mar 2019, at 10.15, Arkadiusz Miśkiewicz  > wrote:
>>
>>  error = 0x55e3e2b40ac0 "Fixed index file
>> /var/mail/piast_efaktury/dovecot.index: log_file_seq 13 -> 15",
>>  nodiskspace = true,
> 
> This was one of the things I was first wondering, but I'm not sure why
> it's not logging an error. Anyway, you're using filesystem quota? And
> this index is large enough that trying to rewrite it brings the user
> over quota?
> 

That's possible, fs quota and user is close to being full:

used: 19061.63 MiB (19987.57 MB)
hard: 19073.49 MiB (2.00 MB)

So some initial write/rewrite of indexes corrupts them and then fixing
doesn't do anything because it can't do anything when it's over quota
(but why it doesn't report over quota errors in logs?)

That's from current try, lmtp trying to write, index already corrupted
earlier:

10068 openat(AT_FDCWD, "/var/mail/piast_efaktury/dovecot.index.log",
O_RDWR|O_APPEND) = 15
10068 fstat(15, {st_mode=S_IFREG|0600, st_size=40, ...}) = 0
10068 pread64(15,
"\1\3(\0\330\2\232\\\17\0\0\0\16\0\0\0(\200\0\0o&\232\\_\10\0\0\0\0\0\0\1\0\0\0\0\0\0\0",
40, 0) = 40
10068 openat(AT_FDCWD, "/var/mail/piast_efaktury/dovecot.index", O_RDWR)
= 16
10068 fstat(16, {st_mode=S_IFREG|0600, st_size=13555228, ...}) = 0
10068 mmap(NULL, 13555228, PROT_READ|PROT_WRITE, MAP_PRIVATE, 16, 0) =
0x7f1abc8e3000
10068 openat(AT_FDCWD, "/var/mail/piast_efaktury/dovecot.index.log.2",
O_RDWR|O_APPEND) = 17
10068 fstat(17, {st_mode=S_IFREG|0600, st_size=32808, ...}) = 0
10068 pread64(17, "\1\3(\0\330\2\232\\\16\0\0\0\r\0\0\0x\200\0\0\277
\232\\|\7\0\0\0\0\0\0\1\0\0\0\0\0\0\0", 40, 0) = 40
10068 close(17) = 0
10068 write(2, "\1\01010068 prefix=lmtp(piast_efaktury): pid=<10068>
session=, \n", 84) = 84
10068 write(2, "\1\00410068 Index
/var/mail/piast_efaktury/dovecot.index: Lost log for seq=13
offset=25648: Missing middle file seq=13 (between 13..4294967295, we
have seqs 14,15): .log.2 contains file_seq=14 (initial_ma"..., 229) = 229
10068 write(2, "\1\00310068 fscking index file
/var/mail/piast_efaktury/dovecot.index\n", 66) = 66
10068 alarm(180)= 0
10068 fcntl(15, F_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0,
l_len=0}) = 0
10068 alarm(0)  = 180
10068 stat("/var/mail/piast_efaktury/dovecot.index.log",
{st_mode=S_IFREG|0600, st_size=40, ...}) = 0
10068 fstat(15, {st_mode=S_IFREG|0600, st_size=40, ...}) = 0
10068 write(2, "\1\00410068 Fixed index file
/var/mail/piast_efaktury/dovecot.index: log_file_seq 13 -> 15\n", 87) = 87
10068 umask(000)= 077
10068 openat(AT_FDCWD, "/var/mail/piast_efaktury/dovecot.index.tmp",
O_RDWR|O_CREAT|O_EXCL, 0600) = 17
10068 umask(077)= 000
10068 fstat(17, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
10068 write(17,
"\7\3x\0\320\0\0\0\f\0\0\0\1\0\0\0\330\2\232\\\4\0\0\0\307A\260R\316\20D\0q<\21\0\0\0\0\0f\300\17\0\0\0\0\0\304\20D\0\303\224B\0\316\20D\0\17\0\0\0(\0\0\0(\0\0\0\0\0\0\0\177\34\232\\\0\0\0\0p]\231\\\240\363C\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\330\2\232\\\10\0\4\0\4\0\5\0cache\0\0\0$\0\0\0\0\0\0\0\0\0\0\0\0\0\7\0maildir\0\214\37\232\\\214\37\232\\\333!\0233\330\37\232\\\214\37\232\\\333!\0233\330\37\232\\T\250\31."...,
208) = 208
10068 write(17,
"]\3242\0\10\0\0\0\0\0\0\0^\3242\0\10\0\0\0\0\0\0\0_\3242\0\10\0\0\0\0\0\0\0`\3242\0\10\0\0\0\0\0\0\0a\3242\0\10\0\0\0\0\0\0\0b\3242\0\10\0\0\0\0\0\0\0c\3242\0\10\0\0\0\0\0\0\0d\3242\0\10\0\0\0\0\0\0\0e\3242\0\10\0\0\0\0\0\0\0f\3242\0\10\0\0\0\0\0\0\0g\3242\0\10\0\0\0\0\0\0\0h\3242\0\10\0\0\0\0\0\0\0i\3242\0\10\0\0\0\0\0\0\0j\3242\0\10\0\0\0\0\0\0\0k\3242\0\10\0\0\0\0\0\0\0l\3242\0\10\0\0\0\0\0\0\0m\3242\0\10\0\0\0"...,
13555020) = 8912688
10068 write(17,
"\241)>\0\10\0\0\0\0\0\0\0\242)>\0\10\0\0\0\0\0\0\0\243)>\0\10\0\0\0\0\0\0\0\244)>\0\10\0\0\0\0\0\0\0\245)>\0\10\0\0\0\0\0\0\0\246)>\0\10\0\0\0\0\0\0\0\247)>\0\10\0\0\0\0\0\0\0\250)>\0\10\0\0\0\0\0\0\0\251)>\0\10\0\0\0\0\0\0\0\252)>\0\10\0\0\0\0\0\0\0\253)>\0\10\0\0\0\0\0\0\0\254)>\0\10\0\0\0\0\0\0\0\255)>\0\10\0\0\0\0\0\0\0\256)>\0\10\0\0\0\0\0\0\0\257)>\0\10\0\0\0\0\0\0\0\260)>\0\10\0\0\0\0\0\0\0\261)>\0\10\0\0\0"...,
4642332) = -1 EDQUOT (Disk quota exceeded)
10068 close(17) = 0
10068 unlink("/var/mail/piast_efaktury/dovecot.index.tmp") = 0
10068 write(2, "\1\00610068 file mail-transaction-log-file.c: line 105
(mail_transaction_log_file_free): assertion failed: (!file->locked)\n",
119) = 119


(I guess I have to finally separate indexes out to separate partition
(outside of quota) because more of these "not dealing right with fs
quota" problems are there, previous one:
https://dovecot.org/pipermail/dovecot/2018-November/113460.html .
Unrelated to current issue)

Other solution could be to make dovecot treat soft quota as hard quota
and just give a bit more hard quota for these index updates. Not sure if
that's doable 

Re: Panic: file mail-transaction-log-file.c: line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

[Arkadiusz Miśkiewicz via dovecot]

On 27/03/2019 16:12, Timo Sirainen wrote:
> On 27 Mar 2019, at 14.58, Timo Sirainen via dovecot  > wrote:
>>
>>> dovecot isn't able to auto fix the indexes and manual deletion is
>>> required in all such cases
>>
>> So if it keeps repeating, it's very strange. Could you send me such
>> broken dovecot.index and dovecot.index.log files (without
>> dovecot.index.cache)? They shouldn't contain anything sensitive (only
>> message flags).
> 
> Tested with the index files you sent. It gets fixed automatically in my
> tests.
> 
> The backtrace shows that after fsck it fails to write the fixed index to
> the disk, because mail_index_write() fails for some reason. Except
> there's no error logged about it, which is rather weird. Do you still
> have the lmtp core? Could you do:
> 
> fr 9
> p *log.index

(gdb) f 9
#9  0x7f25fc9e709d in mail_transaction_log_move_to_memory
(log=0x55e3e2b81eb0) at mail-transaction-log.c:171
171 mail_transaction_log_close(log);
(gdb) *log.index
Undefined command: "".  Try "help".
(gdb) print *log.index
There is no member named index.
(gdb) print *log
$8 = {
  0x55e3e2b81c20,
  views = 0x0,
  filepath = 0x55e3e2c2adc0 "/var/mail/piast_efaktury/dovecot.index.log",
  filepath2 = 0x55e3e2b6f380 "/var/mail/piast_efaktury/dovecot.index.log.2",
  files = 0x55e3e2b7ab20,
  head = 0x55e3e2b7ab20,
  open_file = 0x0,
  dotlock_count = 0,
  dotlock = 0x0,
  nfs_flush = false,
  log_2_unlink_checked = true
}
(gdb) print *(struct mail_index *)0x55e3e2b81c20
$15 = {
  dir = 0x55e3e2ba8970 "/var/mail/piast_efaktury",
  prefix = 0x55e3e2baa1a0 "dovecot.index",
  cache_dir = 0x55e3e2b888c0 "/var/mail/piast_efaktury",
  event = 0x55e3e2b7a930,
  cache = 0x0,
  log = 0x55e3e2b81eb0,
  open_count = 0,
  flags = (MAIL_INDEX_OPEN_FLAG_CREATE |
MAIL_INDEX_OPEN_FLAG_DOTLOCK_USE_EXCL | MAIL_INDEX_OPEN_FLAG_SAVEONLY),
  fsync_mode = FSYNC_MODE_OPTIMIZED,
  fsync_mask = (unknown: 0),
  mode = 384,
  gid = 4294967295,
  gid_origin = 0x55e3e2c2ad60 "/var/mail/piast_efaktury",
  optimization_set = {
{
  rewrite_min_log_bytes = 8192,
  rewrite_max_log_bytes = 131072
},
log = {
  min_size = 32768,
  max_size = 1048576,
  min_age_secs = 300,
  log2_max_age_secs = 172800
},
cache = {
  unaccessed_field_drop_secs = 2592000,
  record_max_size = 65536,
  compress_min_size = 32768,
  compress_delete_percentage = 20,
  compress_continued_percentage = 200,
  compress_header_continue_count = 4
}
  },
  pending_log2_rotate_time = 0,
  extension_pool = 0x55e3e2c2bf90,
  extensions = {
arr = {
  buffer = 0x55e3e2c2bfb8,
  element_size = 48
},
v = 0x55e3e2c2bfb8,
v_modifiable = 0x55e3e2c2bfb8
  },
  ext_hdr_init_id = 0,
  ext_hdr_init_data = 0x0,
  sync_lost_handlers = {
arr = {
  buffer = 0x55e3e2b41900,
  element_size = 8
},
v = 0x55e3e2b41900,
v_modifiable = 0x55e3e2b41900
--Type  for more, q to quit, c to continue without paging--c
  },
  filepath = 0x55e3e2c2ad90 "/var/mail/piast_efaktury/dovecot.index",
  fd = 16,
  map = 0x55e3e2c2c380,
  last_mmap_error_time = 0,
  indexid = 1553597144,
  inconsistency_id = 0,
  last_read_log_file_seq = 13,
  last_read_log_file_tail_offset = 25648,
  fsck_log_head_file_seq = 15,
  fsck_log_head_file_offset = 40,
  sync_commit_result = 0x0,
  lock_method = FILE_LOCK_METHOD_FCNTL,
  max_lock_timeout_secs = 4294967295,
  keywords_pool = 0x55e3e2c2a400,
  keywords = {
arr = {
  buffer = 0x55e3e2c2a5f0,
  element_size = 8
},
v = 0x55e3e2c2a5f0,
v_modifiable = 0x55e3e2c2a5f0
  },
  keywords_hash = {
_table = 0x55e3e2c2a6c0,
_key = 0x55e3e2c2a6c0 "",
_keyp = 0x55e3e2c2a6c0,
_const_key = 0x55e3e2c2a6c0 "",
_value = 0x55e3e2c2a6c0,
_valuep = 0x55e3e2c2a6c0
  },
  keywords_ext_id = 0,
  modseq_ext_id = 1,
  views = 0x0,
  module_contexts = {
arr = {
  buffer = 0x55e3e2b41940,
  element_size = 8
},
v = 0x55e3e2b41940,
v_modifiable = 0x55e3e2b41940
  },
  error = 0x55e3e2b40ac0 "Fixed index file
/var/mail/piast_efaktury/dovecot.index: log_file_seq 13 -> 15",
  nodiskspace = true,
  index_lock_timeout = false,
  index_delete_requested = false,
  index_deleted = false,
  log_sync_locked = true,
  readonly = false,
  mapping = true,
  syncing = false,
  need_recreate = false,
  index_min_write = false,
  modseqs_enabled = false,
  initial_create = false,
  initial_mapped = false,
  fscked = true
}
(gdb)

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

Panic: file mail-transaction-log-file.c: line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

[Arkadiusz Miśkiewicz via dovecot]

Hello.

I have one account with heavy traffic (big mails) and quite often
indexes get corrupted.

This is dovecot 2.3.5 on local fs (xfs), Linux kernel 4.19.20, glibc 2.28.

When corruption happens lmtp and pop3 segfault on accessing it like:

> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(24428): Connect from local  
>   
>
> [0/0]
> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
> session=, Error: Index 
> /var/mail/piast_efaktury/dovecot.index: Lost log for seq=13 offset=25648: 
> Missing middle file seq=13 (between 13..4294967295, we have seqs 14,15): 
> .log.2 contains file_seq=14 (initial_mapped=0, reason=Index mapped)
> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
> session=, Warning: fscking index file 
> /var/mail/piast_efaktury/dovecot.index
> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
> session=, Error: Fixed index file 
> /var/mail/piast_efaktury/dovecot.index: log_file_seq 13 -> 15
> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
> session=, Panic: file mail-transaction-log-file.c: 
> line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)
> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
> session=, Error: Raw backtrace: 
> /usr/lib64/dovecot/libdovecot.so.0(+0xe011b) [0x7f25fc82b11b] -> 
> /usr/lib64/dovecot/libdovecot.so.0(+0xe01b1) [0x7f25fc82b1b1] -> 
> /usr/lib64/dovecot/libdovecot.so.0(+0x4bf56) [0x7f25fc796f56] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x4b17e) [0x7f25fc92f17e] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_transaction_logs_clean+0x5c) 
> [0x7f25fc9e6e3c] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_transaction_log_close+0x34) 
> [0x7f25fc9e6f04] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_transaction_log_move_to_memory+0xed)
>  [0x7f25fc9e709d] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_move_to_memory+0x60) 
> [0x7f25fc9e10b0] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_write+0x1e1) 
> [0x7f25fc9df3a1] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_fsck+0x68a) 
> [0x7f25fc9ca13a] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_map+0x5b0) 
> [0x7f25fc9d4090] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_map+0x13b) 
> [0x7f25fc9cc1eb] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xfcc96) 
> [0x7f25fc9e0c96] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xfd2a7) 
> [0x7f25fc9e12a7] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_open+0x7a) 
> [0x7f25fc9e13aa] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(index_storage_mailbox_open+0xac) 
> [0x7f25fc9bac1c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x84df9) 
> [0x7f25fc968df9] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x84ecf) 
> [0x7f25fc968ecf] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0xbe11c) 
> [0x7f25fc9a211c] -> /usr/lib64/dovecot/plugins/lib20_zlib_plugin.so(+0x46ee) 
> [0x7f25f8ff86ee] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x5bd56) 
> [0x7f25fc93fd56] -> 
> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_open+0x4a) 
> [0x7f25fc93ff3a] -> 
> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver_save_open+0x63) 
> [0x7f25fca49cc3] -> 
> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver_save+0xc9) 
> [0x7f25fca49f29] -> 
> /usr/lib64/dovecot/libdovecot-lda.so.0(mail_deliver+0x1c6) [0x7f25fca4a586] 
> -> dovecot/lmtp [local DATA](lmtp_local_data+0x6a1) [0x55e3e295a551] -> 
> dovecot/lmtp [local DATA](cmd_data_continue+0x25d) [0x55e3e295920d] -> 
> /usr/lib64/dovecot/libdovecot.so.0(+0x65737) [0x7f25fc7b0737]
> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
> session=, Fatal: master: service(lmtp): child 24428 
> killed with signal 6 (core dumped)
> ~


lmtp backtrace: http://ixion.pld-linux.org/~arekm/dovecot/lmtp-gdb.txt
log: http://ixion.pld-linux.org/~arekm/dovecot/lmtp.txt

pop3 backtrace: http://ixion.pld-linux.org/~arekm/dovecot/pop3-gdb.txt
log: http://ixion.pld-linux.org/~arekm/dovecot/pop3.txt


deleting dovecot.index* helps until problem starts again like - first
entry with problem after deleting indexes:

> Mar 26 14:18:12 mbox dovecot[10001]: pop3-login: Login: 
> user=, method=PLAIN, rip=1.1.1.1, lip=2.2.2.2, mpid=5694, 
> session=
> Mar 26 14:18:12 mbox dovecot[10001]: pop3(piast_efaktury): pid=<5694> 
> session=, Error: Index 
> /var/mail/piast_efaktury/dovecot.index: Lost log for seq=13 offset=25648: 
> Missing middle file seq=13 (between 13..4294967295, we have seqs 14,15): 
> .log.2 contains file_seq=14 (initial_mapped=0, reason=Index mapped)
> Mar 26 14:18:12 mbox dovecot[10001]: pop3(piast_efaktury): pid=<5694> 
> session=, Warning: fscking index file 
> /var/mail/piast_efaktury/dovecot.index
> Mar 26 14:18:12 mbox 

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Timo Sirainen via dovecot]

On 27 Mar 2019, at 1.25, Jason Lewis via dovecot  wrote:
> 
> Hi Aki,
> 
> debian jessie backports has been moved to archive.debian.org and
> initially I was unable to install dovecot-dbg because of that. But I've
> managed to resolve that issue now.
> 
> This was the command I ran:
> doveadm -D -f flow fetch imap.envelope mailbox crm-spam.2008.g
> 
> Backtrace follows.

I've a feeling Debian's security fix backports didn't work properly:

> #5  0x7f3a7c34a97d in rfc822_parser_deinit (ctx=0x7ffdc7615e38,
> ctx=0x7ffdc7615e38) at rfc822-parser.h:23

rfc822_parser_deinit() wasn't added until v2.2.31. I think it was added as part 
of a security fix.

>data=data@entry=0x5563c13f3910 "To: bluef...@dickson.st,
> ja...@dickson.st, lewisja...@dickson.st, 05 Jul 2008 16:39:47 -0500
> PDT6Q--q=dns; c=nofws;d sender)
> smtp.mail=matt_coo...@postnewsweektech.com; domainkeys=pass (test mode)
> hea"..., size=size@entry=64,

I tried fetching a mail with these contents in v2.2.27, v2.2.33 and master. 
They all worked fine.

IMAP coredumps for one user

[Odhiambo Washington via dovecot]

FreeBSD-12
Dovecot-2.3.5

I am having problems with one use

Mar 25 21:30:12 imap(gau@crownkenya.com)<91364>:
Fatal: master: service(imap): child 91364 killed with signal 6 (core dumped)
Mar 25 21:30:14 imap(gau@crownkenya.com)<2381>:
Fatal: master: service(imap): child 2381 killed with signal 6 (core dumped)
Mar 26 06:29:26 imap(gau@crownkenya.com)<7644><0aVP7faEI/GaTGQE>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:26 imap(gau@crownkenya.com)<8806>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:26 imap(gau@crownkenya.com)<8493>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:26 imap(gau@crownkenya.com)<9412><2qtP7faEIfGaTGQE>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:28 imap(gau@crownkenya.com)<7644><0aVP7faEI/GaTGQE>:
Fatal: master: service(imap): child 7644 killed with signal 6 (core dumped)
Mar 26 06:29:29 imap(gau@crownkenya.com)<10764><7sW+7faEJvGaTGQE>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:31 imap(gau@crownkenya.com)<8806>:
Fatal: master: service(imap): child 8806 killed with signal 6 (core dumped)
Mar 26 06:29:33 imap(gau@crownkenya.com)<8493>:
Fatal: master: service(imap): child 8493 killed with signal 6 (core dumped)
Mar 26 06:29:36 imap(gau@crownkenya.com)<9412><2qtP7faEIfGaTGQE>:
Fatal: master: service(imap): child 9412 killed with signal 6 (core dumped)
Mar 26 06:29:38 imap(gau@crownkenya.com)<10764><7sW+7faEJvGaTGQE>:
Fatal: master: service(imap): child 10764 killed with signal 6 (core dumped)
Mar 26 06:29:57 imap(gau@crownkenya.com)<13285>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:57 imap(gau@crownkenya.com)<19736>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:57 imap(gau@crownkenya.com)<22060>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:29:57 imap(gau@crownkenya.com)<21367>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:30:00 imap(gau@crownkenya.com)<13285>:
Fatal: master: service(imap): child 13285 killed with signal 6 (core dumped)
Mar 26 06:30:00 imap(gau@crownkenya.com)<49026>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 06:30:02 imap(gau@crownkenya.com)<19736>:
Fatal: master: service(imap): child 19736 killed with signal 6 (core dumped)
Mar 26 06:30:05 imap(gau@crownkenya.com)<22060>:
Fatal: master: service(imap): child 22060 killed with signal 6 (core dumped)
Mar 26 06:30:07 imap(gau@crownkenya.com)<21367>:
Fatal: master: service(imap): child 21367 killed with signal 6 (core dumped)
Mar 26 06:30:10 imap(gau@crownkenya.com)<49026>:
Fatal: master: service(imap): child 49026 killed with signal 6 (core dumped)
Mar 26 07:59:40 imap(gau@crownkenya.com)<27221><06UEMPiESPKaRh6E>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 07:59:40 imap(gau@crownkenya.com)<28883>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 07:59:40 imap(gau@crownkenya.com)<29823><28oEMPiESfKaRh6E>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 07:59:40 imap(gau@crownkenya.com)<27530><04UEMPiER/KaRh6E>:
Panic: file mempool-system.c: line 137 (pool_system_realloc): assertion
failed: (old_size == (size_t)-1 || mem == NULL || old_size <=
malloc_usable_size(mem))
Mar 26 07:59:42 imap(gau@crownkenya.com)<27221><06UEMPiESPKaRh6E>:
Fatal: master: service(imap): child 27221 killed with signal 6 (core dumped)
Mar 26 07:59:42 imap(gau@crownkenya.com)<30775>:
Panic: file mempool-system.c: line 137 

MailCrypt: Encrypted user keys configuration with LDAP & cryptokey generate

[FELINN via dovecot]

Hi, I try to use the MailCrypt plugin with Floder encryption and
encrypted user keys, using LDAP. I use Dovecot 2.2.27 (c0f36b0)
I follow the wiki: https://wiki2.dovecot.org/Plugins/MailCrypt

doveconf -n and dovecot-ldap.conf.ext attached to this message.

I well configured slapd to let dovecot's dn query the userPassword
(hashed password SSHA). I use fusiondirectory-mail plugin:


$ ldapsearch -D 'cn=dovecot,ou=dsa,dc=foo,dc=bar' -W -LLL 
'(&(objectClass=gosaMailAccount)(objectClass=posixAccount)(uid=))' 
'userPassword'
dn: cn=,ou=people,dc=foo,dc=bar
userPassword:: 


The problem is that mails still readable and no keys are generated, even
if a send a mail to this address, or login through webmail. I wait more
than 1h until something happens, Cf:
https://dovecot.org/list/dovecot/2018-September/112763.html

If I try to generate keys manually I get this error:


$ doeveadm mailbox cryptokey generate -u 
doveadm(): Error: mail_crypt_user_generate_keypair() failed: 
mail_crypt_require_encrypted_user_key set, cannot generate user keypair without 
password or key
   Folder Public ID
x ERROR: mail_crypt_require_encrypted_user_key set, cannot generate 
user keypair without password or key
doveadm(): Warning: Timeout leak: 0x7f0c439c0180 
(mail-index-alloc-cache.c:240)


It works with -o plugin/mail_crypt_private_password= of
course, but by hand it's not the goal ><

I probably miss something, I guess that the part of the wiki about sql
and password_query is only for configuration that use SQL for dbuser. Is
there similar things to do with LDAP?

Thank you very much for your time.

-- 
f00wl
FELINN https://felinn.org

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Jason Lewis via dovecot]

Hi Aki,

debian jessie backports has been moved to archive.debian.org and
initially I was unable to install dovecot-dbg because of that. But I've
managed to resolve that issue now.

This was the command I ran:
doveadm -D -f flow fetch imap.envelope mailbox crm-spam.2008.g

Backtrace follows.

Jason

jason@debian:~$ gdb /usr/bin/doveadm /home/jason/core
GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/doveadm...Reading symbols from
/usr/lib/debug/.build-id/8a/e850dc3cde00618eb0c3386b7404fe984c8118.debug...done.
done.
[New LWP 23099]
Core was generated by `/usr/bin/doveadm -D -f flow fetch imap.envelope
mailbox crm-spam.2008.g'.
Program terminated with signal SIGABRT, Aborted.
#0  0x7f3a7bf58067 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
56  ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt full
#0  0x7f3a7bf58067 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
resultvar = 0
pid = 23099
selftid = 23099
#1  0x7f3a7bf59448 in __GI_abort () at abort.c:89
save_stage = 2
act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction =
0x0}, sa_mask = {__val = {1024, 93886903596007, 93886903463748,
  139889164438341, 93886903464748, 0, 93886931979560, 513,
12274393185022739456, 93886931979560, 139889168940995, 93886931979560,
  140727948500176, 4294967040, 139889168941353,
93886931979560}}, sa_flags = 2083919594, sa_restorer = 0x7ffdc7615d01}
sigs = {__val = {32, 0 }}
#2  0x7f3a7c3669a6 in default_fatal_finish (type=,
status=status@entry=0) at failures.c:201
backtrace = 0x5563c13acd60
"/usr/lib/dovecot/libdovecot.so.0(+0x989ae) [0x7f3a7c3669ae] ->
/usr/lib/dovecot/libdovecot.so.0(+0x98a28) [0x7f3a7c366a28] ->
/usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3a7c2fc67e] ->
/usr/lib/d"...
#3  0x7f3a7c366a28 in default_fatal_handler (ctx=0x7ffdc7615d20,
format=, args=) at failures.c:215
status = 0
#4  0x7f3a7c2fc67e in i_panic (format=format@entry=0x7f3a7c399088
"file %s: line %d (%s): assertion failed: (%s)") at failures.c:275
ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0,
timestamp_usecs = 0}
args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area =
0x7ffdc7615e20, reg_save_area = 0x7ffdc7615d60}}
#5  0x7f3a7c34a97d in rfc822_parser_deinit (ctx=0x7ffdc7615e38,
ctx=0x7ffdc7615e38) at rfc822-parser.h:23
No locals.
#6  message_address_parse_real (pool=pool@entry=0x5563c13e75d0,
data=data@entry=0x5563c13f3910 "To: bluef...@dickson.st,
ja...@dickson.st, lewisja...@dickson.st, 05 Jul 2008 16:39:47 -0500
PDT6Q--q=dns; c=nofws;d sender)
smtp.mail=matt_coo...@postnewsweektech.com; domainkeys=pass (test mode)
hea"..., size=size@entry=64,
max_addresses=max_addresses@entry=4294967295,
fill_missing=fill_missing@entry=true) at message-address.c:323
ctx = {pool = 0x5563c13e75d0, parser = {
data = 0x5563c13f3951 " 05 Jul 2008 16:39:47 -0500
PDT6Q--q=dns; c=nofws;d sender)
smtp.mail=matt_coo...@postnewsweektech.com; domainkeys=pass (test mode)
header.From=matt_coo...@postnewsweektech.com",
end = 0x5563c13f3950 ", 05 Jul 2008 16:39:47 -0500
PDT6Q--q=dns; c=nofws;d sender)
smtp.mail=matt_coo...@postnewsweektech.com; domainkeys=pass (test mode)
header.From=matt_coo...@postnewsweektech.com", last_comment =
0x5563c13acb78}, first_addr = 0x5563c13e7910,
  last_addr = 0x5563c13e7a28, addr = {next = 0x0, name = 0x0,
route = 0x0, mailbox = 0x0, domain = 0x0, invalid_syntax = false},
  str = 0x5563c13acc50, fill_missing = true}
#7  0x7f3a7c34a9e5 in message_address_parse
(pool=pool@entry=0x5563c13e75d0,
data=0x5563c13f3910 "To: bluef...@dickson.st, ja...@dickson.st,
lewisja...@dickson.st, 05 Jul 2008 16:39:47 -0500 PDT6Q--q=dns;
c=nofws;d sen---Type  to continue, or q  to quit---
der) smtp.mail=matt_coo...@postnewsweektech.com; domainkeys=pass (test
mode) hea"..., size=64, max_addresses=max_addresses@entry=4294967295,
fill_missing=fill_missing@entry=true) at message-address.c:338
_data_stack_cur_id = 4
addr = 
#8  0x7f3a7c33e374 in imap_envelope_parse_header
(pool=0x5563c13e75d0, data=data@entry=0x5563c13e6ff0,

Re: Panic: file mail-transaction-log-file.c: line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

[Timo Sirainen via dovecot]

On 27 Mar 2019, at 14.58, Timo Sirainen via dovecot  wrote:
> 
>> dovecot isn't able to auto fix the indexes and manual deletion is
>> required in all such cases
> 
> So if it keeps repeating, it's very strange. Could you send me such broken 
> dovecot.index and dovecot.index.log files (without dovecot.index.cache)? They 
> shouldn't contain anything sensitive (only message flags).

Tested with the index files you sent. It gets fixed automatically in my tests.

The backtrace shows that after fsck it fails to write the fixed index to the 
disk, because mail_index_write() fails for some reason. Except there's no error 
logged about it, which is rather weird. Do you still have the lmtp core? Could 
you do:

fr 9
p *log.index

Test SASL authentication via telnet (or similar)

[Alessio Cecchi via dovecot]

Hi,

I'm looking for a way to autenticate my email users via Dovecot SASL TCP 
connections from an external nodejs or python script.


Dovecot configuration is fine, if I set in postfix smtpd_sasl_path = 
inet:127.0.0.1:12345 works fine.


But if a try via "telnet 127.0.0.1 12345" to chat with SASL in dovecot 
log found:


dovecot: auth: Error: Authentication client not compatible with this 
server (mixed old and new binaries?)


What is the right way to chat with SASL via TCP port?

Thanks

--
Alessio Cecchi
Postmaster @ http://www.qboxmail.it
https://www.linkedin.com/in/alessice

Re: v2.3.5.1 released

[Marcelo Coelho via dovecot]

Hi,

Why didn’t you apply this patch to v2.3.5.1?


commit df8addd41d87e61113de22a21a0e61506a8d74c2
Author: Stephan Bosch 
Date:   Tue Mar 12 03:18:33 2019 +0100

   submission-login: client-authenticate - Fix crash occurring when client 
disconnects during authentication.

diff --git a/src/submission-login/client-authenticate.c 
b/src/submission-login/client-authenticate.c
index 8b5422f833..6b70701a1a 100644
--- a/src/submission-login/client-authenticate.c
+++ b/src/submission-login/client-authenticate.c
@@ -98,6 +98,9 @@ void submission_client_auth_result(struct client *client,
   container_of(client, struct submission_client, common);
   struct smtp_server_cmd_ctx *cmd = subm_client->pending_auth;

+   if (subm_client->conn == NULL)
+   return;
+
   subm_client->pending_auth = NULL;
   i_assert(cmd != NULL);

diff --git a/src/submission-login/client.c b/src/submission-login/client.c
index 3e45e556c7..212afb92cf 100644
--- a/src/submission-login/client.c
+++ b/src/submission-login/client.c
@@ -212,6 +212,8 @@ static void client_connection_disconnect(void *context, 
const char *reason)
{
   struct submission_client *client = context;

+   client->pending_auth = NULL;
+   client->pending_starttls = NULL;
   client_disconnect(>common, reason);
}


> On 28 Mar 2019, at 08:41, Aki Tuomi via dovecot  wrote:
> 
> https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz
> https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz.sig
> Binary packages in https://repo.dovecot.org/
> 
> * CVE-2019-7524: Missing input buffer size validation leads into
>   arbitrary buffer overflow when reading fts or pop3 uidl header
>   from Dovecot index. Exploiting this requires direct write access to
>   the index files.
> 
> ---
> Aki Tuomi
> Open-Xchange oy
> 

Restoring mailboxes from backup duplicates messages in POP clients

[Luis F. V. Gomes via dovecot]

Hello

I had a disk problem and had to reformat it. All mailboxes were backed 
up using rsync.
After I restored the mailboxes, the POP clients (Thunderbird) that were 
configured to leave the messages on the mailserver for, let's say, 30 
days, didn't understand that some messages were already transfered and 
the users got duplicated messages in their Inbox.

How can we avoid this?

--

Thanks

Luís F. Gomes
IT Manager
(55)(21) 3527-1220
Departamento de Engenharia Elétrica
PUC-Rio
R. Marques de S. Vicente 225/401L
22451-900 - Rio de Janeiro/RJ

Re: v2.2.27 Panic: file rfc822-parser.h: line 23 (rfc822_parser_deinit): assertion failed: (ctx->data <= ctx->end)

[Jason Lewis via dovecot]

Thanks Timo.

Given the age of these dovecot packages, and this being on debian
oldstable, what should we do next? I'm inclined to just delete the email
in question and move on.

Jason

Timo Sirainen wrote on 28/3/19 12:16 am:
> On 27 Mar 2019, at 1.25, Jason Lewis via dovecot  wrote:
>>
>> Hi Aki,
>>
>> debian jessie backports has been moved to archive.debian.org and
>> initially I was unable to install dovecot-dbg because of that. But I've
>> managed to resolve that issue now.
>>
>> This was the command I ran:
>> doveadm -D -f flow fetch imap.envelope mailbox crm-spam.2008.g
>>
>> Backtrace follows.
> 
> I've a feeling Debian's security fix backports didn't work properly:
> 
>> #5  0x7f3a7c34a97d in rfc822_parser_deinit (ctx=0x7ffdc7615e38,
>> ctx=0x7ffdc7615e38) at rfc822-parser.h:23
> 
> rfc822_parser_deinit() wasn't added until v2.2.31. I think it was added as 
> part of a security fix.
> 
>>data=data@entry=0x5563c13f3910 "To: bluef...@dickson.st,
>> ja...@dickson.st, lewisja...@dickson.st, 05 Jul 2008 16:39:47 -0500
>> PDT6Q--q=dns; c=nofws;d sender)
>> smtp.mail=matt_coo...@postnewsweektech.com; domainkeys=pass (test mode)
>> hea"..., size=size@entry=64,
> 
> I tried fetching a mail with these contents in v2.2.27, v2.2.33 and master. 
> They all worked fine.
> 

-- 
Jason Lewis
http://emacstragic.net

Re: Panic: file mail-transaction-log-file.c: line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

[Timo Sirainen via dovecot]

On 27 Mar 2019, at 12.42, Arkadiusz Miśkiewicz via dovecot 
 wrote:
> 
> 
> Hello.
> 
> I have one account with heavy traffic (big mails) and quite often
> indexes get corrupted.
> 
> This is dovecot 2.3.5 on local fs (xfs), Linux kernel 4.19.20, glibc 2.28.
> 
> When corruption happens lmtp and pop3 segfault on accessing it like:
> 
>> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(24428): Connect from local 
>>  
>>  
>> [0/0]
>> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
>> session=, Error: Index 
>> /var/mail/piast_efaktury/dovecot.index: Lost log for seq=13 offset=25648: 
>> Missing middle file seq=13 (between 13..4294967295, we have seqs 14,15): 
>> .log.2 contains file_seq=14 (initial_mapped=0, reason=Index mapped)
>> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
>> session=, Warning: fscking index file 
>> /var/mail/piast_efaktury/dovecot.index
>> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
>> session=, Error: Fixed index file 
>> /var/mail/piast_efaktury/dovecot.index: log_file_seq 13 -> 15

dovecot.index says that it was generated against dovecot.index.log sequence 13, 
but the .log file already has sequence 15. I could maybe believe such a bug 
with long-running IMAP connections, but this is LMTP. And it's supposed to be 
fixing the problem here..

>> Mar 27 11:13:50 mbox dovecot[22370]: lmtp(piast_efaktury): pid=<24428> 
>> session=, Panic: file mail-transaction-log-file.c: 
>> line 105 (mail_transaction_log_file_free): assertion failed: (!file->locked)

Even though it crashes here, it's already supposed to have fixed the problem.

> dovecot isn't able to auto fix the indexes and manual deletion is
> required in all such cases

So if it keeps repeating, it's very strange. Could you send me such broken 
dovecot.index and dovecot.index.log files (without dovecot.index.cache)? They 
shouldn't contain anything sensitive (only message flags).

Also what's your doveconf -n?

Re: Weird things in the mail queue

[Aki Tuomi via dovecot]

> On 24 March 2019 12:43 Daniel Lange  wrote:
> 
>  
> Hi Aki,
> 
> Am 21.02.19 um 12:55 schrieb Aki Tuomi:
> > 
> > On 21.2.2019 13.47, Lionel Elie Mamane via dovecot wrote:
> >> I noticed a mail stuck in my mail queue. dovecot-lda was returning
> >> error 64 Invalid parameter given. (EX_USAGE).
> >>
> >> Weird, weird, weird. After some sleuthing, I found the sender address
> >> was firstl...@domain.tld, with a UTF8-encoded Unicode U+FEFF ZERO
> >> WIDTH NO-BREAK SPACE character (AKA byte order mark) between "First"
> >> and "Last" :)
> >>
> >> Since that is passed as the -f parameter to dovecot-lda, it was giving
> >> the 64 error.
> > 
> > Your MTA should not be passing this along.
> 
> Unfortunately Postfix does.
> It honors the robustness principle (~Postel's law) and therefore
> accepts envelope senders like
> 
> from=
> or
> from=sm...@nampaichuanlondon.com>
> or
> from=
> (invalid 3-byte UTF-8 .)
> 
> which are increasingly making rounds.
> 
> With a working local delivery these will just feed spamassassin or
> rspamd and all is well. And may be the occasional poor Exchange
> customer's email is delivered, too.
> 
> With Dovecot 2.3.4.1 and 2.3.5 dovecot-lda and lmtp
> these will generate bounces that lead to backscatter spam:
> 
> postfix/pipe[22438]: D8C5E35C2600: to=, relay=dovecot, 
> delay=0.22, delays=0.14/0.01/0/0.08, dsn=5.3.0, status=bounced (command line 
> usage error. Command output: lda: Fatal: Invalid -f parameter: Invalid 
> character in localpart )
> postfix/cleanup[22433]: 0D95435C25EF: message-id=
> postfix/bounce[22440]: D8C5E35C2600: sender non-delivery notification: 
> 0D95435C25EF
> (dovecot-lda case)
> 
> and
> 
> postfix/lmtp[12829]: 6ADF135C2671: to=, 
> relay=redacted[private/dovecot-lmtp], delay=0.17, delays=0.15/0.01/0.01/0, 
> dsn=5.5.2, status=bounced (host redacted[private/dovecot-lmtp] said: 500 
> 5.5.2 Invalid command syntax (in reply to MAIL FROM command))
> ...
> (lmtp case)
> 
> In my book an LDA should do its job and deliver the email. It should
> complain about an invalid envelope sender if (and only if) it needs to
> send a bounce (and thus will send that to MAILER-DAEMON instead). But it
> must in no case refuse to deliver the email to a perfectly valid local
> recipient. Yes, the envelope sender is flawed. But that is not the LDAs
> concern. If the SMTPD was happy enough, the email has been accepted and
> must not lead to a late bounce. In the case we're currently seeing this
> leads to backscatter spam.
> 
> I think the right logic would be to not smtp_address_parse_path the
> enveloper sender unless it is needed for legitimate bounces (e.g.
> mailbox over quota). In that case a failing enveloper sender should be
> replaced for the bounce. In any other case dovecot-lda and lmtp must not
> bother.
> 
> Kind regards,
> Daniel

We are tracking this as DOP-1045. 

Aki

[Dovecot-news] v2.3.5.1 released

[Aki Tuomi via Dovecot-news]

https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz.sig
Binary packages in https://repo.dovecot.org/

    * CVE-2019-7524: Missing input buffer size validation leads into
  arbitrary buffer overflow when reading fts or pop3 uidl header
  from Dovecot index. Exploiting this requires direct write access to
  the index files.

---
Aki Tuomi
Open-Xchange oy



signature.asc
Description: OpenPGP digital signature
___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news

v2.2.36.3 released

[Aki Tuomi via dovecot]

https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz.sig

    * CVE-2019-7524: Missing input buffer size validation leads into
  arbitrary buffer overflow when reading fts or pop3 uidl header
  from Dovecot index. Exploiting this requires direct write access to
  the index files.

---
Aki Tuomi
Open-Xchange oy

[Dovecot-news] CVE-2019-7524: Buffer overflow when reading extension header from dovecot index files

[Aki Tuomi via Dovecot-news]

Product: Dovecot
Vendor: OX Software GmbH
Internal reference: DOV-2964 (Bug ID)
Vulnerability type: CWE-120
Vulnerable version: 2.0.14 - 2.3.5
Vulnerable component: fts, pop3-uidl-plugin
Report confidence: Confirmed
Researcher credits: Found in internal testing
Solution status: Fixed by Vendor
Fixed version: 2.3.5.1, 2.2.36.3
Vendor notification: 2019-02-05
Solution date: 2019-03-21
Public disclosure: 2019-03-28
CVE reference: CVE-2019-7524
CVSS: 3.0/AV:L/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H/E:P/RL:O/RC:C (8.8)
 
Vulnerability Details:
When reading FTS or POP3-UIDL header from dovecot index, the input
buffer size is not bound, and data is copied to target structure causing
stack overflow.

Risk:
This can be used for local root privilege escalation or executing
arbitrary code in dovecot process context. This requires ability to
directly modify dovecot indexes.
Steps to reproduce:
Produce dovecot.index.log entry that creates an FTS header which has
more than 12 bytes of data.
Trigger dovecot indexer-worker or run doveadm index.
Dovecot will crash.

Mitigations:
Since 2.3.0 dovecot has been compiled with stack smash protection, ASLR,
read-only GOT tables and other techniques that make exploiting this bug
much harder.

Solution:
Operators should update to the latest Patch Release. The only workaround
is to disable FTS and pop3-uidl plugin.

--
Aki Tuomi
Open-Xchange Oy



signature.asc
Description: OpenPGP digital signature
___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news

[Dovecot-news] v2.2.36.3 released

[Aki Tuomi via Dovecot-news]

https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz
https://dovecot.org/releases/2.3/dovecot-2.2.36.3.tar.gz.sig

    * CVE-2019-7524: Missing input buffer size validation leads into
  arbitrary buffer overflow when reading fts or pop3 uidl header
  from Dovecot index. Exploiting this requires direct write access to
  the index files.

---
Aki Tuomi
Open-Xchange oy

___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news

Re: v2.3.5.1 released

[Aki Tuomi via dovecot]

2.3.5.1 was only for releasing CVE. We have decided not to add
non-related fixes into patch releases containing CVE releases for clarity.

Aki

On 28.3.2019 13.57, Marcelo Coelho via dovecot wrote:
> Hi,
>
> Why didn’t you apply this patch to v2.3.5.1?
>
>
> commit df8addd41d87e61113de22a21a0e61506a8d74c2
> Author: Stephan Bosch 
> Date:   Tue Mar 12 03:18:33 2019 +0100
>
>submission-login: client-authenticate - Fix crash occurring when client 
> disconnects during authentication.
>
> diff --git a/src/submission-login/client-authenticate.c 
> b/src/submission-login/client-authenticate.c
> index 8b5422f833..6b70701a1a 100644
> --- a/src/submission-login/client-authenticate.c
> +++ b/src/submission-login/client-authenticate.c
> @@ -98,6 +98,9 @@ void submission_client_auth_result(struct client *client,
>container_of(client, struct submission_client, common);
>struct smtp_server_cmd_ctx *cmd = subm_client->pending_auth;
>
> +   if (subm_client->conn == NULL)
> +   return;
> +
>subm_client->pending_auth = NULL;
>i_assert(cmd != NULL);
>
> diff --git a/src/submission-login/client.c b/src/submission-login/client.c
> index 3e45e556c7..212afb92cf 100644
> --- a/src/submission-login/client.c
> +++ b/src/submission-login/client.c
> @@ -212,6 +212,8 @@ static void client_connection_disconnect(void *context, 
> const char *reason)
> {
>struct submission_client *client = context;
>
> +   client->pending_auth = NULL;
> +   client->pending_starttls = NULL;
>client_disconnect(>common, reason);
> }
>
>
>> On 28 Mar 2019, at 08:41, Aki Tuomi via dovecot  wrote:
>>
>> https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz
>> https://dovecot.org/releases/2.3/dovecot-2.3.5.1.tar.gz.sig
>> Binary packages in https://repo.dovecot.org/
>>
>> * CVE-2019-7524: Missing input buffer size validation leads into
>>   arbitrary buffer overflow when reading fts or pop3 uidl header
>>   from Dovecot index. Exploiting this requires direct write access to
>>   the index files.
>>
>> ---
>> Aki Tuomi
>> Open-Xchange oy
>>

Re: MailCrypt: Encrypted user keys configuration with LDAP & cryptokey generate

[FELINN via dovecot]

Here are attachments.

-- 
f00wl
FELINN https://felinn.org
# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.15.18-9-pve x86_64 Debian 9.8
auth_username_format = %n
auth_verbose = yes
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
login_trusted_networks = 192.168.10.100/24
mail_attribute_dict = file:%h/mail/dovecot-attributes
mail_location = mbox:~/mail
mail_plugins = quota quota fts fts_squat mail_crypt
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/conf.d/dovecot-ldap.conf.ext
  driver = ldap
}
passdb {
  driver = pam
}
plugin {
  mail_crypt_curve = secp521r1
  mail_crypt_require_encrypted_user_key = yes
  mail_crypt_save_version = 2
  quota = fs:User quota:user
  quota2 = fs:Disk quota
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_dir = ~/sieve
}
protocols = imap pop3 lmtp
service lmtp {
  inet_listener lmtp {
port = 24
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
}
shutdown_clients = no
ssl_cert = 

dovecot-ldap.conf.ext
Description: application/vnd.novadigm.ext