Re: Moving emails tagged as spam to the Junk folder

[Larry Rosenman]

You need Dovecot 2.2+ and Pigeonhole.


-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: larry...@gmail.com
US Mail: 5708 Sabbia Drive, Round Rock, TX 78665-2106
 

On 9/11/17, 5:09 PM, "dovecot on behalf of Mika Leppänen" 
 wrote:

  I am running Ubuntu 16.04. I am following this guide: 
https://wiki2.dovecot.org/HowTo/AntispamWithSieve. At the end when I try 
to compile the .sieve files, I get this error: sievec(root): Fatal: 
Plugin 'sieve_imapsieve' not found from directory 
/usr/lib/dovecot/modules/sieve.  Does anyone know about this?
I have dovecot-sieve installed and I'm using Postfix and Spamassassin.

-- 
Mika Leppänen -- mika.leppa...@iki.fi

Moving emails tagged as spam to the Junk folder

[Mika Leppänen]

 I am running Ubuntu 16.04. I am following this guide: 
https://wiki2.dovecot.org/HowTo/AntispamWithSieve. At the end when I try 
to compile the .sieve files, I get this error: sievec(root): Fatal: 
Plugin 'sieve_imapsieve' not found from directory 
/usr/lib/dovecot/modules/sieve.  Does anyone know about this?

I have dovecot-sieve installed and I'm using Postfix and Spamassassin.

--
Mika Leppänen -- mika.leppa...@iki.fi

Re: Dovecot and Letsencrypt certs

[Joseph Tam]

 writes:


"writing a script to check the certs" - there is no need to write any
scripts. As one mentioned, it's done by a hook to certbot. Please read
the manuals for LE or certbot. The issue you have is quite common and
of course certbot designed to do it for you.


Won't work, of course, if you employ the least-privilege security principle
and run the certbot as a non-privileged user.  You'll need a script with
administrator privileges to detect cert renewals and restart the service.

I can't willy-nilly restart dovecot to pick up renewed certs without
webmail disruptions.  (My webmail uses persistent IMAP sessions.)
All users get dumped and need to re-authenticate.  If a user happens to
be drafting a message that took 2 hours to compose, I will surely hear
about it.  I should probably install a IMAP proxy to isolate the effects
of restarts.  Most mail readers cope with restarts just fine, though.

Joseph Tam 

Re: Share Website certificate with SSL/STL Dovecot IMAP and Postifix SMTP

[Joseph Tam]

Marco Marco writes writes:


I've bought a certificate from the authority for my website to use to
access in https mode.

Is it possible to share the same pairs to authenticate the emails sent
by postfix and Dovecot in order to avoid that client as Hotmail.it or
Gmail intercept these as Spam?


By "same pairs", I assume you mean key and certificate.

Yes, just as long as you use the same Subject for all your services (e.g.
"mail.yourdomain.com") or you have purchased a wildcard certificate
e.g. "*.yourdomain.com" to authenticate "webmail.yourdomain.com",
"imap.yourdomain.com", "smtp.yourdomain.com", etc.)

Certificates do not authenticate services or protocols, but server names
(typically).

Joseph Tam 

Re: sieve stopped working and doveadm mailbox list without -s shows less folders then with

[Ralf Becker]

Noone an idea how it can be that subscribed folders are more then all
folders and how to repair that situation?

Ralf

Am 07.09.17 um 16:42 schrieb Ralf Becker:
> Dovecot 2.2.31 with mailboxes in mdbox format.
>
> Since a couple of days some mailboxes have the problem, that sieve rules
> moving mails to folders stop working and .sieve.log in mailbox shows:
>
> sieve: info: started log at Sep 07 13:57:17.
> error:
> msgid=<20170907155704.egroupware.s4ythvjrr12wsijlpkbk...@somedomain.egroupware.de>:
> failed to store into mailbox 'INBOX/Munser': Mailbox doesn't exist:
> INBOX/Munser.
>
> When I do a doveadm mailbox list -s -u @ I get all folders
> incl. the one mentioned above, while doveadm mailbox list without -s
> shows just
> user
> INBOX
>
> I already tried doveadm force-resync -u @ INBOX, but it
> did not change anything.
>
> I also moved the mailbox in filesystem to an other name and tried to
> restore it from there, which helped with most broken mailbox problems in
> the pre 2.2.31 aftermath, but that failed completly:
>
> /var/dovecot/imap/ # mv  .broken
>
> /var/dovecot/imap/ # doveadm force-resync -u @ INBOX
>
> /var/dovecot/imap/ # sudo -u dovecot doveadm -Dv import -u
> @ -s mdbox:$(pwd)/.broken/mdbox
> INBOX all
> Debug: Loading modules from directory: /usr/lib/dovecot
> Debug: Module loaded: /usr/lib/dovecot/lib01_acl_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/lib10_quota_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/lib15_notify_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/lib20_mail_log_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/lib20_replication_plugin.so
> Debug: Loading modules from directory: /usr/lib/dovecot/doveadm
> Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so
> Debug: Skipping module doveadm_expire_plugin, because dlopen() failed:
> Error relocating
> /usr/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so:
> expire_set_lookup: symbol not found (this is usually intentional, so
> just ignore this message)
> Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so
> Debug: Module loaded: /usr/lib/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
> Debug: Skipping module doveadm_fts_plugin, because dlopen() failed:
> Error relocating /usr/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so:
> fts_backend_rescan: symbol not found (this is usually intentional, so
> just ignore this message)
> Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen()
> failed: Error relocating
> /usr/lib/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so:
> mail_crypt_box_get_public_key: symbol not found (this is usually
> intentional, so just ignore this message)
> doveadm(@ 54303): Debug: Added userdb setting:
> plugin/master_user=@
> doveadm(@ 54303): Debug: Added userdb setting:
> plugin/userdb_acl_groups=admins@,hts büro@,hts@
> doveadm(@ 54303): Debug: Added userdb setting:
> plugin/userdb_quota_rule=*:bytes=1048576
> doveadm(@): Debug: Effective uid=90, gid=101,
> home=/var/dovecot/imap//
> doveadm(@): Debug: Quota root: name=User quota
> backend=dict
> args=:ns=INBOX/:file:/var/dovecot/imap///dovecot-quota
> doveadm(@): Debug: Quota rule: root=User quota mailbox=*
> bytes=107374182400 messages=0
> doveadm(@): Debug: Quota grace: root=User quota
> bytes=10737418240 (10%)
> doveadm(@): Debug: dict quota: user=@,
> uri=file:/var/dovecot/imap///dovecot-quota, noenforcing=0
> doveadm(@): Debug: Namespace inboxes: type=private,
> prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=no
> location=mdbox:~/mdbox
> doveadm(@): Debug: fs:
> root=/var/dovecot/imap///mdbox, index=, indexpvt=,
> control=, inbox=, alt=
> doveadm(@): Debug: acl: initializing backend with data: vfile
> doveadm(@): Debug: acl: acl username = @
> doveadm(@): Debug: acl: owner = 0
> doveadm(@): Debug: acl vfile: Global ACLs disabled
> doveadm(@): Debug: Namespace users: type=shared,
> prefix=user/%n/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no
> location=mdbox:%h/mdbox:INDEXPVT=~/shared/%u
> doveadm(@): Debug: shared: root=/run/dovecot, index=,
> indexpvt=, control=, inbox=, alt=
> doveadm(@): Debug: acl: initializing backend with data: vfile
> doveadm(@): Debug: acl: acl username = @
> doveadm(@): Debug: acl: owner = 0
> doveadm(@): Debug: acl vfile: Global ACLs disabled
> doveadm(@): Debug: Namespace subs: type=private, prefix=,
> sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes
> location=mdbox:~/mdbox
> doveadm(@): Debug: fs:
> root=/var/dovecot/imap///mdbox, index=, indexpvt=,
> control=, inbox=, alt=
> doveadm(@): Debug: acl: initializing backend with data: vfile
> doveadm(@): Debug: acl: acl username = @
> doveadm(@): Debug: acl: owner = 0
> doveadm(@): Debug: acl vfile: Global ACLs disabled
> doveadm(@): Debug: quota: quota_over_flag check:
> quota_over_script unset - skipping
> doveadm(@): user-lookup(@)Debug: Added
> userdb setting: plugin/master_user=@
> doveadm(@): user-lookup(@)Debug: Added
> userdb setting: 

Returning error response to the client from passdb (with dict protocol)

[Nagy, Attila]

Hi,

When using a passdb with (UNIX socket) dict protocol, I can return a 
JSON with fields. When I want to deny the user, I can send a JSON like this:


{"fail":true}

which yields (on POP) a friendly -ERR [AUTH] Authentication failed.

Is it possible to return the response text itself? I want to tell the 
user why the authentication has failed.

concering dovecot logging

[Rajesh M]

hi

had a question concerning dovecot lda logging.

deliver_log_format = From:<%f>-<%e> :: Subject:<%s> :: Status:<%$> ::
MsgID:<%m> :: Size<%p> :: vSize<%w>

am getting %e ie envelope sender as MAILER-DAEMON in the logs

Sep 11 08:35:50 lda(n...@xxx.com): Info: sieve: 
From:- :: Subject: :: 
Status:> :: 
MsgID:<> 
:: Size<12497> :: vSize<12687>

what could be the reason ?

thanks,
rajesh

Re: Fail2ban 'Password mismatch' regex

[Marcus Rueckert]

maybe look at weakforced?

   darix

-- 
   openSUSE - SUSE Linux is my linux
   openSUSE is good for you
   www.opensuse.org

Re: Problems to configure IMAP Quota

[Jacques Belin]

Le dimanche 10 septembre 2017 14:35:16,
Alex JOST  a écrit:

> > 1) It seems that when we use the sql dict, a map contaning the patern
> > "priv/quota/limit/messages" is mandatory. Not specifying it  in
> > dovecot-dict-sql.conf.ext issues an error.  As we are interested only by
> > the storage size and don't want to process the number of messages, how
> > to get rid of it ?
> 
> Set it to unlimited?

It is not the point.
I don't want to have to manage the number of messages, only the storage
size.

If the IMAP QUOTA RFCs, the number of messages is given only as an
example of resources that can be accessible by IMAP QUOTA. 
So why dovecot made it mandatory ?

> > 2) We store the storage value in kilobytes.
> > But the plugin seems to only accept storage in bytes, Is there any way
> > to let know to dovecot that the value stored in the database is in
> > kilobytes ? (of course, I tried to put "size/1024" in the value_field
> > field of the map. It works, but generate a lot of error lines in the
> > dovecot log as it create a syntaxly incorrect MySQL uodate request...)
> 
> Using 'storage' instead of 'bytes' should provide limits as kilobytes.

It is what I have already done, from the begnining.

But that word deals about how the limit is defined in the quota rule
definition, not about the values stored in the database. 
In the database we use, the sizes are stored in kilobytes, but dovecot
wants bytes (strange choice BTW, as when we deal about disk storage we
talk only in terms of sectors size, clusters, etc... which are now
always defined as multiples of kilobytes...)...


Jacques.
-- 
The last man connected to the Interet was browsing some old WebSites.
"You have new mail" appeared on the screen...
--- adapted from a short Fredric Brown's story

Re: Is it possible to disable pipelining in imapc?

[Aki Tuomi]

On 11.09.2017 16:14, Nagy, Attila wrote:
> On 09/11/2017 03:01 PM, Aki Tuomi wrote:
>> On 11.09.2017 15:56, Nagy, Attila wrote:
>>> On 09/11/2017 12:12 PM, Aki Tuomi wrote:
 Is there some reason you can't use normal proxy instead of imap
 backend?
 That is,return proxy, host=imap_backend, port=1430? There seems to
 be no
 pipeline setting currently for imapc in v2.2.

>>> Yes, because it's a dumb IMAP server, which doesn't implement a lot of
>>> things, like SEARCH, FETCH BODYSTRUCTURE and similar.
>>> Dovecot is used as a smart proxy, which makes it possible to use it as
>>> a fully featured IMAP server.
>> Does the dumb server advertize pipelining?
>>
> Exactly how does an IMAP server advertise pipelining other than it
> says it's an IMAP server? :)
> There is no such capability, extension etc. It's in RFC3501, which
> defines IMAP v4r1.
>
> Dovecot has a proxy_nopipelining setting for its IMAP proxy, I just
> want to use that (ie: no pipelining) with imapc too. Whether it's
> dynamically configurable, even with the existing proxy_nopipelining,
> or with an imapc_features flag, it doesn't matter for me.
>
> Of course, setting this per user would be nicer.

There currently is no such feature. Guess such feature could be added at
some point.

Aki

Re: Is it possible to disable pipelining in imapc?

[Nagy, Attila]

On 09/11/2017 03:01 PM, Aki Tuomi wrote:

On 11.09.2017 15:56, Nagy, Attila wrote:

On 09/11/2017 12:12 PM, Aki Tuomi wrote:

Is there some reason you can't use normal proxy instead of imap backend?
That is,return proxy, host=imap_backend, port=1430? There seems to be no
pipeline setting currently for imapc in v2.2.


Yes, because it's a dumb IMAP server, which doesn't implement a lot of
things, like SEARCH, FETCH BODYSTRUCTURE and similar.
Dovecot is used as a smart proxy, which makes it possible to use it as
a fully featured IMAP server.

Does the dumb server advertize pipelining?

Exactly how does an IMAP server advertise pipelining other than it says 
it's an IMAP server? :)
There is no such capability, extension etc. It's in RFC3501, which 
defines IMAP v4r1.


Dovecot has a proxy_nopipelining setting for its IMAP proxy, I just want 
to use that (ie: no pipelining) with imapc too. Whether it's dynamically 
configurable, even with the existing proxy_nopipelining, or with an 
imapc_features flag, it doesn't matter for me.


Of course, setting this per user would be nicer.

Re: Is it possible to disable pipelining in imapc?

[Aki Tuomi]

On 11.09.2017 15:56, Nagy, Attila wrote:
> On 09/11/2017 12:12 PM, Aki Tuomi wrote:
>> Is there some reason you can't use normal proxy instead of imap backend?
>> That is,return proxy, host=imap_backend, port=1430? There seems to be no
>> pipeline setting currently for imapc in v2.2.
>>
> Yes, because it's a dumb IMAP server, which doesn't implement a lot of
> things, like SEARCH, FETCH BODYSTRUCTURE and similar.
> Dovecot is used as a smart proxy, which makes it possible to use it as
> a fully featured IMAP server.

Does the dumb server advertize pipelining?

Aki

Re: Is it possible to disable pipelining in imapc?

[Nagy, Attila]

On 09/11/2017 12:12 PM, Aki Tuomi wrote:

Is there some reason you can't use normal proxy instead of imap backend?
That is,return proxy, host=imap_backend, port=1430? There seems to be no
pipeline setting currently for imapc in v2.2.

Yes, because it's a dumb IMAP server, which doesn't implement a lot of 
things, like SEARCH, FETCH BODYSTRUCTURE and similar.
Dovecot is used as a smart proxy, which makes it possible to use it as a 
fully featured IMAP server.

Re: Per-user quota (passwd)

[Aki Tuomi]

Hi!
Just so you know, autocreate/autosubscribe is deprecated, you should use

namespace {
  mailbox INBOX {
   auto = subscribe
 }
}

can you run

doveadm user administra...@email.dom

to verify that mail_home gets set correctly?

Aki

On 11.09.2017 14:48, Evgeniy Korneechev wrote:
> Hi.
> But there is one problem...
>
> protocol lda {
>   mail_plugins = " quota autocreate  sieve quota"
>   plugin {
> quota = dict:user::file:/var/vmail/glu_vrem/%u/.quotausage
>   }
> }
> protocol imap {
>   mail_plugins = " quota autocreate autocreate imap_quota"
>   plugin {
> autocreate = INBOX
> autocreate2 = Sent
> autocreate3 = Trash
> autocreate4 = Drafts
> autocreate5 = Junk
> autosubscribe = INBOX
> autosubscribe2 = Sent
> autosubscribe3 = Trash
> autosubscribe4 = Drafts
> autosubscribe5 = Junk
> quota = dict:user::file:/var/vmail/glu_vrem/%u/.quotausage
>   }
> }
>
>> Users with non-standard quota in passwd-file:
 administrator:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
 userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
 administra...@email.dom:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
 userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
>> Others from passwd.
>
>
> Files ".quotausage" are created in different directories:
> /var/vmail/glu_vrem/administrator\DOM/.quotausage
> /var/vmail/glu_vrem/administra...@email.dom/.quotausage
> And must in the folder "/var/vmail/glu_vrem/administrator"
>
> How fix it?
>
>
> - Исходное сообщение -
>> От: "Evgeniy Korneechev" 
>> Кому: "Aki Tuomi" 
>> Копия: "dovecot" 
>> Отправленные: Пятница, 30 Июнь 2017 г 10:39:16
>> Тема: Re: Per-user quota (passwd)
>> Our solution:
>> userdb {
>>  args = /etc/imap.passwd
>>  driver = passwd-file
>>  override_fields = home=/var/vmail/glu_vrem/%u
>> }
>> userdb {
>>  driver = passwd
>>  override_fields = home=/var/vmail/glu_vrem/%u
>> }
>>
>> Users with non-standard quota in passwd-file:
 administrator:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
 userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
 administra...@email.dom:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
 userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
>> Others from passwd.
>>
>> Its works! Thanks.
>>
>>
>> - Исходное сообщение -
>>> От: "Aki Tuomi" 
>>> Кому: "dovecot" 
>>> Отправленные: Четверг, 29 Июнь 2017 г 14:40:44
>>> Тема: Re: Per-user quota (passwd)
>>> Oh you have multiple db's, I missed that.
>>>
>>> Remove auth_username_format, and instead
>>>
>>> userdb {
>>>  args = /etc/imap.passwd username_format=%Ln
>>>  driver = passwd-file
>>>  override_fields = home=/var/vmail/glu_vrem/%u
>>> }
>>>
>>> On 29.06.2017 14:35, Evgeniy Korneechev wrote:
 with auth_username_format = %Ln:

 Jun 28 14:43:41 auth: Debug: master in: USER1iivanov...@example.com
 service=lda
 Jun 28 14:43:41 auth-worker(18369): Debug: passwd(iivanov.ia): lookup
 Jun 28 14:43:41 auth-worker(18369): Info: passwd(iivanov.ia): unknown user 
  -
 trying the next userdb
 Jun 28 14:43:41 auth-worker(18369): Debug: sql(iivanov.ia): SELECT email as
 user, maildir as home, CONCAT('maildir:', maildir, '/Maildir') as mail, 
 uid,
 gid, \
  CONCAT('*:storage=', quota, 'B') AS quota_rule, CONCAT(maildir, 
 '/.sieve') as
  sieve FROM mail_user WHERE (login = 'iivanov.ia' OR email = 
 'iivanov.ia')
 Jun 28 14:43:41 auth-worker(18369): Info: sql(iivanov.ia): unknown user

 Maybe %Lu?

 - Исходное сообщение -
> От: "Aki Tuomi" 
> Кому: "dovecot" 
> Отправленные: Четверг, 29 Июнь 2017 г 14:10:07
> Тема: Re: Per-user quota (passwd)
> Yes.
>
> Aki
>
> On 29.06.2017 14:07, Evgeniy Korneechev wrote:
>> And if such a user:
>> email (from AD) = ivanov...@example.com
>> login (from AD), %n = iivanov
>> %d = DOM
>> %u = iivanov@DOM
>>
>> ?
>>
>> - Исходное сообщение -
>>> От: "Aki Tuomi" 
>>> Кому: "dovecot" 
>>> Отправленные: Четверг, 29 Июнь 2017 г 13:59:05
>>> Тема: Re: Per-user quota (passwd)
>>> or use, as I indicated before, auth_username_format = %Ln
>>>
>>> Aki
>>>
>>>
>>> On 29.06.2017 13:58, Evgeniy Korneechev wrote:
 Hi!
 workaround:
 administrator:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
 

Re: Per-user quota (passwd)

[Evgeniy Korneechev]

Hi.
But there is one problem...

protocol lda {
  mail_plugins = " quota autocreate  sieve quota"
  plugin {
quota = dict:user::file:/var/vmail/glu_vrem/%u/.quotausage
  }
}
protocol imap {
  mail_plugins = " quota autocreate autocreate imap_quota"
  plugin {
autocreate = INBOX
autocreate2 = Sent
autocreate3 = Trash
autocreate4 = Drafts
autocreate5 = Junk
autosubscribe = INBOX
autosubscribe2 = Sent
autosubscribe3 = Trash
autosubscribe4 = Drafts
autosubscribe5 = Junk
quota = dict:user::file:/var/vmail/glu_vrem/%u/.quotausage
  }
}

> Users with non-standard quota in passwd-file:
>>> administrator:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
>>> userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
>>> administra...@email.dom:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
>>> userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
> 
> Others from passwd.



Files ".quotausage" are created in different directories:
/var/vmail/glu_vrem/administrator\DOM/.quotausage
/var/vmail/glu_vrem/administra...@email.dom/.quotausage
And must in the folder "/var/vmail/glu_vrem/administrator"

How fix it?


- Исходное сообщение -
> От: "Evgeniy Korneechev" 
> Кому: "Aki Tuomi" 
> Копия: "dovecot" 
> Отправленные: Пятница, 30 Июнь 2017 г 10:39:16
> Тема: Re: Per-user quota (passwd)

> Our solution:
> userdb {
>  args = /etc/imap.passwd
>  driver = passwd-file
>  override_fields = home=/var/vmail/glu_vrem/%u
> }
> userdb {
>  driver = passwd
>  override_fields = home=/var/vmail/glu_vrem/%u
> }
> 
> Users with non-standard quota in passwd-file:
>>> administrator:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
>>> userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
>>> administra...@email.dom:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
>>> userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
> 
> Others from passwd.
> 
> Its works! Thanks.
> 
> 
> - Исходное сообщение -
>> От: "Aki Tuomi" 
>> Кому: "dovecot" 
>> Отправленные: Четверг, 29 Июнь 2017 г 14:40:44
>> Тема: Re: Per-user quota (passwd)
> 
>> Oh you have multiple db's, I missed that.
>> 
>> Remove auth_username_format, and instead
>> 
>> userdb {
>>  args = /etc/imap.passwd username_format=%Ln
>>  driver = passwd-file
>>  override_fields = home=/var/vmail/glu_vrem/%u
>> }
>> 
>> On 29.06.2017 14:35, Evgeniy Korneechev wrote:
>>> with auth_username_format = %Ln:
>>>
>>> Jun 28 14:43:41 auth: Debug: master in: USER1iivanov...@example.com
>>> service=lda
>>> Jun 28 14:43:41 auth-worker(18369): Debug: passwd(iivanov.ia): lookup
>>> Jun 28 14:43:41 auth-worker(18369): Info: passwd(iivanov.ia): unknown user  
>>> -
>>> trying the next userdb
>>> Jun 28 14:43:41 auth-worker(18369): Debug: sql(iivanov.ia): SELECT email as
>>> user, maildir as home, CONCAT('maildir:', maildir, '/Maildir') as mail, uid,
>>> gid, \
>>>  CONCAT('*:storage=', quota, 'B') AS quota_rule, CONCAT(maildir, 
>>> '/.sieve') as
>>>  sieve FROM mail_user WHERE (login = 'iivanov.ia' OR email = 
>>> 'iivanov.ia')
>>> Jun 28 14:43:41 auth-worker(18369): Info: sql(iivanov.ia): unknown user
>>>
>>> Maybe %Lu?
>>>
>>> - Исходное сообщение -
 От: "Aki Tuomi" 
 Кому: "dovecot" 
 Отправленные: Четверг, 29 Июнь 2017 г 14:10:07
 Тема: Re: Per-user quota (passwd)
 Yes.

 Aki

 On 29.06.2017 14:07, Evgeniy Korneechev wrote:
> And if such a user:
> email (from AD) = ivanov...@example.com
> login (from AD), %n = iivanov
> %d = DOM
> %u = iivanov@DOM
>
> ?
>
> - Исходное сообщение -
>> От: "Aki Tuomi" 
>> Кому: "dovecot" 
>> Отправленные: Четверг, 29 Июнь 2017 г 13:59:05
>> Тема: Re: Per-user quota (passwd)
>> or use, as I indicated before, auth_username_format = %Ln
>>
>> Aki
>>
>>
>> On 29.06.2017 13:58, Evgeniy Korneechev wrote:
>>> Hi!
>>> workaround:
>>> administrator:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
>>> userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
>>> administra...@email.dom:*:95400500:95400513:Administrator:/home/DOM/administrator:/bin/bash::userdb_quota_rule=*:bytes=10G
>>> userdb_mail=maildir:/var/vmail/glu_vrem/administrator/Maildir
>>>
>>> But this is not a solution for 1000 users ...
>>> Maybe is there extra field "userdb_mail=administra...@email.dom" for 
>>> email to
>>> passwd-file?
>>>
>>> - Исходное 

Re: pop3-login core dump when using TLSSTART on version dovecot-2.2.32 (INTERNAL)

[Aki Tuomi]

Can you outline the exact steps you perform to get this?

Aki

On 11.09.2017 13:42, arvid.ei...@telenor.com wrote:
> Hi,
>
> I switched back to 2.2.27 with the same config that I am using for 2.2.32 and 
> it work fine.  
>
> Sep 11 11:49:37 imap-login: Info: Login: user=, method=PLAIN, 
> rip=88.89.118.45, lip=148.123.160.116, mpid=18709, TLS, 
> session=
> Sep 11 11:49:40 imap(mailuser) Session-ID v7o22OZYrsdYWXYt RemoteIP 
> 88.89.118.45  Maildir /var/nextmail/nfs2.flex14/49/79/841 Info: Logged out 
> in=4518 out=273720 deleted 0 expunged 0 trashed 0
> Sep 11 11:49:40 imap-login: Debug: SSL alert: close notify [88.89.118.45]
>
> How could I proceed?  Any clue?  It is quite annoying to see this entry in 
> the log for each session.
>
> Arvid
>
>
>
>
> -Original Message-
> From: Aki Tuomi [mailto:aki.tu...@dovecot.fi] 
> Sent: 11. september 2017 09:18
> To: Eikås Arvid; dovecot@dovecot.org
> Subject: Re: pop3-login core dump when using TLSSTART on version 
> dovecot-2.2.32 (INTERNAL)
>
> Hi!
>
> I tried to reproduce this problem with dovecot-2.2.32 and OpenSSL 1.0.1k and 
> was not able to. I enabled -DREF_CHECK on OpenSSL, but to no avail, the 
> process did not crash. Is there something else you've done?
>
> Aki
>
>
> On 11.09.2017 08:07, arvid.ei...@telenor.com wrote:
>> Hi,
>>
>> Here is the gdb output.
>>
>> Arvid
>>
>> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-94.el7 Copyright (C) 2013 
>> Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later 
>> 
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "x86_64-redhat-linux-gnu".
>> For bug reporting instructions, please see:
>> ...
>> Reading symbols from 
>> /local/misc/mail/dovecot-32/libexec/dovecot/pop3-login...done.
>> [New LWP 15894]
>> Core was generated by `dovecot-test/pop3-login'.
>> Program terminated with signal 6, Aborted.
>> #0  0x7ff0bd9cf1d7 in raise () from /lib64/libc.so.6 Missing 
>> separate debuginfos, use: debuginfo-install 
>> glibc-2.17-157.el7_3.1.x86_64
>> (gdb) bt full
>> #0  0x7ff0bd9cf1d7 in raise () from /lib64/libc.so.6 No symbol 
>> table info available.
>> #1  0x7ff0bd9d08c8 in abort () from /lib64/libc.so.6 No symbol 
>> table info available.
>> #2  0x7ff0bd3c0f2f in engine_unlocked_finish (e=0x1c51c60, 
>> unlock_for_handlers=1) at eng_init.c:115
>> to_return = 1
>> #3  0x7ff0bd3c1064 in ENGINE_finish (e=0x1c51c60) at eng_init.c:150
>> to_return = 1
>> #4  0x7ff0be0f9300 in ssl_proxy_deinit () from 
>> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
>> No symbol table info available.
>> #5  0x7ff0be0f4472 in main_deinit () from 
>> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
>> No symbol table info available.
>> #6  0x7ff0be0f479f in login_binary_run () from 
>> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
>> No symbol table info available.
>> #7  0x004032da in main (argc=1, argv=0x7ffe3059f3f8) at 
>> client.c:356 No locals.
>>
>>
>>
>> -Original Message-
>> From: Aki Tuomi [mailto:aki.tu...@dovecot.fi]
>> Sent: 8. september 2017 14:08
>> To: Eikås Arvid; dovecot@dovecot.org
>> Subject: Re: pop3-login core dump when using TLSSTART on version 
>> dovecot-2.2.32 (OPEN)
>>
>> I assume you mean STARTTLS. Can you provide gdb /path/to/bin /path/to/core 
>> and provide output of bt full?
>>
>> Aki
>>
>>
>> On 08.09.2017 15:01, arvid.ei...@telenor.com wrote:
>>> Hi,
>>>
>>> Pop3-login are CORE-dumping when I log on with TLSSTART, I believe the same 
>>> will happen with imap-logon to, but I have not tested it yet.
>>> The TLS session is coming up and it works fine until I log off, then it's 
>>> core dump.  Open sslvesrion is   openssl-1.0.2k.
>>> We ran dovecot-2.2.27 before we upgraded to dovecote-2.2.32, and that 
>>> seems to work fine. (not core dumping)
>>>
>>>
>>> Arvid
>>>
>>>
>>> LOG
>>> Sep 05 14:27:34 pop3-login: Debug: SSL: elliptic curve secp384r1 will 
>>> be used for ECDH and ECDHE key exchanges Sep 05 14:30:30 pop3-login:
>>> Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE 
>>> key exchanges Sep 05 14:30:30 pop3-login: Debug: SSL: elliptic curve
>>> secp384r1 will be used for ECDH and ECDHE key exchanges Sep 05
>>> 14:30:42 pop3-login: Debug: SSL: elliptic curve secp384r1 will be 
>>> used for ECDH and ECDHE key exchanges Sep 05 14:30:42 pop3-login: Debug:
>>> SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key 
>>> exchanges Sep 05 14:30:50 pop3-login: Info: Login: user=, 
>>> method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=18361, secured, 
>>> session=<65m8ZXBYtpN/AAAB> Sep 05 14:30:50 pop3-login: Error:
>>> ENGINE_finish, bad functional reference count Sep 05 14:30:50
>>> pop3-login: Fatal: master: 

RE: pop3-login core dump when using TLSSTART on version dovecot-2.2.32 (INTERNAL)

[Arvid.Eikas]

Hi,

I switched back to 2.2.27 with the same config that I am using for 2.2.32 and 
it work fine.  

Sep 11 11:49:37 imap-login: Info: Login: user=, method=PLAIN, 
rip=88.89.118.45, lip=148.123.160.116, mpid=18709, TLS, 
session=
Sep 11 11:49:40 imap(mailuser) Session-ID v7o22OZYrsdYWXYt RemoteIP 
88.89.118.45  Maildir /var/nextmail/nfs2.flex14/49/79/841 Info: Logged out 
in=4518 out=273720 deleted 0 expunged 0 trashed 0
Sep 11 11:49:40 imap-login: Debug: SSL alert: close notify [88.89.118.45]

How could I proceed?  Any clue?  It is quite annoying to see this entry in the 
log for each session.

Arvid




-Original Message-
From: Aki Tuomi [mailto:aki.tu...@dovecot.fi] 
Sent: 11. september 2017 09:18
To: Eikås Arvid; dovecot@dovecot.org
Subject: Re: pop3-login core dump when using TLSSTART on version dovecot-2.2.32 
(INTERNAL)

Hi!

I tried to reproduce this problem with dovecot-2.2.32 and OpenSSL 1.0.1k and 
was not able to. I enabled -DREF_CHECK on OpenSSL, but to no avail, the process 
did not crash. Is there something else you've done?

Aki


On 11.09.2017 08:07, arvid.ei...@telenor.com wrote:
> Hi,
>
> Here is the gdb output.
>
> Arvid
>
> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-94.el7 Copyright (C) 2013 
> Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> ...
> Reading symbols from 
> /local/misc/mail/dovecot-32/libexec/dovecot/pop3-login...done.
> [New LWP 15894]
> Core was generated by `dovecot-test/pop3-login'.
> Program terminated with signal 6, Aborted.
> #0  0x7ff0bd9cf1d7 in raise () from /lib64/libc.so.6 Missing 
> separate debuginfos, use: debuginfo-install 
> glibc-2.17-157.el7_3.1.x86_64
> (gdb) bt full
> #0  0x7ff0bd9cf1d7 in raise () from /lib64/libc.so.6 No symbol 
> table info available.
> #1  0x7ff0bd9d08c8 in abort () from /lib64/libc.so.6 No symbol 
> table info available.
> #2  0x7ff0bd3c0f2f in engine_unlocked_finish (e=0x1c51c60, 
> unlock_for_handlers=1) at eng_init.c:115
> to_return = 1
> #3  0x7ff0bd3c1064 in ENGINE_finish (e=0x1c51c60) at eng_init.c:150
> to_return = 1
> #4  0x7ff0be0f9300 in ssl_proxy_deinit () from 
> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
> No symbol table info available.
> #5  0x7ff0be0f4472 in main_deinit () from 
> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
> No symbol table info available.
> #6  0x7ff0be0f479f in login_binary_run () from 
> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
> No symbol table info available.
> #7  0x004032da in main (argc=1, argv=0x7ffe3059f3f8) at 
> client.c:356 No locals.
>
>
>
> -Original Message-
> From: Aki Tuomi [mailto:aki.tu...@dovecot.fi]
> Sent: 8. september 2017 14:08
> To: Eikås Arvid; dovecot@dovecot.org
> Subject: Re: pop3-login core dump when using TLSSTART on version 
> dovecot-2.2.32 (OPEN)
>
> I assume you mean STARTTLS. Can you provide gdb /path/to/bin /path/to/core 
> and provide output of bt full?
>
> Aki
>
>
> On 08.09.2017 15:01, arvid.ei...@telenor.com wrote:
>> Hi,
>>
>> Pop3-login are CORE-dumping when I log on with TLSSTART, I believe the same 
>> will happen with imap-logon to, but I have not tested it yet.
>> The TLS session is coming up and it works fine until I log off, then it's 
>> core dump.  Open sslvesrion is   openssl-1.0.2k.
>> We ran dovecot-2.2.27 before we upgraded to dovecote-2.2.32, and that 
>> seems to work fine. (not core dumping)
>>
>>
>> Arvid
>>
>>
>> LOG
>> Sep 05 14:27:34 pop3-login: Debug: SSL: elliptic curve secp384r1 will 
>> be used for ECDH and ECDHE key exchanges Sep 05 14:30:30 pop3-login:
>> Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE 
>> key exchanges Sep 05 14:30:30 pop3-login: Debug: SSL: elliptic curve
>> secp384r1 will be used for ECDH and ECDHE key exchanges Sep 05
>> 14:30:42 pop3-login: Debug: SSL: elliptic curve secp384r1 will be 
>> used for ECDH and ECDHE key exchanges Sep 05 14:30:42 pop3-login: Debug:
>> SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key 
>> exchanges Sep 05 14:30:50 pop3-login: Info: Login: user=, 
>> method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=18361, secured, 
>> session=<65m8ZXBYtpN/AAAB> Sep 05 14:30:50 pop3-login: Error:
>> ENGINE_finish, bad functional reference count Sep 05 14:30:50
>> pop3-login: Fatal: master: service(pop3-login): child 18359 killed 
>> with signal 6 (core dumped)
>>
>>
>>
>>
>>
>>
>> From  ./crypto/engine/eng_init.c
>>
>> .
>> int engine_unlocked_finish(ENGINE *e, int unlock_for_handlers) {
>> int to_return = 1;
>>
>> /*
>>  * Reduce the 

Re: Is it possible to disable pipelining in imapc?

[Aki Tuomi]

On 11.09.2017 13:09, Nagy, Attila wrote:
> On 09/11/2017 11:14 AM, Aki Tuomi wrote:
>>
>> On 11.09.2017 11:59, Nagy, Attila wrote:
>>> On 09/11/2017 10:42 AM, Sami Ketola wrote:
> On 11 Sep 2017, at 11.24, Nagy, Attila  wrote:
> I use dovecot with a broken IMAP server (which doesn't properly
> implement command pipelining amongst others) as an imapc backend.
> Dovecot issues the above command sequence (SELECT and UID FETCH
> pipelined), which doesn't work with this server.
>
> Therefore I'm requesting an imapc_features setting,
> IMAPC_FEATURE_NO_PIPELINING, which disables PIPELINING in dovecot
> imapc. Similarly to other workarounds, like
> IMAPC_FEATURE_FETCH_MSN_WORKAROUNDS,
> IMAPC_FEATURE_FETCH_FIX_BROKEN_MAILS et al.
 Dovecot IMAPC should not pipeline unless remote advertises pipelining
 support in CAPABILITY response.
 Are you saying that you have imap server which advertses support for
 it but then does not support it
 after all?

>>> It's a capability in POP3 and an extension in (E)SMTP.
>>> I don't know about pipelining capability in IMAP (BTW, the server
>>> doesn't advertise such capability, in fact, it doesn't support any
>>> capabilities), it's standard, plain old IMAP RFC:
>>> https://tools.ietf.org/html/rfc3501#section-5.5
>>>
>>> The whole topic is about a bad IMAP server, which doesn't properly
>>> implement pipelining (hence IMAP RFC), like others, for which dovecot
>>> implements a workaround in imapc.
>> But are you using it like with doveadm? or proxying connection to? How
>> are you invoking imapc?
>>
> Dovecot acts as a frontend to this broken server and I redirect users
> with passdb/userdb via dict protocol, returning entries like:
> js = {
>   'password': '',
>   'nopassword': '',
>   'host':'imap_backend',
>   'port':1430,
>   'driver':'imap',
>   'proxy_nopipelining':'y',
>   'userdb_user':qdata['Auth-User'],
>   'userdb_imapc_user':qdata['Auth-User'],
>   'userdb_imapc_password':qdata['Auth-Pass'],
>   'destuser':qdata['Auth-User'],
>   }
>
> (and this is where I tried proxy_nopipelining as you can see, but of
> course it's irrelevant)

Is there some reason you can't use normal proxy instead of imap backend?
That is,return proxy, host=imap_backend, port=1430? There seems to be no
pipeline setting currently for imapc in v2.2.

Aki

Re: Is it possible to disable pipelining in imapc?

[Nagy, Attila]

On 09/11/2017 11:14 AM, Aki Tuomi wrote:


On 11.09.2017 11:59, Nagy, Attila wrote:

On 09/11/2017 10:42 AM, Sami Ketola wrote:

On 11 Sep 2017, at 11.24, Nagy, Attila  wrote:
I use dovecot with a broken IMAP server (which doesn't properly
implement command pipelining amongst others) as an imapc backend.
Dovecot issues the above command sequence (SELECT and UID FETCH
pipelined), which doesn't work with this server.

Therefore I'm requesting an imapc_features setting,
IMAPC_FEATURE_NO_PIPELINING, which disables PIPELINING in dovecot
imapc. Similarly to other workarounds, like
IMAPC_FEATURE_FETCH_MSN_WORKAROUNDS,
IMAPC_FEATURE_FETCH_FIX_BROKEN_MAILS et al.

Dovecot IMAPC should not pipeline unless remote advertises pipelining
support in CAPABILITY response.
Are you saying that you have imap server which advertses support for
it but then does not support it
after all?


It's a capability in POP3 and an extension in (E)SMTP.
I don't know about pipelining capability in IMAP (BTW, the server
doesn't advertise such capability, in fact, it doesn't support any
capabilities), it's standard, plain old IMAP RFC:
https://tools.ietf.org/html/rfc3501#section-5.5

The whole topic is about a bad IMAP server, which doesn't properly
implement pipelining (hence IMAP RFC), like others, for which dovecot
implements a workaround in imapc.

But are you using it like with doveadm? or proxying connection to? How
are you invoking imapc?

Dovecot acts as a frontend to this broken server and I redirect users 
with passdb/userdb via dict protocol, returning entries like:

js = {
  'password': '',
  'nopassword': '',
  'host':'imap_backend',
  'port':1430,
  'driver':'imap',
  'proxy_nopipelining':'y',
  'userdb_user':qdata['Auth-User'],
  'userdb_imapc_user':qdata['Auth-User'],
  'userdb_imapc_password':qdata['Auth-Pass'],
  'destuser':qdata['Auth-User'],
  }

(and this is where I tried proxy_nopipelining as you can see, but of 
course it's irrelevant)

Re: doveadm expunge returns error code 68

[Aki Tuomi]

On 08.09.2017 15:38, Florian Lohoff wrote:
> Hi Aki,
>
> On Wed, Sep 06, 2017 at 11:26:30AM +0300, Aki Tuomi wrote:
>> On 05.09.2017 14:39, Florian Lohoff wrote:
>>> Hi,
>>> i am running basically this from cron:
>>>
>>> /usr/bin/doveadm -v expunge -u * mailbox INBOX.Spam SENTBEFORE 90d
>>> /usr/bin/doveadm -v expunge -u * mailbox INBOX.Trash SENTBEFORE 90d
>>> /usr/bin/doveadm -v expunge -u * mailbox INBOX.Junk SENTBEFORE 90d
>>>
>>> Now i am experiencing that the first line e.g. expunging INBOX.Spam
>>> returns the error code 68. No syslog/Output whatsoever. I tried
>>> running with -vvv -D which gives me a lot of output (for all mailboxes)
>>> but still i am unable to pinpoint the problem. The later 2 commands
>>> return "0" as expected.
>>>
>>> Where does the error return code come from and what does 68 mean?
>> You probably should use -A instead of -u *.
> My problem is not that it does not process all users. The problem is
> that i dont get ANY error message except the exit code. No logging
> at all. I tried to find any define in the source-code with 68 and failed
> to find one. So i am completely clueless where the problem is.
>
> And -A returns the very same exit code - on Jessie and Stretch.
>
> Flo

Hi!

Can you try doveadm -Dv?

Aki

Re: What INTERNALDATE does dovecot with mbox storage set on a COPY'd message?

[Aki Tuomi]

On 09.09.2017 12:33, Steinar Bang wrote:
> When a message is copied to a folder on dovecot with mbox storage, is
> the mtime of the saved mbox file set to the time of the save?  
>
> Or is the mtime set to the Date: field of the source message that is
> saved?
>
> If there is a difference in the behaviour, do someone know the dovecot
> version number where the change happened?
>
> The reason I'm asking is a problem reported on the Gnus imap client in
> August 2016, where messaged moved by Gnus showed up with the wrong order
> and/or wrong date in other imap clients.
>
> (The reason for the different message ordering was that the other clients
> use the INTERNALDATE and that was changed to the time of message move
> when Gnus was used. Gnus use the Date of the source message)
>
> Thanks!
>
>
> - Steinar (Gnus user)

Hi!

Internaldate is picked from the separating 'From' line in mbox file.

"From u...@example.org Thu Oct 20 18:44:06 2016"

Aki

Re: Is it possible to disable pipelining in imapc?

[Aki Tuomi]

On 11.09.2017 11:59, Nagy, Attila wrote:
> On 09/11/2017 10:42 AM, Sami Ketola wrote:
>>> On 11 Sep 2017, at 11.24, Nagy, Attila  wrote:
>>> I use dovecot with a broken IMAP server (which doesn't properly
>>> implement command pipelining amongst others) as an imapc backend.
>>> Dovecot issues the above command sequence (SELECT and UID FETCH
>>> pipelined), which doesn't work with this server.
>>>
>>> Therefore I'm requesting an imapc_features setting,
>>> IMAPC_FEATURE_NO_PIPELINING, which disables PIPELINING in dovecot
>>> imapc. Similarly to other workarounds, like
>>> IMAPC_FEATURE_FETCH_MSN_WORKAROUNDS,
>>> IMAPC_FEATURE_FETCH_FIX_BROKEN_MAILS et al.
>>
>> Dovecot IMAPC should not pipeline unless remote advertises pipelining
>> support in CAPABILITY response.
>> Are you saying that you have imap server which advertses support for
>> it but then does not support it
>> after all?
>>
> It's a capability in POP3 and an extension in (E)SMTP.
> I don't know about pipelining capability in IMAP (BTW, the server
> doesn't advertise such capability, in fact, it doesn't support any
> capabilities), it's standard, plain old IMAP RFC:
> https://tools.ietf.org/html/rfc3501#section-5.5
>
> The whole topic is about a bad IMAP server, which doesn't properly
> implement pipelining (hence IMAP RFC), like others, for which dovecot
> implements a workaround in imapc.

But are you using it like with doveadm? or proxying connection to? How
are you invoking imapc?

Aki

Re: Is it possible to disable pipelining in imapc?

[Nagy, Attila]

On 09/11/2017 10:42 AM, Sami Ketola wrote:

On 11 Sep 2017, at 11.24, Nagy, Attila  wrote:
I use dovecot with a broken IMAP server (which doesn't properly implement 
command pipelining amongst others) as an imapc backend.
Dovecot issues the above command sequence (SELECT and UID FETCH pipelined), 
which doesn't work with this server.

Therefore I'm requesting an imapc_features setting, 
IMAPC_FEATURE_NO_PIPELINING, which disables PIPELINING in dovecot imapc. 
Similarly to other workarounds, like IMAPC_FEATURE_FETCH_MSN_WORKAROUNDS, 
IMAPC_FEATURE_FETCH_FIX_BROKEN_MAILS et al.


Dovecot IMAPC should not pipeline unless remote advertises pipelining support 
in CAPABILITY response.
Are you saying that you have imap server which advertses support for it but 
then does not support it
after all?


It's a capability in POP3 and an extension in (E)SMTP.
I don't know about pipelining capability in IMAP (BTW, the server 
doesn't advertise such capability, in fact, it doesn't support any 
capabilities), it's standard, plain old IMAP RFC:

https://tools.ietf.org/html/rfc3501#section-5.5

The whole topic is about a bad IMAP server, which doesn't properly 
implement pipelining (hence IMAP RFC), like others, for which dovecot 
implements a workaround in imapc.

Re: Is it possible to disable pipelining in imapc?

[Sami Ketola]

> On 11 Sep 2017, at 11.24, Nagy, Attila  wrote:
> I use dovecot with a broken IMAP server (which doesn't properly implement 
> command pipelining amongst others) as an imapc backend.
> Dovecot issues the above command sequence (SELECT and UID FETCH pipelined), 
> which doesn't work with this server.
> 
> Therefore I'm requesting an imapc_features setting, 
> IMAPC_FEATURE_NO_PIPELINING, which disables PIPELINING in dovecot imapc. 
> Similarly to other workarounds, like IMAPC_FEATURE_FETCH_MSN_WORKAROUNDS, 
> IMAPC_FEATURE_FETCH_FIX_BROKEN_MAILS et al.


Dovecot IMAPC should not pipeline unless remote advertises pipelining support 
in CAPABILITY response.
Are you saying that you have imap server which advertses support for it but 
then does not support it
after all?

Sami

Re: Is it possible to disable pipelining in imapc?

[Nagy, Attila]

On 09/11/2017 09:17 AM, Aki Tuomi wrote:


On 08.09.2017 15:29, Nagy, Attila wrote:

On 09/08/2017 01:53 PM, Aki Tuomi wrote:

On 08.09.2017 14:50, Nagy, Attila wrote:

Hi,

I've a broken IMAP server, which doesn't support pipelining and fails
on dovecot's attempt to do this ([C] is dovecot's imapc, [S] is the
IMAP server):

[C] 24 LOGIN "user" "pass"
[S] 23 OK
[C] 25 SELECT "INBOX"
[C] 23 UID FETCH 2 (INTERNALDATE)
[S] 25 OK
[S] 23 BAD No mailbox selected

Sadly, fixing the server would be a hard task, turning off pipelining
in dovecot's imapc (which already supports many workarounds for
different servers) could be easy...

... except I can see no place to do this in imapc (only in pop3c and
imap proxy, which I guess doesn't apply here).

If I'm right, may I ask for a knob for such dumb servers?

Thanks,

For proxy, use proxy_nopipelining


Are you sure it works with imapc? I've tried it, and dovecot still
does the above.

So what are you exactly doing?


I use dovecot with a broken IMAP server (which doesn't properly 
implement command pipelining amongst others) as an imapc backend.
Dovecot issues the above command sequence (SELECT and UID FETCH 
pipelined), which doesn't work with this server.


Therefore I'm requesting an imapc_features setting, 
IMAPC_FEATURE_NO_PIPELINING, which disables PIPELINING in dovecot imapc. 
Similarly to other workarounds, like 
IMAPC_FEATURE_FETCH_MSN_WORKAROUNDS, 
IMAPC_FEATURE_FETCH_FIX_BROKEN_MAILS et al.

Re: Dovecot and Letsencrypt certs

[Arkadiusz Miśkiewicz]

On Friday 08 of September 2017, Ralph Seichter wrote:
> On 08.09.2017 16:20, LuKreme wrote:

> > However, it seems like checking the certs is something that dovecot
> > should be doing on its own.
> 
> What is Dovecot supposed to do? Keep track of the certificate expiry
> date? 

That was already discussed but due to other reason. dovecot shouldn't load SSL 
certificates into memory and instead open  & load cert on demand (when client 
connects and requests particular domain via SNI (or default if no SNI)).

Why? Because dovecot *cannot* handle thousands of virtual domains and SSL 
certificates for these. It wastes so much RAM and timeouts on reloads in such 
case. Tested here. [1]

That's why the only sensible solution is to work like exim - load cert from 
disk on demand.

That fixes both problems - ram wasting/timeouts and refreshing certificates.


> -Ralph

1. https://dovecot.org/list/dovecot/2016-October/105855.html

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

Re: Fail2ban 'Password mismatch' regex

[James Brown]

> On 11 Sep 2017, at 5:38 pm, Christian Kivalo  wrote:
> 
>> Many thanks Christian.
>> Added that, but it still doesn’t match:
>> $ fail2ban-regex "Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): 
>> sql(u...@bordo.com.au,::1,L2xqieNYeM4B>): Password 
>> mismatch (given password: 2)"
>> "^%(__prefix_line)sauth: Info: sql\(\S+,,\<\S+\>\): (Password 
>> mismatch|unknown user)( \((SHA1 of given password: [0-9a-f]{5,40}|given 
>> password: \w*)\))?$"
> Your log has "auth-worker(10094): sql" whereas the fail2ban regex has 
> ")sauth: Info: sql\(\". When you change that to ")sauth-worker: sql\(\" does 
> it work then?
> 
> Try to reduce the regex to a working minimum and then add parts back until it 
> breaks…


Thanks Christian.

That didn’t work either:

$ fail2ban-regex "Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): 
sql(u...@bordo.com.au,::1,): Password 
mismatch (given password: 2)" "^%(__prefix_line)sauth-worker: 
sql\(\S+,,\<\S+\>\): (Password mismatch|unknown user)( \((SHA1 of given 
password: [0-9a-f]{5,40}|given password: \w*)\))?$"

Running tests
=

Use   failregex line : ^%(__prefix_line)sauth-worker: sql\(\S+,,\<\...
Use  single line : Sep 11 15:52:49 mail dovecot[54239]: auth-worker(1...


Results
===

Failregex: 0 total


Should there be something after “sauth-worker” for the ‘(10094)’?

Will keep trying deleting stuff till it works.

Thanks,

James.




smime.p7s
Description: S/MIME cryptographic signature

Re: Fail2ban 'Password mismatch' regex

[Christian Kivalo]

Many thanks Christian.

Added that, but it still doesn’t match:

$ fail2ban-regex "Sep 11 15:52:49 mail dovecot[54239]: 
auth-worker(10094): 
sql(u...@bordo.com.au,::1,L2xqieNYeM4B>): Password 
mismatch (given password: 2)"
"^%(__prefix_line)sauth: Info: sql\(\S+,,\<\S+\>\): (Password 
mismatch|unknown user)( \((SHA1 of given password: [0-9a-f]{5,40}|given 
password: \w*)\))?$"
Your log has "auth-worker(10094): sql" whereas the fail2ban regex has 
")sauth: Info: sql\(\". When you change that to ")sauth-worker: sql\(\" 
does it work then?


Try to reduce the regex to a working minimum and then add parts back 
until it breaks...


[...]


Any other suggestions?

Thanks,

James.


--
 Christian Kivalo

Re: Fail2ban 'Password mismatch' regex

[James Brown]

> On 11 Sep 2017, at 5:10 pm, Christian Kivalo  wrote:
> 
> On 2017-09-11 08:57, James Brown wrote:
>> I have turned on 'auth_debug_passwords=yes’ in dovecot.conf.
>> I’m trying to get Fail2ban to detect this log line:
>> Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): 
>> sql(u...@bordo.com.au 
>> ,::1,): Password 
>> mismatch (given password: 2)
>> I’ve added it as the last line of my dovecot filter regex:
>> failregex =
>> ^%(__prefix_line)s(%(__pam_auth)s(\(dovecot:auth\))?:)?\s+authentication 
>> failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* 
>> rhost=(\s+user=\S*)?\s*$
>>^%(__prefix_line)s(pop3|imap)-login: (Info: )?(Aborted 
>> login|Disconnected)(: Inactivity)? \(((auth failed, \d+ attempts)( in \d+ 
>> secs)?|tried to use (disabled|disallo$
>>^%(__prefix_line)s(Info|dovecot: 
>> auth\(default\)|auth-worker\(\d+\)): pam\(\S+,\): pam_authenticate\(\) 
>> failed: (User not known to the underlying authentication$
>>^%(__prefix_line)s(auth|auth-worker\(\d+\)): 
>> (pam|passwd-file)\(\S+,\): unknown user\s*$
>>^%(__prefix_line)s(auth|auth-worker\(\d+\)): Info: 
>> ldap\(\S*,,\S*\): invalid credentials\s*$
>>^%(__prefix_line)sauth: Info: sql\(\S+,\): (Password 
>> mismatch|unknown user)( \((SHA1 of given password: [0-9a-f]{5,40}|given 
>> password: \w*)\))?$
>  ^%(__prefix_line)sauth: Info: sql\(\S+,,\<\S+\>\): 
> (Password mismatch|unknown user)( \((SHA1 of given password: 
> [0-9a-f]{5,40}|given password: \w*)\))?$
>^^^
> You are missing the ID after the host part.
> -- 
> Christian Kivalo
> 
Many thanks Christian.

Added that, but it still doesn’t match:

$ fail2ban-regex "Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): 
sql(u...@bordo.com.au,::1,): Password 
mismatch (given password: 2)" "^%(__prefix_line)sauth: Info: 
sql\(\S+,,\<\S+\>\): (Password mismatch|unknown user)( \((SHA1 of given 
password: [0-9a-f]{5,40}|given password: \w*)\))?$"

Running tests
=

Use   failregex line : ^%(__prefix_line)sauth: Info: sql\(\S+,,\<\S...
Use  single line : Sep 11 15:52:49 mail dovecot[54239]: auth-worker(1...


Results
===

Failregex: 0 total

Ignoreregex: 0 total

Date template hits:
|- [# of hits] date format
|  [1] (?:DAY )?MON Day 24hour:Minute:Second(?:\.Microseconds)?(?: Year)?
`-

Lines: 1 lines, 0 ignored, 0 matched, 1 missed
[processed in 0.00 sec]

|- Missed line(s):
|  Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): 
sql(u...@bordo.com.au,::1,): Password 
mismatch (given password: 2)
`-

Any other suggestions?

Thanks,

James.


smime.p7s
Description: S/MIME cryptographic signature

Re: pop3-login core dump when using TLSSTART on version dovecot-2.2.32 (INTERNAL)

[Aki Tuomi]

Hi!

I tried to reproduce this problem with dovecot-2.2.32 and OpenSSL 1.0.1k
and was not able to. I enabled -DREF_CHECK on OpenSSL, but to no avail,
the process did not crash. Is there something else you've done?

Aki


On 11.09.2017 08:07, arvid.ei...@telenor.com wrote:
> Hi,
>
> Here is the gdb output.
>
> Arvid
>
> GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-94.el7
> Copyright (C) 2013 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later 
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-redhat-linux-gnu".
> For bug reporting instructions, please see:
> ...
> Reading symbols from 
> /local/misc/mail/dovecot-32/libexec/dovecot/pop3-login...done.
> [New LWP 15894]
> Core was generated by `dovecot-test/pop3-login'.
> Program terminated with signal 6, Aborted.
> #0  0x7ff0bd9cf1d7 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install 
> glibc-2.17-157.el7_3.1.x86_64
> (gdb) bt full
> #0  0x7ff0bd9cf1d7 in raise () from /lib64/libc.so.6
> No symbol table info available.
> #1  0x7ff0bd9d08c8 in abort () from /lib64/libc.so.6
> No symbol table info available.
> #2  0x7ff0bd3c0f2f in engine_unlocked_finish (e=0x1c51c60, 
> unlock_for_handlers=1) at eng_init.c:115
> to_return = 1
> #3  0x7ff0bd3c1064 in ENGINE_finish (e=0x1c51c60) at eng_init.c:150
> to_return = 1
> #4  0x7ff0be0f9300 in ssl_proxy_deinit () from 
> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
> No symbol table info available.
> #5  0x7ff0be0f4472 in main_deinit () from 
> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
> No symbol table info available.
> #6  0x7ff0be0f479f in login_binary_run () from 
> /local/nextmail/dovecot/lib64/dovecot/libdovecot-login.so.0
> No symbol table info available.
> #7  0x004032da in main (argc=1, argv=0x7ffe3059f3f8) at client.c:356
> No locals.
>
>
>
> -Original Message-
> From: Aki Tuomi [mailto:aki.tu...@dovecot.fi] 
> Sent: 8. september 2017 14:08
> To: Eikås Arvid; dovecot@dovecot.org
> Subject: Re: pop3-login core dump when using TLSSTART on version 
> dovecot-2.2.32 (OPEN)
>
> I assume you mean STARTTLS. Can you provide gdb /path/to/bin /path/to/core 
> and provide output of bt full?
>
> Aki
>
>
> On 08.09.2017 15:01, arvid.ei...@telenor.com wrote:
>> Hi,
>>
>> Pop3-login are CORE-dumping when I log on with TLSSTART, I believe the same 
>> will happen with imap-logon to, but I have not tested it yet.
>> The TLS session is coming up and it works fine until I log off, then it's 
>> core dump.  Open sslvesrion is   openssl-1.0.2k.
>> We ran dovecot-2.2.27 before we upgraded to dovecote-2.2.32, and that 
>> seems to work fine. (not core dumping)
>>
>>
>> Arvid
>>
>>
>> LOG
>> Sep 05 14:27:34 pop3-login: Debug: SSL: elliptic curve secp384r1 will 
>> be used for ECDH and ECDHE key exchanges Sep 05 14:30:30 pop3-login: 
>> Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE 
>> key exchanges Sep 05 14:30:30 pop3-login: Debug: SSL: elliptic curve 
>> secp384r1 will be used for ECDH and ECDHE key exchanges Sep 05 
>> 14:30:42 pop3-login: Debug: SSL: elliptic curve secp384r1 will be used 
>> for ECDH and ECDHE key exchanges Sep 05 14:30:42 pop3-login: Debug: 
>> SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key 
>> exchanges Sep 05 14:30:50 pop3-login: Info: Login: user=, 
>> method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=18361, secured, 
>> session=<65m8ZXBYtpN/AAAB> Sep 05 14:30:50 pop3-login: Error: 
>> ENGINE_finish, bad functional reference count Sep 05 14:30:50 
>> pop3-login: Fatal: master: service(pop3-login): child 18359 killed 
>> with signal 6 (core dumped)
>>
>>
>>
>>
>>
>>
>> From  ./crypto/engine/eng_init.c
>>
>> .
>> int engine_unlocked_finish(ENGINE *e, int unlock_for_handlers) {
>> int to_return = 1;
>>
>> /*
>>  * Reduce the functional reference count here so if it's the terminating
>>  * case, we can release the lock safely and call the finish() handler
>>  * without risk of a race. We get a race if we leave the count until
>>  * after and something else is calling "finish" at the same time -
>>  * there's a chance that both threads will together take the count from 2
>>  * to 0 without either calling finish().
>>  */
>> e->funct_ref--;
>> engine_ref_debug(e, 1, -1);
>> if ((e->funct_ref == 0) && e->finish) {
>> if (unlock_for_handlers)
>> CRYPTO_w_unlock(CRYPTO_LOCK_ENGINE);
>> to_return = e->finish(e);
>> if (unlock_for_handlers)
>> CRYPTO_w_lock(CRYPTO_LOCK_ENGINE);
>> if (!to_return)
>> return 0;
>> }
>> #ifdef REF_CHECK
>> if (e->funct_ref < 0) 

Re: Is it possible to disable pipelining in imapc?

[Aki Tuomi]

On 08.09.2017 15:29, Nagy, Attila wrote:
> On 09/08/2017 01:53 PM, Aki Tuomi wrote:
>>
>> On 08.09.2017 14:50, Nagy, Attila wrote:
>>> Hi,
>>>
>>> I've a broken IMAP server, which doesn't support pipelining and fails
>>> on dovecot's attempt to do this ([C] is dovecot's imapc, [S] is the
>>> IMAP server):
>>>
>>> [C] 24 LOGIN "user" "pass"
>>> [S] 23 OK
>>> [C] 25 SELECT "INBOX"
>>> [C] 23 UID FETCH 2 (INTERNALDATE)
>>> [S] 25 OK
>>> [S] 23 BAD No mailbox selected
>>>
>>> Sadly, fixing the server would be a hard task, turning off pipelining
>>> in dovecot's imapc (which already supports many workarounds for
>>> different servers) could be easy...
>>>
>>> ... except I can see no place to do this in imapc (only in pop3c and
>>> imap proxy, which I guess doesn't apply here).
>>>
>>> If I'm right, may I ask for a knob for such dumb servers?
>>>
>>> Thanks,
>> For proxy, use proxy_nopipelining
>>
> Are you sure it works with imapc? I've tried it, and dovecot still
> does the above.

So what are you exactly doing?

Aki

Re: Fail2ban 'Password mismatch' regex

[Christian Kivalo]

On 2017-09-11 08:57, James Brown wrote:

I have turned on 'auth_debug_passwords=yes’ in dovecot.conf.

I’m trying to get Fail2ban to detect this log line:

Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): 
sql(u...@bordo.com.au 
,::1,): 
Password mismatch (given password: 2)


I’ve added it as the last line of my dovecot filter regex:

failregex =
^%(__prefix_line)s(%(__pam_auth)s(\(dovecot:auth\))?:)?\s+authentication 
failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* 
rhost=(\s+user=\S*)?\s*$
^%(__prefix_line)s(pop3|imap)-login: (Info: )?(Aborted 
login|Disconnected)(: Inactivity)? \(((auth failed, \d+ attempts)( in 
\d+ secs)?|tried to use (disabled|disallo$
^%(__prefix_line)s(Info|dovecot: 
auth\(default\)|auth-worker\(\d+\)): pam\(\S+,\): 
pam_authenticate\(\) failed: (User not known to the underlying 
authentication$
^%(__prefix_line)s(auth|auth-worker\(\d+\)): 
(pam|passwd-file)\(\S+,\): unknown user\s*$
^%(__prefix_line)s(auth|auth-worker\(\d+\)): Info: 
ldap\(\S*,,\S*\): invalid credentials\s*$
^%(__prefix_line)sauth: Info: sql\(\S+,\): (Password 
mismatch|unknown user)( \((SHA1 of given password: [0-9a-f]{5,40}|given 
password: \w*)\))?$
  ^%(__prefix_line)sauth: Info: sql\(\S+,,\<\S+\>\): 
(Password mismatch|unknown user)( \((SHA1 of given password: 
[0-9a-f]{5,40}|given password: \w*)\))?$

^^^
You are missing the ID after the host part.


Have spent ages googling and trying different variations.

Does anyone have a fail2ban regex that would work on the above Dovecot 
log line?


(Running latest versions of Dovecot and fail2ban)

Many thanks,

James.


--
 Christian Kivalo

Fail2ban 'Password mismatch' regex

[James Brown]

I have turned on 'auth_debug_passwords=yes’ in dovecot.conf.

I’m trying to get Fail2ban to detect this log line:

Sep 11 15:52:49 mail dovecot[54239]: auth-worker(10094): sql(u...@bordo.com.au 
,::1,): Password 
mismatch (given password: 2)

I’ve added it as the last line of my dovecot filter regex:

failregex = 
^%(__prefix_line)s(%(__pam_auth)s(\(dovecot:auth\))?:)?\s+authentication 
failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* 
rhost=(\s+user=\S*)?\s*$
^%(__prefix_line)s(pop3|imap)-login: (Info: )?(Aborted 
login|Disconnected)(: Inactivity)? \(((auth failed, \d+ attempts)( in \d+ 
secs)?|tried to use (disabled|disallo$
^%(__prefix_line)s(Info|dovecot: 
auth\(default\)|auth-worker\(\d+\)): pam\(\S+,\): pam_authenticate\(\) 
failed: (User not known to the underlying authentication$
^%(__prefix_line)s(auth|auth-worker\(\d+\)): 
(pam|passwd-file)\(\S+,\): unknown user\s*$
^%(__prefix_line)s(auth|auth-worker\(\d+\)): Info: 
ldap\(\S*,,\S*\): invalid credentials\s*$
^%(__prefix_line)sauth: Info: sql\(\S+,\): (Password 
mismatch|unknown user)( \((SHA1 of given password: [0-9a-f]{5,40}|given 
password: \w*)\))?$

Have spent ages googling and trying different variations.

Does anyone have a fail2ban regex that would work on the above Dovecot log line?

(Running latest versions of Dovecot and fail2ban)

Many thanks,

James.

smime.p7s
Description: S/MIME cryptographic signature