Re: Errors after upgrade to v2.3.17

[Aki Tuomi]

Hi Adrian, Aurel.

I am not sure why Adrian consider that .dovecot.lda-dupes would have been 
changed like this?

Your problem is that you have configured mail home and mail location to point 
to same place, causing dovecot to consider .dovecot.lda-dupes as a maildir 
folder. 

This is a very common misconfiguration, which leads into exactly this kind of 
issues.

The recommended configuration is to configure mail_location=maildir:~/mail to 
avoid this issue.

Aki

> On 02/12/2021 20:42 Adrian Minta  wrote:
> 
> 
> Hi Aurel,
> apparently before 2.3.17 '.dovecot.lda-dupes' and '.dovecot.lda-dupes.lock' 
> where files, but starting with this release they are full folders with 
> cur/new/tmp in them.
> 
> Don't know if this is a bug or something is missing from the docs.
> 
> On 12/2/21 1:55 PM, Aurel Mihai wrote:
> 
> > Hello,
> > 
> > after upgrade to version 2.3.17, we enconter a lot of errors, such as:
> >  
> > 
> > Dec 2 13:41:22 mail5 dovecot: 
> > lmtp(user@domain)<41944>: Error: lmtp-server: conn 
> > unix:pid=40487,uid=107 [22]: rcpt user@domain: 
> > stat(/home/virtual0/domain/user/.dovecot.lda-dupes/tmp) failed: Not a 
> > directory
> > 
> > 
> > 
> > 
> > 
> > 
> > Any help please?
> > 
> > Aurel
> > 
> > 
> > 
> >  
> -- 
> Best regards,
> Adrian Minta
> 
> 
>

DoveCot Virtual Mailboxes With MySql

[postfix]

PLEASE HELP - Dovecot Virtual Mailboxes Using MySql

I have a thread detailing my problem with config here:
https://ubuntuforums.org/showthread.php?t=2469564

 

VPS Ubuntu 20.04 postfix 3.4.13 and dovecot 2.3.7.2

I've got multiple domains each having multiple emails.

I'm using Virtual Mailboxes and MySql.

My first domain: softlinksys.com has an MX pointing at mail.softlinksys.com.
You can easily verify it. The IP address shown on https://mxtoolbox.com/ is:
194.163.45.150
  which is my VPS IP address.

 

I have a virtual_domains record in mysql with softlinksys.com as the domain.

I have an email address mail...@softlinksys.com
  in virtual_users in mysql.

In the client app: Thunderbird the imap server is set as
mail.softlinksys.com

I can't connect that user to the mailbox from Thunderbird.

If I change the imap server in Thunderbird to softlinksys.com the mailbox
will connect

I can't send an email to mail...@softlinksys.com
 . It bounces whether MySql
virtual_domains.name = softlinksys.com or mail.softlinksys.com.

 

I've tried both softlinksys.com and mail.softlinksys.com in the database
record and in Thunderbird.

I just can't make it work.

PLEASE HELP! How can I resolve this problem?

 

Kristy Atkins

ViviData SaaS

 

Re: execve(/usr/bin/sieve-test) failed: Argument list too long

[Aki Tuomi]

> On 02/12/2021 17:16 Patrick Cernko  wrote:
> 
>  
> Hi Dovecot developers,
> 
> while debugging the above error message from sieve-test, I found out, 
> that the content of directive ssl_ca is added as env var SSL_CA by 
> doveconf on execve and sieve-test now uses doveconf.
> 
> In our setup, ssl_ca is set to
> ssl_ca =  on our director servers. We have backend servers with certificates 
> signed by two different CAs and to avoid problems if a backend switches 
> to a different CA, I decided to allow all "known" CAs. The corresponding 
> env var SSL_CA has more than 230500 bytes, which causes execve to fail 
> with error E2BIG.
> 
> I found a workaround for the problem by setting
> ssl_ca =  Where this file contains only the two CAs used atm. However I would like 
> to request a fix for this issue as others might also want to have all 
> "known" CAs set for dovecot director backend connections.
> 
> Best,
> -- 
> Patrick Cernko  +49 681 9325 5815
> Joint Administration: Information Services and Technology
> Max-Planck-Institute fuer Informatik & Softwaresysteme

Hi!

Thanks for reporting this issue, it's related to a known issue and will be 
fixed.

Aki

Re: LDAP Help

[Mihai Badici]

On 12/2/21 10:25 PM, Günther J. Niederwimmer wrote:

Hello Dovecot professionals,

I have a working user authentication with LDAP, now I want to allow the users
to use mailAterneteAddress for their account, unfortunately I can't find any
filter settings for dovecot that this works? I just can't find the right
settings for LDAP (FreeIPA).

Does anyone of you have any hints or links so that I can get on with it.
Somehow I don't understand how I can umconvigure the dovecot-ldap.conf.ext

Thank you for your help.



allow it for delivery or for authentication? There are different cases. 
Delivery is done by MTA but usually the final delivery use LDA or LMTP 
in order to ensure sieve filtering.

Re: Requested CRAM-MD5 scheme, but we have only CRYPT

[Alexander Dalloz]

Am 02.12.2021 um 10:11 schrieb Christian Mack:

You only can use CRAM-MD5 when your authentication source provides plain
passwords.


And that's the biggest issue with these shared secret authentication 
mechanisms: that you have to store the passwords unencrypted/unsalted. 
Never a good idea to store passwords in plain text.


Alexander

LDAP Help

[Günther J . Niederwimmer]

Hello Dovecot professionals,

I have a working user authentication with LDAP, now I want to allow the users 
to use mailAterneteAddress for their account, unfortunately I can't find any 
filter settings for dovecot that this works? I just can't find the right 
settings for LDAP (FreeIPA).

Does anyone of you have any hints or links so that I can get on with it.
Somehow I don't understand how I can umconvigure the dovecot-ldap.conf.ext

Thank you for your help.
-- 
mit freundlichen Grüßen / best regards

  Günther J. Niederwimmer

Re: Errors after upgrade to v2.3.17

[Adrian Minta]

Hi Aurel,

apparently before 2.3.17 '.dovecot.lda-dupes' and 
'.dovecot.lda-dupes.lock' where files, but starting with this release 
they are full folders with cur/new/tmp in them.


Don't know if this is a bug or something is missing from the docs.

On 12/2/21 1:55 PM, Aurel Mihai wrote:

Hello,
after upgrade to version 2.3.17, we enconter a lot of errors, such as:

Dec  2 13:41:22 mail5 dovecot: 
lmtp(user@domain)<41944>: Error: lmtp-server: 
conn unix:pid=40487,uid=107 [22]: rcpt user@domain: 
stat(/home/virtual0/domain/user/.dovecot.lda-dupes/tmp) failed: Not a 
directory




Any help please?
Aurel

**


--
Best regards,
Adrian Minta

RE: Searching 30 GB mailbox

[Scott]

Why are you still going through Dovecot for this ?

@ 4.5 million messages you could just reprogram your app to search solr 
directly, it will return the uid of the message and you can use that directly 
in imap uid fetch.

Scott

-Original Message-
From: dovecot  On Behalf Of Einar Bjarni 
Halldórsson
Sent: Thursday, December 2, 2021 7:47 AM
To: dovecot@dovecot.org
Subject: Re: Searching 30 GB mailbox


>
> If I search for the UID I get
>
> --
> 3 fetch 5166713 ENVELOPE
> 3 BAD Error in IMAP command FETCH: Invalid messageset (0.001 + 0.000 
> secs).
> --
> If I run the search in IMAP I get
>
> --
> 4 SEARCH HEADER Messgage-ID
> <64jf4cstuxzuvatjbbrvp3ibjnmbeguvxzoa1osm...@min.isnic.is>
> * SEARCH
> * 4465180 EXISTS
> * 88 RECENT
> 4 OK Search completed (29.674 + 0.000 + 29.523 secs).
> --
>

I just realised I'm misreading the return from the search. Redid it and this is 
from telnet

--
3 search header MESSAGE-ID 64jf4CStuxzUvATjBbRVP3IbJnMbEguVXzOa1OSmjIg
* SEARCH 4327049
3 OK Search completed (28.955 + 0.000 + 28.837 secs).
--

Same search from command line

--
doveadm search -u hostmasterlog mailbox INBOX HEADER MESSAGE-ID 
64jf4CStuxzUvATjBbRVP3IbJnMbEguVXzOa1OSmjIg
c92f64f79f0d1ed01e6d5b314f04886c 5166713
--

.einar

LDAP and user duplicated with replication

[Claudio Corvino]

Hi,

I have two IMAP/LMTP Dovecot server in replica (version 2.3.4.1), I use 
LDAP/AD for /userdb, /replica is working.


When I do a search like:

/doveadm replicator status '*'/

I receive user duplicated, with and without the domain part, for example:

/test/

/t...@domain.com/

but they are the same user; this lead the replicator doing twice the 
work of replication.


I think this is related to //etc/dovecot/dovecot-ldap.conf/ that is 
configured in this way:


/hosts = xxx/

/base = dc=xxx,dc=xxx
ldap_version=3
auth_bind = yes
dn = cn=xxx,cn=Users,dc=xxx,dc=xxx
dnpass = xxx
scope = subtree
user_attrs = 
sAMAccountName=home=/mnt/mail-storage-lv0007/%$,=uid=501,=gid=501

pass_attrs = sAMAccountName=user
user_filter = (&(objectclass=person)(samaccountname=%n))
pass_filter= (&(objectclass=person)(samaccountname=%n))
iterate_attrs = sAMAccountName=user
# With following filter we exclude all objects without an email address, 
all computers and all inactive accounts

/

/iterate_filter = 
(&(objectCategory=person)(mail=*)(objectClass=user)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))

/


I think that *iterate_attrs* and *iterate_filter* should be fixed, but I 
don't know how/./


Please could someone give me any hints?

Thanks/
/

Regards/
/


smime.p7s
Description: S/MIME Cryptographic Signature

execve(/usr/bin/sieve-test) failed: Argument list too long

[Patrick Cernko]

Hi Dovecot developers,

while debugging the above error message from sieve-test, I found out, 
that the content of directive ssl_ca is added as env var SSL_CA by 
doveconf on execve and sieve-test now uses doveconf.


In our setup, ssl_ca is set to
ssl_ca = on our director servers. We have backend servers with certificates 
signed by two different CAs and to avoid problems if a backend switches 
to a different CA, I decided to allow all "known" CAs. The corresponding 
env var SSL_CA has more than 230500 bytes, which causes execve to fail 
with error E2BIG.


I found a workaround for the problem by setting
ssl_ca = Where this file contains only the two CAs used atm. However I would like 
to request a fix for this issue as others might also want to have all 
"known" CAs set for dovecot director backend connections.


Best,
--
Patrick Cernko  +49 681 9325 5815
Joint Administration: Information Services and Technology
Max-Planck-Institute fuer Informatik & Softwaresysteme



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Searching 30 GB mailbox

[Einar Bjarni Halldórsson]

If I search for the UID I get

--
3 fetch 5166713 ENVELOPE
3 BAD Error in IMAP command FETCH: Invalid messageset (0.001 + 0.000 
secs).

--
If I run the search in IMAP I get

--
4 SEARCH HEADER Messgage-ID 
<64jf4cstuxzuvatjbbrvp3ibjnmbeguvxzoa1osm...@min.isnic.is>

* SEARCH
* 4465180 EXISTS
* 88 RECENT
4 OK Search completed (29.674 + 0.000 + 29.523 secs).
--



I just realised I'm misreading the return from the search. Redid it and 
this is from telnet


--
3 search header MESSAGE-ID 64jf4CStuxzUvATjBbRVP3IbJnMbEguVXzOa1OSmjIg
* SEARCH 4327049
3 OK Search completed (28.955 + 0.000 + 28.837 secs).
--

Same search from command line

--
doveadm search -u hostmasterlog mailbox INBOX HEADER MESSAGE-ID 
64jf4CStuxzUvATjBbRVP3IbJnMbEguVXzOa1OSmjIg

c92f64f79f0d1ed01e6d5b314f04886c 5166713
--

.einar

Re: Searching 30 GB mailbox

[Einar Bjarni Halldórsson]

If you have fts_enforced = yes then that SEARCH probably goes to solr even if 
message-id is found from cache. So the delay is probably on solr side.

Can you try to remove fts_enforced for that one particular user?

I only added fts_enforced yesterday. When I sent the original post I 
wasn't running with fts_enforced.


I did finally manage to query solr with curl with the query dovecot 
issues and I get this response:






  0
  1
  
    {    "params": {    "q": "{!lucene q.op=AND} 
hdr:64jf4cstuxzuvatjbbrvp3ibjnmbeguvxzoa1osm...@min.isnic.is", 
"fl": "uid,score",    "sort": "uid asc",    "fq": 
"+box:c92f64f79f0d1ed01e6d5b314f04886c +user:hostmasterlog", "rows": 
5327110,    "wt": xml }}

  

numFoundExact="true">

  
    5166713
    131.64821



--

If I search for the UID I get

--
3 fetch 5166713 ENVELOPE
3 BAD Error in IMAP command FETCH: Invalid messageset (0.001 + 0.000 secs).
--
If I run the search in IMAP I get

--
4 SEARCH HEADER Messgage-ID 
<64jf4cstuxzuvatjbbrvp3ibjnmbeguvxzoa1osm...@min.isnic.is>

* SEARCH
* 4465180 EXISTS
* 88 RECENT
4 OK Search completed (29.674 + 0.000 + 29.523 secs).
--

Isn't it likely that the dovecot index and solr index are out of sync? 
Solr seems to point to a UID that doesn't exist in dovecot. If so, how 
can I fix it?


.einar

Errors after upgrade to v2.3.17

[Aurel Mihai]

Hello,
after upgrade to version 2.3.17, we enconter a lot of errors, such as:

Dec  2 13:41:22 mail5 dovecot:
lmtp(user@domain)<41944>:
Error: lmtp-server: conn unix:pid=40487,uid=107 [22]: rcpt user@domain:
stat(/home/virtual0/domain/user/.dovecot.lda-dupes/tmp) failed: Not a
directory

Here is my config:
#dovecot -n
# 2.3.17 (e2aa53df5b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.17 (054dddfa)
# OS: Linux 4.19.0-18-amd64 x86_64 Debian 10.11 nfs
# Hostname:
auth_cache_verify_password_with_worker = yes
auth_failure_delay = 5 secs
auth_mechanisms = plain login
auth_verbose = yes
auth_worker_max_count = 256
default_client_limit = 1
default_process_limit = 1
default_vsz_limit = 1 G
dict {
  lastlogin = mysql:/etc/dovecot/mysql/dovecot-dict-lastlogin.conf
  quotadict = mysql:/etc/dovecot/mysql/dovecot-dict-quota.conf
}
disable_plaintext_auth = no
first_valid_uid = 100
imap_hibernate_timeout = 5 secs
imap_idle_notify_interval = 3 mins
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l pid=%p %c
login_trusted_networks = 
mail_fsync = always
mail_location = maildir:/home/virtual0/%d/%u
mail_nfs_index = yes
mail_nfs_storage = yes
mail_plugins = zlib quota mail_log notify
mail_privileged_group = mail
maildir_stat_dirs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
metric imap_command {
  filter = event=imap_command_finished AND tagged_reply_state=OK
  group_by = cmd_name
}
metric imap_select_no {
  filter = event=imap_command_finished AND cmd_name=SELECT AND
tagged_reply_state=NO
}
metric imap_select_no_notfound {
  filter = event=imap_command_finished AND cmd_name=SELECT AND
tagged_reply="NO*Mailbox doesn't exist:*"
}
metric storage_http_gets {
  filter = event=http_request_finished AND category=storage AND method=get
}
mmap_disable = yes
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Spam {
auto = subscribe
autoexpunge = 30 days
special_use = \Junk
  }
  mailbox Trash {
auto = subscribe
autoexpunge = 60 days
special_use = \Trash
  }
  mailbox virtual/All {
special_use = \All
  }
  prefix =
}
passdb {
  args = /etc/dovecot/mysql/dovecot-sql.conf
  driver = sql
}
plugin {
  last_login_dict = proxy::lastlogin
  last_login_key = # hidden, use -P to show it
  mail_log_events = delete undelete expunge copy mailbox_delete
mailbox_rename
  mail_log_fields = uid box msgid size
  quota = dict:user::proxy::quotadict
  quota_rule2 = Trash:ignore
  quota_rule3 = Spam:ignore
  quota_status_nouser = DUNNO
  quota_status_overquota = 552 5.2.2 The email account that you tried to
reach is over quota
  quota_status_success = DUNNO
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = ~/.dovecot.sieve
  sieve_before = /etc/dovecot/sieve/default.sieve
  sieve_dir = ~/sieve
  trash = /etc/dovecot/dovecot-trash.conf
  zlib_save = gz
  zlib_save_level = 6
}
protocols = imap sieve pop3 lmtp
service anvil {
  client_limit = 5
  unix_listener anvil-auth-penalty {
mode = 00
  }
}
service auth {
  client_limit = 5
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
  unix_listener auth-master {
group = mail
mode = 0660
user = vmail
  }
  user = root
  vsz_limit = 1 G
}
service dict {
  unix_listener dict {
mode = 0660
user = vmail
  }
}
service imap-hibernate {
  unix_listener imap-hibernate {
group = $default_internal_group
mode = 0660
  }
}
service imap-login {
  inet_listener imap {
address = *
port = 143
  }
  inet_listener imaps {
address = *
port = 993
  }
  process_min_avail = 16
  service_count = 1
}
service imap {
  extra_groups = $default_internal_group
  process_min_avail = 16
  unix_listener imap-master {
user = $default_internal_user
  }
  vsz_limit = 2 G
}
service lmtp {
  process_min_avail = 16
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
  vsz_limit = 1 G
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  process_min_avail = 16
  service_count = 1
}
service managesieve {
  vsz_limit = 1 G
}
service pop3-login {
  inet_listener pop3 {
address = *
port = 110
  }
  inet_listener pop3s {
address = *
port = 995
  }
  process_min_avail = 16
  service_count = 1
}
service pop3 {
  process_min_avail = 16
  vsz_limit = 1 G
}
service quota-status {
  executable = /usr/lib/dovecot/quota-status -p postfix
  process_min_avail = 16

Re: Requested CRAM-MD5 scheme, but we have only CRYPT

[Christian Mack]

Hello

auth_mechanisms are only for encrypting passwords while authenticating.
They have nothing to do with transport encryption aka TLS and STARTTLS.

You only can use CRAM-MD5 when your authentication source provides plain
passwords.
As you use password hashes in your authentication source, you have to
disable it.
Else a client will try to send you the CRAM-MD encrypted password, which
you can not check for validity.

Hope this clears it a bit.


Kind regards,
Christian Mack

On 01.12.21 23:26, absolutely_f...@libero.it wrote:
> Hi,
> I wondering if I can simply disable CRAM-MD5 and/or DIGEST-MD5.
> Are they useful in case of SSL or TLS connections?
> Thankyou 
> 
>> Il 01/12/2021 18:42 Aki Tuomi  ha scritto:
>>
>>  
>> auth_mechanisms = plain login digest-md5 cram-md5
>>
>> You still advertise them though.
>>
>> Aki


-- 
Christian Mack
Universität Konstanz
Kommunikations-, Informations-, Medienzentrum (KIM)
Abteilung IT-Dienste Forschung und Lehre
78457 Konstanz
+49 7531 88-4416



smime.p7s
Description: S/MIME Cryptographic Signature

Re: Searching 30 GB mailbox

[Sami Ketola]

> On 2. Dec 2021, at 10.57, Einar Bjarni Halldórsson  wrote:
> root@ht-mailstore01:/data/mail/hostmasterlog/mdbox/mailboxes/INBOX/dbox-Mails 
> # doveadm dump . | grep -c hdr.MESSAGE-ID
> 4464736
> 
> In dovecot config I have `fts_enforced = yes` and after the search for 
> Message-ID finishes and returns UID, I can do a FETCH .. ENVELOPE on it and 
> it responds immediately.
> I was under the impression that fts_enforced forces all searches, headers and 
> body, to go to solr. Then all dovecot would have to do was to return the UID 
> returned by solr.
> Unless solr doesn't return UID and dovecot has to take the result from solr 
> and lookup the UID, and with a full 1 GB cache file it always has to scan the 
> whole index? 
> 
> Nothing seems to be completely broken, I always receive a result, it's just 
> that it takes 30 seconds when I really want it to be ~5 seconds at most.
> 
> I guess if we can't find a solution and 30 seconds becomes a real problem, 
> we'll split the mailbox up by years. It should help with the size of the 
> cache. It makes the searching code a little more complicated since it has to 
> figure out the sent date before it can search, but it's doable.

If you have fts_enforced = yes then that SEARCH probably goes to solr even if 
message-id is found from cache. So the delay is probably on solr side.

Can you try to remove fts_enforced for that one particular user?

Sami

Re: Searching 30 GB mailbox

[Einar Bjarni Halldórsson]

You can inspect the index files with doveadm dump to check what is 
cached. Not sure how it went with mdbox storage driver.
According to `man doveadm-dump` it just seems to dump index files. I 
tried dumping the cache file but it complains that it can't auto 
detect the file type.



Try this:

[root@ketola .INBOX]# pwd
/vmail/s...@ketola.io/index/.INBOX
[root@ketola .INBOX]# doveadm dump . | grep -c hdr.message-id
11007



root@ht-mailstore01:/data/mail/hostmasterlog/mdbox/mailboxes/INBOX/dbox-Mails 
# doveadm dump . | grep -c hdr.MESSAGE-ID

4464736

In dovecot config I have `fts_enforced = yes` and after the search for 
Message-ID finishes and returns UID, I can do a FETCH .. ENVELOPE on it 
and it responds immediately.
I was under the impression that fts_enforced forces all searches, 
headers and body, to go to solr. Then all dovecot would have to do was 
to return the UID returned by solr.
Unless solr doesn't return UID and dovecot has to take the result from 
solr and lookup the UID, and with a full 1 GB cache file it always has 
to scan the whole index?


Nothing seems to be completely broken, I always receive a result, it's 
just that it takes 30 seconds when I really want it to be ~5 seconds at 
most.


I guess if we can't find a solution and 30 seconds becomes a real 
problem, we'll split the mailbox up by years. It should help with the 
size of the cache. It makes the searching code a little more complicated 
since it has to figure out the sent date before it can search, but it's 
doable.


.einar