Re: Sieve: Avoiding duplicates in a folder while keeping in Inbox

[Stephan Bosch via dovecot]

Op 13-5-2024 om 14:14 schreef Nick Urbanik via dovecot:

On Thu, 2024-04-25 at 14:36 +1000, Nick Urbanik via dovecot wrote:

On Tue, 2024-04-23 at 09:42 +1000, Nick Urbanik via dovecot wrote:

Dear Folks,

I am in the process of learning Sieve, and want to be able to keep
an
email in Inbox, while also keeping *one* copy in another folder.
However, the same mail is copied from Inbox multiple times when I
run
sieve-filter -Wev .dovecot.sieve Inbox

My attempt to prevent this fails:
...
} elsif header :comparator "i;octet" :contains "From"
"pay...@paypal.com.au"  {
     if not duplicate {
     fileinto "paypal";
     }
     keep;
}

Please can anyone enlighten me?

If there is a Dovecot Sieve mailing list, please point me to it.


OK, let me rephrase the problem:

I want to keep a copy of an email in Inbox and put a copy into another
folder, but only once, even if I run sieve-filter as above multiple
times. I know there are many who know how to do this and I would really
appreciate your generous suggestions.


The sieve-filter tool was mainly created to mend mailboxes after 
failures in sieve execution at delivery time. It currently does not 
provide access to the duplicate database and any actions that operate on 
it are ignored.


What exactly are you using sieve-filter for? Why are the Dovecot LDA or 
the Dovecot LMTP service with the Sieve plugin not fulfilling your needs?


Regards,

Stephan.
___
dovecot mailing list -- dovecot@dovecot.org
To unsubscribe send an email to dovecot-le...@dovecot.org

Re: Dovecot & OAuth

[Stephan Bosch via dovecot]

Op 10-12-2019 om 16:44 schreef mizuki:

Thank you Stephan,

I'm wondering if I can read the track of the status of bug reports? 
Could you please advice?


No, this is for our own internal reference.


Thanks.
Mizuki

On Sun, Dec 8, 2019 at 6:40 AM Stephan Bosch > wrote:




On 06/12/2019 20:54, Aki Tuomi via dovecot wrote:
> Hi!
>
> It seems there is a bug in the oauth2 driver, it loads the cert
files wrong way. I'll make an internal bug report of this.

Tracking as DOP-1590.

Regards,

Stephan.


>> On 06/12/2019 16:42 mizuki mailto:mizuki0...@gmail.com>> wrote:
>>
>>
>> Hi,
>>
>> For troubleshooting purposes, I change the read/write
permissions on the certs and confirmed 'dovecot' can read them w/o
problem, but still seeing the same errors. :(
>> Mizuki
>>
>>
>> On Fri, Dec 6, 2019 at 1:35 AM Aki Tuomi
mailto:aki.tu...@open-xchange.com>>
wrote:
>>>
>>> Is the key/cert pair readable by dovecot user? auth process
does not run as root.
>>>
>>>
>>>
>>>
>>> You can add
>>>
>>>
>>>
>>>
>>> service auth {
>>>
>>> extra_groups = ssl_cert
>>>
>>> }
>>>
>>>
>>>
>>>
>>> and chgrp the cert to ssl_cert to allow access to the cert.
>>>
>>>
>>>
>>>
>>> Aki
>>>
 On 06/12/2019 04:16 mizuki via dovecot mailto:dovecot@dovecot.org>> wrote:







 I changed some of the tls options following the document, now
config is following:







 tokeninfo_url =
https://keycloak.com/auth/realms/mail/protocol/openid-connect/token
 introspection_url =

https://dovecot:7598e21b-ec34-481f-80d0-059bddae0...@keycloak.com/auth/realms/demo/protocol/openid-connect/token/introspect
 introspection_mode = post
 debug = yes
 rawlog_dir = /tmp/oauth2
 #force_introspection = yes
 username_attribute = username
 #active_attribute = active
 #active_value = true
 tls_ca_cert_file = /etc/pki/CA/certs/incommon-rsa-server-ca.crt
 tls_cert_file = /etc/pki/dovecot/certs/dovecot.pem
 tls_key_file = /etc/pki/dovecot/private/dovecot.pem


 ---




 The debug log is showing now slightly different msg ex:

 Dec 5 21:09:59 mktst4 dovecot: auth: Error:
oauth2(mizuki,10.0.2.1,<29b4iv+YKuuCx5Tr>): oauth2 failed:
Couldn't initialize SSL context: Can't load SSL certificate: There
is no valid PEM certificate.




 Still not able to connect to the keyclaok server. :(





 PS: Dovecot & Keycloak severs are both using the same legit
cert/key pair with CA file configured.




 Thanks!

 Mizuki






 On Thu, Dec 5, 2019 at 3:06 PM Aki Tuomi <
aki.tu...@open-xchange.com > wrote:


> Before declaring it not ready for prime time, did you try
setting
>
> tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt
>
> In the oauth2 configuration file as documented in
https://doc.dovecot.org/configuration_manual/authentication/oauth2 ?
>
> Aki
>
>> On 05/12/2019 21:58 mizuki via dovecot <
dovecot@dovecot.org > wrote:
>>
>>
>> Hi all,
>>
>> We'd like to enable OAuth with Keycloak in Dovecot, after
enabling 'OAUTHBEARER XOAUTH2' in Dovecot based on online
document, I can confirm Dovecot is ready for OAuth using openssl
command, however when the auth request comes in, it failed in
establishing a SSL connection with Keycloak server on port 443,
shown as following in debug logs. I can confirming using commands
'openssl s_client -connect :443' or 'curl -v
https://>

Re: Dovecot & OAuth

[Stephan Bosch via dovecot]

On 06/12/2019 20:54, Aki Tuomi via dovecot wrote:

Hi!

It seems there is a bug in the oauth2 driver, it loads the cert files wrong 
way. I'll make an internal bug report of this.


Tracking as DOP-1590.

Regards,

Stephan.



On 06/12/2019 16:42 mizuki  wrote:


Hi,

For troubleshooting purposes, I change the read/write permissions on the certs 
and confirmed 'dovecot' can read them w/o problem, but still seeing the same 
errors. :(
Mizuki


On Fri, Dec 6, 2019 at 1:35 AM Aki Tuomi  wrote:
  
Is the key/cert pair readable by dovecot user? auth process does not run as root.
  



  
You can add
  



  
service auth {
  
extra_groups = ssl_cert
  
}
  



  
and chgrp the cert to ssl_cert to allow access to the cert.
  



  
Aki
  

On 06/12/2019 04:16 mizuki via dovecot  wrote:
  



  



  
I changed some of the tls options following the document, now config is following:
  



  



  
tokeninfo_url = https://keycloak.com/auth/realms/mail/protocol/openid-connect/token

introspection_url = 
https://dovecot:7598e21b-ec34-481f-80d0-059bddae0...@keycloak.com/auth/realms/demo/protocol/openid-connect/token/introspect
introspection_mode = post
debug = yes
rawlog_dir = /tmp/oauth2
#force_introspection = yes
username_attribute = username
#active_attribute = active
#active_value = true
tls_ca_cert_file = /etc/pki/CA/certs/incommon-rsa-server-ca.crt
tls_cert_file = /etc/pki/dovecot/certs/dovecot.pem
tls_key_file = /etc/pki/dovecot/private/dovecot.pem

  
---
  



  
The debug log is showing now slightly different msg ex:
  
Dec 5 21:09:59 mktst4 dovecot: auth: Error: oauth2(mizuki,10.0.2.1,<29b4iv+YKuuCx5Tr>): oauth2 failed: Couldn't initialize SSL context: Can't load SSL certificate: There is no valid PEM certificate.
  



  
Still not able to connect to the keyclaok server. :(


  



  
PS: Dovecot & Keycloak severs are both using the same legit cert/key pair with CA file configured.
  



  
Thanks!
  
Mizuki


  



  
  
On Thu, Dec 5, 2019 at 3:06 PM Aki Tuomi < aki.tu...@open-xchange.com> wrote:


  

Before declaring it not ready for prime time, did you try setting
  
tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt
  
In the oauth2 configuration file as documented in https://doc.dovecot.org/configuration_manual/authentication/oauth2 ?
  
Aki
  

On 05/12/2019 21:58 mizuki via dovecot < dovecot@dovecot.org> wrote:


Hi all,

We'd like to enable OAuth with Keycloak in Dovecot, after enabling 'OAUTHBEARER XOAUTH2' in 
Dovecot based on online document, I can confirm Dovecot is ready for OAuth using openssl 
command, however when the auth request comes in, it failed in establishing a SSL connection 
with Keycloak server on port 443, shown as following in debug logs. I can confirming using 
commands 'openssl s_client -connect :443' or 'curl -v 
https://

Re: Accessing the -a and -d from dovecot-lda

[Stephan Bosch via dovecot]

On 26/11/2019 07:09, Aristedes Maniatis via dovecot wrote:


I've got exim set up to deliver like this:


command = /usr/local/libexec/dovecot/dovecot-lda -a 
$original_local_part@$original_domain -d $local_part@$domain  -f $sender_address


If an email to is bob+sa...@ish.com.au then we should have "-a 
bob+sa...@subdomain.ish.com.au -d b...@ish.com.au".



Now in dovecot, only b...@ish.com.au is a real user but I want to be 
able to use both addresses in different places. For LDAP lookup I need 
to reference b...@ish.com.au. But in sieve, I need to see the original 
address with plus notation.


However, dovecot appears to get the "-a" address into %u and the "-d" 
address can't be accessed.


https://doc.dovecot.org/configuration_manual/config_file/config_variables/


Is what is the point of dovecot-lda having both -a and -d options? Can 
I get at both bits of data?




In Sieve, the "envelope" test always uses the original envelope TO 
address (-a) for the "to" envelope part.


Regards,

Stephan.

Re: ssl_min_protocol = TLSv1.3 does not work

[Stephan Bosch via dovecot]

On 27/11/2019 08:13, Aki Tuomi via dovecot wrote:

On 26.11.2019 19.34, Laurens Post via dovecot wrote:

Hi all,

I'm trying to set up my server with support for TLS 1.3 only, but that
does not seem to be supported.
First off, TLS 1.3 itself does work fine, so it's not the config or
ssl library, and 1.3-only works fine with Postfix. The problem is only
in disabling TLS 1.2 for Dovecot.
On connection, I'm getting an error that 1.3 is an "Unknown
ssl_min_protocol setting".
Reading the source code, it seems that
`openssl_min_protocol_to_options` in
`src/lib-ssl-iostream/iostream-openssl-common.c` is simply missing an
entry like

{ SSL_TXT_TLSV1_3, TLS1_3_VERSION, SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 |
SSL_OP_NO_TLSv1_1 | SSL_OP_NO_TLSv1_2 }

Is this a bug, something intentional, or has it simply not been added
yet because nobody has been crazy enough to ask for it?

Kind regards,

Laurens

Just haven't gotten round to implement this yet. Will get there.


Tracking as DOP-1592.

Fix would likely be something like this (not tested):

diff --git a/src/lib-ssl-iostream/iostream-openssl-common.c 
b/src/lib-ssl-iostream/iostream-openssl-common.c

index d79c986ed8..e2a4c42ccb 100644
--- a/src/lib-ssl-iostream/iostream-openssl-common.c
+++ b/src/lib-ssl-iostream/iostream-openssl-common.c
@@ -9,6 +9,14 @@
 #include 
 #include 

+#if OPENSSL_VERSION_NUMBER >= 0x10101000L
+/* Weirdly, OpenSSL fails to define this constant, even though TLSv1.3 is
+   supported. */
+#ifndef SSL_TXT_TLSV1_3
+#define SSL_TXT_TLSV1_3 "TLSv1.3"
+#endif
+#endif
+
 /* openssl_min_protocol_to_options() scans this array for name and returns
    version and opt. opt is used with SSL_set_options() and version is 
used with

    SSL_set_min_proto_version(). Using either method should enable the same
@@ -23,6 +31,10 @@ static const struct {
    { SSL_TXT_TLSV1_1, TLS1_1_VERSION, SSL_OP_NO_SSLv3 | 
SSL_OP_NO_TLSv1 },

    { SSL_TXT_TLSV1_2, TLS1_2_VERSION,
    SSL_OP_NO_SSLv3 | SSL_OP_NO_TLSv1 | SSL_OP_NO_TLSv1_1 },
+#if OPENSSL_VERSION_NUMBER >= 0x10101000L
+   { SSL_TXT_TLSV1_3, TLS1_3_VERSION, SSL_OP_NO_SSLv3 | 
SSL_OP_NO_TLSv1 |

+ SSL_OP_NO_TLSv1_1 | SSL_OP_NO_TLSv1_2 },
+#endif
 };
 int openssl_min_protocol_to_options(const char *min_protocol, long *opt_r,
    int *version_r)



Regards,

Stephan.

Re: v2.2.36 to v2.3.8 maildirlock bug

[Stephan Bosch via dovecot]

On 22/10/2019 05:44, lty--- via dovecot wrote:

Dear all,
  Dovecot upgraded from v2.2.36 to v2.3.8 There is no problem with the 
configuration, but the lock directory seems to have an error.


[root@spam dovecot]# /usr/libexec/dovecot/maildirlock /path/to/maildir 1
Panic: BUG: No IOs or timeouts set. Not waiting for infinity.
19276[root@spam dovecot]# Error: Raw backtrace: 
/usr/lib64/dovecot/libdovecot.so.0(+0xe227a) [0x7f1cb567a27a] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0xe2321) [0x7f1cb567a321] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0x40f31) [0x7f1cb55d8f31] -> 
/usr/lib64/dovecot/libdovecot.so.0(+0xf93b4) [>xrf1cb56913b4] -> /usr/ 
Lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x28) 
[0x7f1cb5693918] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x5c) 
[0x7f1cb5691aac] -> /usr/lib64/dovecot/libdovecot. 
So.0(io_loop_run+0x38) [0x7f1cb5691cc8] -> 
/usr/libexec/dovecot/maildirlock(main+0x242) [0x7f1cb5b522c2] -> 
/lib64/libc.so.6(__libc_start_main+0x100) [0x7f1cb5222d20] -> / 
Usr/libexec/dovecot/maildirlock(+0xfa9) [0x7f1cb5b51fa9]



After google seems to be a bug, this is the address
Https://bugs.gentoo.org/682498

If this is not fixed, we are temporarily unable to upgrade to use the 
new version. Will the next version be fixed? We are looking forward to 
seeing that we have returned v2.2.36


The maildirlock tool will likely be deprecated. You should use dsync 
instead for making backups.


Regards,

Stephan.

Re: IMAP session hangs on 8k-long commands if COMPRESS=DEFLATE is enabled

[Stephan Bosch via dovecot]

On 13/11/2019 00:22, Guilhem Moulin via dovecot wrote:

Hi there,

Dovecot 2.3.7 appears to hang when the client sends a long command after
enabling the IMAP COMPRESS extension [RFC 4978].  PoC script attached
along with the doveconf(1) output.

Without COMPRESS=DEFLATE, and with the default ‘imap_max_line_length’
value (64k) I'm able send commands up to 65539 bytes long (that's 3
bytes more than 2¹⁶, so maybe the leading tag and the trailing CRLF
aren't counted), after which Dovecot rightfully replies with tagged BAD
responses.

 $ COMPRESS=n /tmp/many-fetch.pl 65539
 […]
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[65539 bytes]
 S: x OK Fetch completed (0.005 + 0.001 + 0.004 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[65540 bytes]
 S: x BAD Error in IMAP command UID FETCH: IMAP command line too large 
(0.001 + 0.001 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[65541 bytes]
 S: x BAD Error in IMAP command UID FETCH: IMAP command line too large 
(0.001 + 0.001 secs).

With COMPRESS=DEFLATE however, the server never sends anything back when
the command size exceeds 8192 bytes:

 $ COMPRESS=y /tmp/many-fetch.pl 8192
 […]
 S: x OK [READ-ONLY] Examine completed (0.001 + 0.000 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[8191 bytes]
 S: x OK Fetch completed (0.001 + 0.000 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[8192 bytes]
 S: x OK Fetch completed (0.001 + 0.000 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[8193 bytes]
 [ _hangs until the server terminates the session for inactivity_ ]
 S: * BYE Disconnected for inactivity.

strace(1) show that Dovecot didn't send the response, even though the
server log indicates that it's able to fully read the command.

Interestingly, when setting ‘imap_max_line_length’ to 4096, the server
also hangs after receiving a command of size >4096 bytes (instead of
replying with a tagged BAD response).

 $ COMPRESS=y /tmp/many-fetch.pl 4096
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[4095 bytes]
 S: x OK Fetch completed (0.001 + 0.000 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[4096 bytes]
 S: x OK Fetch completed (0.001 + 0.000 secs).
 C: x UID FETCH 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,… 
[4097 bytes]
 [ _hangs until the server terminates the session for inactivity_ ]

I'm also able to reproduce the buggy behavior against Dovecot versions
found in other Debian releases.  It's identical with 2.3.4 (Buster), the
server also hang past 8192 bytes.  2.2.27 (Stretch) and 2.2.13 (Jessie)
are able to send replies to 8k-long commands, but hang past 65536 bytes.
For all 3 releases, without COMPRESS=DEFLATE Dovecot doesn't hang and
rightfully sends tagged BAD responses to commands of size >65539 bytes.

So while the hanging behavior appears to be a long-standing bug, the
fact that's it's now hit at min($imap_max_line_length, 8192) instead of
$imap_max_line_length seems to be a regression from somewhere between
2.2.27 and 2.3.4.


I was able to confirm this problem with the current master. Tracking 
internally as DOP-1591.


Regards,

Stephan.

Re: lmtp proxy 'Invalid FROM: Missing domain'

[Stephan Bosch via dovecot]

On 24/11/2019 19:55, Marc Roos via dovecot wrote:
  
If I add a domain, error changes in:

connection.c: line 380 (connection_update_properties): assertion failed:
(conn->remote_port != 0)


-Original Message-
Subject: lmtp proxy 'Invalid FROM: Missing domain'


Looks like I have a correct working proxy on pop3. On both backend
servers I can run

cat /tmp/test.msg | /usr/libexec/dovecot/lmtp

Giving
250 2.1.5 OK
354 OK
Info: Disconnect from local: Connection closed (in DATA)

However if I run the same command on the proxy, I am getting this error
'Invalid FROM: Missing domain'. How is this FROM even relevant if
message just needs to be delivered to RCPT TO.


Both of these issues should be fixed for 2.3.9.

Regards,

Stephan.

Re: LMTP-Process stays in RCPT TO state

[Stephan Bosch via dovecot]

On 06/12/2019 11:08, Pascal Christen via dovecot wrote:

Hi

I tried to get some logs: https://pastebin.com/Z8xVzpzW

As you can see the process isn't shutdown and still in transaction as
long dovecot is running. It destroyed the transaction when I stopped
Dovecot. And this behavior only happens when the mailbox of user is full...

Any Ideas how to debug this correctly?


The provided logs look incomplete. The reception of the RCPT command by 
the server-side is not in the log, only the generation of the response. 
Also much of the client-side activity is not in the logs.


Also, a raw log of the TCP protocol exchange between Dovecot and Exim 
and also between proxy and backend can be useful. If  the connections 
are plaintext you can use wireshark or - if not - you can use the 
Dovecot rawlog facilities.


Regards,

Stephan.

So far, I haven't been able to reproduce anything weird at this end.
Can you provide:

- Output from `dovecot -n'
- Protocol logs from the connections between Exim and Dovecot
director/proxy and between Dovecot director/proxy and Dovecot backend
(e.g. using ngrep when connections are plaintext or using the rawlog
facility).
- Dovecot debug logs produced with `log_debug=category:lmtp' for both
director/proxy and backend

Regards,

Stephan.

Re: Sieve redirect does not collect the sendmail child process correctly (Dovecot 2.3.4.1, Pigeonhole 0.5.4)

[Stephan Bosch via dovecot]

On 03/11/2019 18:01, Delian Krustev via dovecot wrote:

On Saturday, November 2, 2019 10:40:45 PM EET Stephan Bosch 
 wrote:

I think we already know about this issue (tracked as DOP-942). It
apparently happens because qmail masks the SIGCHLD signal while Dovecot
doesn't unmask it before waiting for children.

You're right.
Isn't this a simple fix - a call to sigprocmask ?

This should not be a problem in qmail-lspawn/qmail-local as their code
has not been changed for years.
So there should have been a breaking change in Dovecot/Pigeonhole.


Does this fix it for you?

https://github.com/stephanbosch/dovecot-core/commit/f12062737aee89c2b41d3aa79044d003cc9337d2

Regards,

Stephan.



What follows is some more debugging from me.

The chain of processes that lead to dovecot-lda and sendmail execution look 
like this:

   PID
32457   qmail-lspawn
  6923qmail-local
  6924 sh -c vlelivermail
  6925  sh -c preline /usr/local/bin/vpop-lda
  6926   vpop-lda(perl script) -> deliver -> doveconf -> deliver
  6927 grep
  6928 pwd
  6929 sendmail -> qmail-inject
  6930   qmail-queue


Just before executing sendmail, dovecot-lda is registering a signal handler for 
SIGCHLD:

22:55:15.294647 epoll_create(128)   = 12
22:55:15.294668 fcntl(12, F_GETFD)  = 0
22:55:15.294687 fcntl(12, F_SETFD, FD_CLOEXEC) = 0
22:55:15.294708 epoll_ctl(12, EPOLL_CTL_ADD, 5, 
{EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=47455536, u64=94699781365040}}) = 0
22:55:15.294740 rt_sigaction(SIGCHLD, {sa_handler=0x7f15404b3640, sa_mask=[], 
sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f1540229840}, NULL, 
8) = 0
22:55:15.294763 pipe([13, 14])  = 0
22:55:15.294788 clone(child_stack=NULL, 
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0x7f15401ec2d0) = 6929


Later dovecot-lda checks non-blockingly whether sendmail has exited and waits 
for epoll IO event:

22:55:15.295175 wait4(6929, 0x562102d4fee4, WNOHANG, NULL) = 0
22:55:15.295211 epoll_wait(12, [], 1, 3) = 0
22:55:45.325580 kill(6929, SIGTERM) = 0
22:55:45.325769 epoll_wait(12, [], 1, 5000) = 0
22:55:50.331108 kill(6929, SIGKILL) = 0


I'm not sure why it is waiting for IO event on FD 5, which is a PIPE FD 
initialized much earlier,
with no actual reads/writes on it:

22:55:15.286845 pipe([5, 6])= 0
22:55:15.286870 fcntl(5, F_GETFL)   = 0 (flags O_RDONLY)
22:55:15.286889 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
22:55:15.286909 fcntl(6, F_GETFL)   = 0x1 (flags O_WRONLY)
22:55:15.286928 fcntl(6, F_SETFL, O_WRONLY|O_NONBLOCK) = 0

The pipe used for communication with the sendmail child is this:

22:55:15.294763 pipe([13, 14])  = 0
22:55:15.294788 clone(child_stack=NULL, 
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0x7f15401ec2d0) = 6929
22:55:15.294928 close(13)   = 0
22:55:15.294969 fcntl(14, F_GETFL)  = 0x1 (flags O_WRONLY)
22:55:15.295006 fcntl(14, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
22:55:15.295039 lseek(14, 0, SEEK_CUR)  = -1 ESPIPE (Illegal seek)
22:55:15.295065 getsockname(14, 0x7ffd59156a40, [28]) = -1 ENOTSOCK (Socket 
operation on non-socket)
22:55:15.295109 write(14, "X-Sieve: Pigeonhole Sieve 0.5.4 
()\nX-Sieve-Redirected-From: ...", 1461) = 1461
22:55:15.295150 close(14)   = 0
22:55:15.295175 wait4(6929, 0x562102d4fee4, WNOHANG, NULL) = 0
22:55:15.295211 epoll_wait(12, [], 1, 3) = 0
22:55:45.325580 kill(6929, SIGTERM) = 0




This is the process creation/waiting and signal mask debugging:


qmail-lspawn (PID 32457)
22:55:15.227742 rt_sigprocmask(SIG_BLOCK, [CHLD], NULL, 8) = 0
...
22:55:15.227970 clone(child_stack=NULL, 
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0x7f8a117de7d0) = 6923
...
22:55:15.228237 rt_sigprocmask(SIG_UNBLOCK, [CHLD], NULL, 8) = 0
22:55:15.228267 select(4, [0 3], NULL, NULL, NULL) = 1 (in [3])
22:55:50.334325 rt_sigprocmask(SIG_BLOCK, [CHLD], NULL, 8) = 0
22:55:50.334367 read(3, "Execution of Sieve filters was aborted due to temporary 
failure\n", 128) = 64
22:55:50.334406 rt_sigprocmask(SIG_UNBLOCK, [CHLD], NULL, 8) = 0
22:55:50.334440 select(4, [0 3], NULL, NULL, NULL) = ? ERESTARTNOHAND (To be 
restarted if no handler)
22:55:50.335625 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=6923, 
si_uid=1002, si_status=111, si_utime=0, si_stime=0} ---
22:55:50.335649 wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 111}], WNOHANG, 
NULL) = 6923
22:55:50.335683 close(4)= 0
22:55:50.335710 wait4(-1, 0x7fff05f68d8c, WNOHANG, NULL) = -1 ECHILD (No child 
processes)
22:55:50.335742 rt_sigreturn({mask=[]}) = -1 EINTR (Interrupted system call)
22:55:50.335773 rt_sigprocmask(SIG_BLOCK, [CHLD], NULL, 8) = 0
22:55:50.335801 rt_sigprocmask(SIG_UNBLOCK, [CHLD], NULL, 8) = 0
22:55:50.335827 select(4, [0 3], NULL, NULL, NULL) = 1 (in [3])
22:55:50.335874 rt_sigprocmask(SIG_BLOCK, [CHLD], NULL, 8) = 0
22:55:50.335899 read(3, "", 128)  

Re: dovecot sieve duplicates detection

[Stephan Bosch via dovecot]

On 17/08/2018 09:14, Stephan Bosch wrote:



Op 14/05/2018 om 23:03 schreef James Cassell:


On Mon, May 14, 2018, at 4:52 PM, Stephan Bosch wrote:


Op 25/04/2018 om 22:49 schreef James Cassell:

On Wed, Apr 25, 2018, at 3:20 PM, Stephan Bosch wrote:

Specify the ID used for duplicate checking explicitly using the
:uniqueid argument (https://tools.ietf.org/html/rfc7352#section-3.1).
Using the variables extenion, compose the uniqueid from the 
message-id

and the mailbox name.

In my experience with dovecot's implementation, you can set the ID 
only once in a script.  If you try to filter duplicates based on 
multiple IDs, only the first (or last, I don't remember) takes effect.



Do you have a detailed example of the supposed wrong behavior?

I don't have them readily available. Basically, the result of the 
first duplicate test in a script is taken as the result of any future 
duplicate test, even if the parameters to that future duplicate test 
in the same script are different and would otherwise result in a 
different output. The duplicate test is only evaluated once and its 
results are substituted everywhere.


For example, I might want to flag a message as a new conversation if 
I have not seen another message with the same subject. In the same 
script, I might want to discard messages that are exactly identical 
including message ID among others. The dovecot behavior would be to 
discard all messages that match a subject of previously received 
message.


I finally managed to review this issue and I can confirm that this is 
a bug.


Fix released in 2.3.9.

Regards,

Stephan.

Re: Error: proxy: Remote returned invalid banner: 220

[Stephan Bosch via dovecot]

On 29/11/2019 19:01, Marc Roos via dovecot wrote:

I had a working proxy setup added sieve to it, and out of the blue I get
this

Error: proxy: Remote returned invalid banner: 220

No idea what to do, nothing even in the mail list archive


Looks a lot like you're mixing up protocols, but I need your 
configuration to confirm (output from `dovecot -n`).


Regards,

Stephan.

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

[Stephan Bosch via dovecot]

Op 20-11-2019 om 8:50 schreef Jens Rey via dovecot:

Hi,

You can attach GDB to imap with follow-fork-mode set to 'child', so 
you can maybe see what causes the segfault.


now I am confused. Hopefully, I used GDB correctly, but as far as I 
can see, everything seemed to work.


The only difference I can see is that I executed the whole thing as 
root, but I would hope doveadm uses the correct user (vmail). At least 
on the file system all emails still belong to vmail, so I guess that 
works.


For a quick reference, here is what I executed:

root@mail:~# gdb --args doveadm exec imap -u user@domain
...
(gdb) set follow-fork-mode child
(gdb) r
. SELECT Spam
. MOVE 1 Inbox

And the (imho) relevant output:

imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Establishing connection

[New process 13335]
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked child process
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Connected to 
program

[Thread debugging using libthread_db enabled]
Using host libthread_db library 
"/lib/x86_64-linux-gnu/libthread_db.so.1".

process 13335 is executing new program: /bin/dash
process 13335 is executing new program: /usr/bin/rspamc
[Thread debugging using libthread_db enabled]
Using host libthread_db library 
"/lib/x86_64-linux-gnu/libthread_db.so.1".

[New Thread 0x7fffee7ff700 (LWP 13338)]
[New Thread 0x7fffebffe700 (LWP 13339)]
[New Thread 0x7fffe97fd700 (LWP 13340)]
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished 
streaming payload to program
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Finished 
input to program
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Disconnected
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Waiting for 
program to finish after 886 msecs

[Thread 0x7fffe97fd700 (LWP 13340) exited]
[Thread 0x7fffebffe700 (LWP 13339) exited]
[Thread 0x77fd1140 (LWP 13335) exited]
[Inferior 2 (process 13335) exited normally]
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Child process 
ended
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Debug: program 
exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (13335): Destroy
imap(user@domain)<13290><+PcoBrjs1F3qMwAAcD3JoA>: Info: sieve: pipe 
action: piped message to program `rspamd-learn-ham.sh'



There is more output, lots more, but we are talking about spam after 
all... If it is needed, please say so and I will provide it.


Hmm, so now it suddenly works. So, why would it not fail when run in GDB?



Regards,
Jens

Re: Panic: file smtp-client-connection.c: line 1142 (smtp_client_connection_established)

[Stephan Bosch via dovecot]

On 19/11/2019 13:25, Dirk via dovecot wrote:
I'm using dovecot in a multi server setup. Two directors forward pop3, 
imap and smtp to  backend dovecots with their central storage on NFS.

Yesterday I stumbled upon these log lines:

2019-11-18 20:17:51 lmtp(17274): Info: Connect from 172.22.1.6
2019-11-18 20:17:51 lmtp(17274): Panic: file smtp-client-connection.c: 
line 1142 (smtp_client_connection_established): assertion failed: 
(!conn->connect_succeeded)
2019-11-18 20:17:51 lmtp(17274): Fatal: master: service(lmtp): child 
17274 killed with signal 6 (core not dumped - 
https://dovecot.org/bugreport.html#coredumps - set 
/proc/sys/fs/suid_dumpable to 2)


Is a core dump required to find the source of that error?



This bug is known (DOV-3600), fixed and currently scheduled for release 
in 2.3.10.


Regards,

Stephan.


Thanks for any help!


$ doveconf -n
# 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
# OS: Linux 4.9.0-8-amd64 x86_64
# Hostname: m03.company.com 
auth_cache_negative_ttl = 65 secs
auth_cache_ttl = 3 mins
auth_cache_verify_password_with_worker = yes
auth_mechanisms = plain login
auth_verbose = yes
debug_log_path = /dev/stdout
default_client_limit = 10400
default_process_limit = 1024
default_vsz_limit = 1 G
deliver_log_format = msgid=%m: %s %{session_time} %$
director_mail_servers = 10.10.10.2 10.10.10.3
director_servers = 10.10.1.2 10.10.1.3
imap_id_log = *
imap_max_line_length = 2 M
import_environment = TZ CORE_OUTOFMEM CORE_ERROR
info_log_path = /dev/stdout
instance_name = dovecot_director
listen = 10.10.1.3,::
lmtp_proxy = yes
log_path = /dev/stderr
log_timestamp = "%Y-%m-%d %H:%M:%S "
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e 
%c %k session=<%{session}>

mail_max_userip_connections = 250
passdb {
  args = /etc/dovecot/sql/dovecot-dict-sql-passdb.conf
  driver = sql
  result_failure = return-fail
  result_internalfail = return-fail
  result_success = continue-ok
}
passdb {
  args = proxy=y starttls=any-cert master=direc...@m03.company.com 
 pass=#hidden_use-P_to_show# 
nopassword=y

  driver = static
  skip = unauthenticated
}
pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, uidl=%u
protocols = imap lmtp pop3
service auth-worker {
  user = $default_internal_user
}
service director {
  fifo_listener login/proxy-notify {
    mode = 0600
    user = $default_login_user
  }
  inet_listener {
    port = 9090
  }
  unix_listener director-userdb {
    mode = 0600
    user = vmail
  }
  unix_listener login/director {
    mode = 0666
  }
}
service imap-login {
  executable = imap-login director
  process_min_avail = 2
  vsz_limit = 1 G
}
service lmtp {
  inet_listener lmtp {
    port = 24
  }
  process_min_avail = 2
  unix_listener lmtp {
    mode = 00
  }
  user = vmail
  vsz_limit = 1 G
}
service log {
  user = dovenull
}
service pop3-login {
  executable = pop3-login director
  process_min_avail = 2
  vsz_limit = 1 G
}
service pop3 {
  user = vmail
}
ssl_cert = 

Re: Learning ham/spam with IMAPSieve: sa-learn-ham.sh terminated abnormally, signal 11

[Stephan Bosch via dovecot]

On 17/11/2019 19:40, Jens Rey via dovecot wrote:

Since I want to avoid debugging old and already fixed bugs, I updated to

# dovecot --version
2.3.8 (9df20d2db)

Now the debug log is a bit more chatty, but the error remains:

Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Created
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass 
environment: USER=u...@domain.tld
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass 
environment: HOME=/var/vmail/mailboxes/domain.tld/user
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Pass 
environment: HOST=mail.domain.tld

Debug: Mailbox INBOX: UID 22039: Opened mail because: mail stream
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: 
Establishing connection
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh: Forked 
child process
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
Connected to program
Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
write(program stdin) failed: Broken pipe
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
Failed to run program
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
Disconnected
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
Child process ended
Error: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
Terminated abnormally with signal 11
Debug: program exec:/usr/lib/dovecot/sieve/rspamd-learn-ham.sh (9891): 
Destroy
Error: sieve: pipe action: failed to pipe message to program 
`rspamd-learn-ham.sh': refer to server log for more information. 
[2019-11-17 19:32:40]

sieve: left message in mailbox 'INBOX'
Error: sieve: Execution of script 
/usr/lib/dovecot/sieve/report-ham.sieve failed


Dovecot forks a child process to execute the program. Subsequently, at 
some point, the child process fails with a segmentation fault. Granted, 
that could still be happening in Dovecot code, but since the child 
process quickly executes the actual program (the shell in this case), it 
is more likely that this problem emerges outside Dovecot.


You can attach GDB to imap with follow-fork-mode set to 'child', so you 
can maybe see what causes the segfault.


Regards,

Stephan.

Re: IMAP4 extensions for Visual Voicemail (VVM)

[Stephan Bosch via dovecot]

Op 15-11-2019 om 16:15 schreef Harald Leithner via dovecot:

If you use an Iphone (and your mobile proivder supports it) then you are
using it ;-)

just my 2 cents


https://source.android.com/devices/tech/config/voicemail

Android apparently supports it as well.

Regards,

Stephan.



Am 21.10.2019 um 12:26 schrieb Stephan Bosch via dovecot:


Op 20-10-2019 om 22:33 schreef Jean-Daniel via dovecot:



Le 20 oct. 2019 à 22:24, Mauricio Tavares via dovecot
mailto:dovecot@dovecot.org>> a écrit :

On Sun, Oct 20, 2019 at 10:43 AM Rajesh Bansal via dovecot
mailto:dovecot@dovecot.org>> wrote:

Hi Team,



I need to develop Visual VoiceMail solution. In this solution I need
a IMAP4 server, from which I can get a hit for each mail retrieval.
Can anyone help me if dovecot can be used for this purpose.


  That is rather vague. Do you want to do something like ol' biff
or what we used to do with Asterix 10 years ago (get an email with the
voicemail as as attachment)?

I guess he is talking about that:
https://www.gsma.com/newsroom/wp-content/uploads/2012/07/OMTP_VVM_Specification_1_3.pdf


I skimmed through this. It adds quite a few strange custom bits to the
IMAP protocol. Part of this are a few new commands and METADATA items,
which should be quite easy to implement. However, the amendments also
include various attachment-based QUOTA limits. Dovecot doesn't support
this and adding that is not trivial.

Also, this specification is quite old and outdated (it even mentions an
MD5 authentication mechanism). Is this even actively used anywhere?

Regards,

Stephan.

Re: dovecot ignores users sieve-script

[Stephan Bosch via dovecot]

Op 7-11-2019 om 13:30 schreef Jakobus Schürz via dovecot:

Hi there!

I recognized that dovecot ignores users sieve-script. The global and the
post-sieve-scripts are working, but users sievescript is ignored.

You can see the sieve trace-files last lines

   65:   finishing match with result: not matched
   65: jump if result is true
   65:   not jumping
   65: jumping to line 67
   67: jumping to line 77
   ## Finished executing script 'global'
  
  
   ## Started executing script 'junk'

    2: header test
    2:   starting `:contains' match with `i;ascii-casemap' comparator:
    2:   extracting `X-Spam' headers from message
    2:   finishing match with result: not matched
    2: jump if result is false
    2:   jumping to line 6
   ## Finished executing script 'junk'

Between global and junk-Script should sieve run users sieve script.

Before the Upgrade to 2.3.8 sieve was working correct. Now ALL messages
go to INBOX, no one is filtered anymore.

compiling the sievescript with

sudo -u vmail sievec sievefilter.sieve

gives no errormessage...

My users sievefilter is:

/var/mail/jakobus.schuerz/sieve/sievefilter.sieve

This directory contains:
lrwxrwxrwx 1 vmail vmail    17 Nov  7 12:44 jakobus.schuerz.sieve ->
sievefilter.sieve
-rw--- 1 vmail vmail 36834 Nov  7 13:25 jakobus.schuerz.svbin
-rw--- 1 vmail vmail 45588 Nov  7 11:43 sievefilter.sieve


Can you enable mail_debug and check the logs?


The dovecot conf dump:

# 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf

# Pigeonhole version 0.5.8 (b7b03ba2)
# OS: Linux 4.19.0-6-amd64 x86_64 Debian 10.1 # Hostname:
mail.mydomain.example
auth_mechanisms = plain login cram-md5 digest-md5
auth_socket_path = /var/run/dovecot/auth-userdb
disable_plaintext_auth = no
first_valid_uid = 1000
imap_capability = +XDOVECOT +SPECIAL-USE
imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags delay-newmail
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *, ::1
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
tls=%k
login_trusted_networks = 127.0.0.1/8 192.168.0.0/24 192.168.1.0/24
172.17.0.0/24 172.18.0.0/24
mail_gid = vmail
mail_home =  /var/mail/%u
mail_location =
maildir:~/Maildir:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/Maildir/%u:CONTROL=/var/lib/dovecot/db/control/Maildir/%u
mail_plugins = zlib quota acl listescape mail_log notify virtual fts
fts_lucene
mail_privileged_group = vmail
mail_server_admin = mailto:user@mydomain.example
mail_shared_explicit_inbox = yes
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext imapsieve vnd.dovecot.imapsieve
namespace Geteilt {
hidden = no
inbox = no
list = children
location =
maildir:/var/mail/%%u/Maildir:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/shared/%u/%%u
prefix = Geteilt/%%n/
separator = /
subscriptions = no
type = shared
}
namespace History {
hidden = no
inbox = no
list = children
location =
maildir:/var/mail/archiv/%u/:LAYOUT=fs:INDEX=/var/lib/dovecot/db/indexes/archiv/%u:CONTROL=/var/lib/dovecot/db/control/archiv/%u
mailbox Archiv {
auto = subscribe
special_use = \Archive
}
prefix = Archiv/
separator = /
subscriptions = no
type = private
}
namespace Synoptic {
hidden = no
ignore_on_failure = no
list = children
location =
virtual:/var/mail/virtual/synoptic:INDEX=/var/lib/dovecot/db/indexes/virtual/synoptic/%u
mailbox AKTUELL {
auto = subscribe
special_use = \All
}
mailbox Markierte {
auto = subscribe
special_use = \Flagged
}
mailbox Neue {
auto = subscribe
}
mailbox Rechnungen {
auto = subscribe
}
mailbox WICHTIG {
auto = subscribe
}
prefix = Synoptic/
separator = /
subscriptions = no
}
namespace Tulln {
list = children
location =
maildir:/var/mail/public/tulln:SUBSCRIPTIONS=subscriptions/%u:LAYOUT=fs:INDEXPVT=/var/lib/dovecot/db/indexes/public/tulln/%u
mailbox Familie {
auto = subscribe
}
mailbox Versicherungen {
auto = subscribe
}
mailbox familyextracard {
auto = subscribe
}
prefix = Tulln/
separator = /
subscriptions = no
type = public
}
namespace inbox {
hidden = no
inbox = yes
location = mailbox "Deleted Messages" {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox Drafts {
auto = no
special_use = \Drafts
}
mailbox Entwürfe {
auto = no
special_use = \Drafts
}
mailbox "Gelöschte Elemente" {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox "Gelöschte Objekte" {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox Gesendet {
auto = no
special_use = \Sent
}
mailbox "Gesendete Elemente" {
auto = no
special_use = \Sent
}
mailbox "Gesendete Objekte" {
auto = no
special_use = \Sent
}
mailbox Junk {
auto = subscribe
autoexpunge = 30 days
special_use = \Junk
}
mailbox Mistkübel {
auto = no
autoexpunge = 30 days
special_use = \Trash
}
mailbox 

Re: Sieve redirect does not collect the sendmail child process correctly (Dovecot 2.3.4.1, Pigeonhole 0.5.4)

[Stephan Bosch via dovecot]

On 02/11/2019 03:05, Delian Krustev via dovecot wrote:

Hi,

I got this simple test Sieve config:

#
# Sieve Filter
# Generated by Ingo (http://www.horde.org/apps/ingo/) (11/01/2019, 10:46:06 PM)

# Forwards
if true {
 redirect "krus...@krustev.net";
}
# Forward Keep Action
if true {
 keep;
 stop;
}
#


Which used to work fine with:

   # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
   # Pigeonhole version 0.4.16 (fed8554)
   # OS: Linux 4.9.0-8-amd64 x86_64 Debian 9.11

Then I've tried migrating to:

   # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf
   # Pigeonhole version 0.5.4 ()
   # OS: Linux 4.19.0-6-amd64 x86_64 Debian 10.1

And started getting the following errors:

   # cat .dovecot.sieve.log
   sieve: info: started log at Nov 01 22:55:50.
   error: msgid=<4221584.AlbrL4BjWX@home>: failed to redirect message to 
: Failed to execute sendmail (temporary failure).


I think we already know about this issue (tracked as DOP-942). It 
apparently happens because qmail masks the SIGCHLD signal while Dovecot 
doesn't unmask it before waiting for children.


Regards,

Stephan.



A google search lead me to:

   https://bugs.archlinux.org/task/56933

which blames systemd service changes:

   NoNewPrivileges=true

, however I'm not running dovecot via systemd but use DJB's daemontools for
service supervision .

The page above suggested to change from:

   sendmail_path = /usr/sbin/sendmail

to

   submission_host = localhost

in "conf.d/15-lda.conf", which worked fine.
But since I don't have other problems with "sendmail" execution I've decided
to debug this further.

I have dovecot integrated with Qmail:

# apt-cache policy qmail
qmail:
   Installed: 1.06-6+b1
   Candidate: 1.06-6+b1
   Version table:
  *** 1.06-6+b1 900
 900 http://deb.debian.org/debian buster/main amd64 Packages

so sendmail's implementation comes from Qmail:

   # ls -al `which sendmail`
   lrwxrwxrwx 1 root root 14 Feb 23  2015 /usr/sbin/sendmail -> qmail-sendmail


I've strace-d the qmail-lspawn process which executes dovecot's LDA, which
processes the Sieve rules and executes "sendmail".
The sendmail process executes and returns successfully, the mail is delivered,
however the Sieve code does not wait for the child process correctly and tries
to kill the already exited process after 30 seconds timeout.

Here are the problematic Sieve syscalls:

22:55:15.283482 execve("/usr/lib/dovecot/deliver", ["/usr/lib/dovecot/deliver", "-e", 
"-a", "tes...@krustev.net"], 0x5620b4330c40 /* 61 vars */) = 0
...
22:55:15.286845 pipe([5, 6])= 0
22:55:15.286870 fcntl(5, F_GETFL)   = 0 (flags O_RDONLY)
22:55:15.286889 fcntl(5, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
22:55:15.286909 fcntl(6, F_GETFL)   = 0x1 (flags O_WRONLY)
22:55:15.286928 fcntl(6, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
22:55:15.286947 fcntl(5, F_GETFD)   = 0
22:55:15.286966 fcntl(5, F_SETFD, FD_CLOEXEC) = 0
22:55:15.286985 fcntl(6, F_GETFD)   = 0
22:55:15.287003 fcntl(6, F_SETFD, FD_CLOEXEC) = 0
22:55:15.287024 epoll_create(128)   = 7
22:55:15.287043 fcntl(7, F_GETFD)   = 0
22:55:15.287062 fcntl(7, F_SETFD, FD_CLOEXEC) = 0
22:55:15.287084 epoll_ctl(7, EPOLL_CTL_ADD, 5, 
{EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=47201248, u64=94699781110752}}) = 0
...
22:55:15.294625 epoll_ctl(7, EPOLL_CTL_DEL, 5, 0x7ffd59156b8c) = 0
22:55:15.294647 epoll_create(128)   = 12
22:55:15.294668 fcntl(12, F_GETFD)  = 0
22:55:15.294687 fcntl(12, F_SETFD, FD_CLOEXEC) = 0
22:55:15.294708 epoll_ctl(12, EPOLL_CTL_ADD, 5, 
{EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=47455536, u64=94699781365040}}) = 0

   Dovecot/Sieve will wait for this pipe FD 5 later, but no event will arrive.

22:55:15.294740 rt_sigaction(SIGCHLD, {sa_handler=0x7f15404b3640, sa_mask=[], 
sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO, sa_restorer=0x7f1540229840}, NULL, 
8) = 0
22:55:15.294763 pipe([13, 14])  = 0
22:55:15.294788 clone(child_stack=NULL, 
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, 
child_tidptr=0x7f15401ec2d0) = 6929


   The sendmail child process with PID 6929 is forked here.


22:55:15.294928 close(13)   = 0
22:55:15.294969 fcntl(14, F_GETFL)  = 0x1 (flags O_WRONLY)
22:55:15.295006 fcntl(14, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
22:55:15.295039 lseek(14, 0, SEEK_CUR)  = -1 ESPIPE (Illegal seek)
22:55:15.295065 getsockname(14, 0x7ffd59156a40, [28]) = -1 ENOTSOCK (Socket 
operation on non-socket)
22:55:15.295109 write(14, "EDITTED LONG MESSAGE DATA\n", 1461) = 1461
22:55:15.295150 close(14)   = 0
22:55:15.295175 wait4(6929, 0x562102d4fee4, WNOHANG, NULL) = 0
22:55:15.295211 epoll_wait(12, [], 1, 3) = 0


   Here's where the 30 second's timeout is requested and waited.


22:55:45.325580 kill(6929, SIGTERM) = 0
22:55:45.325769 epoll_wait(12, [], 1, 5000) = 0
22:55:50.331108 kill(6929, SIGKILL) = 0
22:55:50.332867 wait4(6929, 

Re: encrypt incoming emails with public gpg key before they are stored to maildir

[Stephan Bosch via dovecot]

On 30/10/2019 15:29, Fourhundred Thecat via dovecot wrote:

Hello,

I have asked on the postfix mailing list for a solution, how to encrypt
incoming emails with public gpg key

My original idea was to use a smtpd-milter, which would encrypt all
incoming plaintext messages of given user, using the users public gpg
key. This way, it would look as if the original sender has sent the
message encrypted.

Somebody suggested this might be better done in Dovecot, as it is
Dovecot who stores the emails in Maildir.

Does anybody have any suggestions how to implement this scheme in Dovecot ?


Something like this?

https://perot.me/encrypt-specific-incoming-emails-using-dovecot-and-sieve

Regards,

Stephan.


I am including the original email below:


when new email arrives, and it is not already encrypted, I would like to
run it through a filter, which would encrypt the message with my public
gpg key, as if the original sender has sent the email encrypted.

Why do I want to do this ? Why not ask the sender to send encrypted
messages to start with ?

Lets say my bank sends me emails. I cannot force my bank to use gpg
encryption. I am happy they use email at all, instead of paper mail.

My email server is untrusted. It can be hacked into and emails stolen.
Full disk encryption will not help, because the disk must be decrypted
during runtime.

With my scheme, all emails would be stored encrypted on my server, and
decryption key does not exist on the server (emails are decrypted on my
local client)

What would be the best way to implement this ?

Can such filter work, without ever storing plaintext email on disk ?

thanks,

Re: IMAP4 extensions for Visual Voicemail (VVM)

[Stephan Bosch via dovecot]

Op 20-10-2019 om 22:33 schreef Jean-Daniel via dovecot:



Le 20 oct. 2019 à 22:24, Mauricio Tavares via dovecot 
mailto:dovecot@dovecot.org>> a écrit :


On Sun, Oct 20, 2019 at 10:43 AM Rajesh Bansal via dovecot
mailto:dovecot@dovecot.org>> wrote:


Hi Team,



I need to develop Visual VoiceMail solution. In this solution I need 
a IMAP4 server, from which I can get a hit for each mail retrieval. 
Can anyone help me if dovecot can be used for this purpose.



 That is rather vague. Do you want to do something like ol' biff
or what we used to do with Asterix 10 years ago (get an email with the
voicemail as as attachment)?


I guess he is talking about that: 
https://www.gsma.com/newsroom/wp-content/uploads/2012/07/OMTP_VVM_Specification_1_3.pdf


I skimmed through this. It adds quite a few strange custom bits to the 
IMAP protocol. Part of this are a few new commands and METADATA items, 
which should be quite easy to implement. However, the amendments also 
include various attachment-based QUOTA limits. Dovecot doesn't support 
this and adding that is not trivial.


Also, this specification is quite old and outdated (it even mentions an 
MD5 authentication mechanism). Is this even actively used anywhere?


Regards,

Stephan.

Re: Vacation Fcc

[Stephan Bosch via dovecot]

On 11/10/2019 10:24, Gregory Heytings via dovecot wrote:


Hi list,

I use Dovecot's Pigeonhole Sieve implementation, and there is one 
little feature I'm missing:  When a vacation message is sent, I'd like 
to keep a copy of that message in my "sent" mailbox.


There is an RFC draft (draft-ietf-extra-sieve-fcc-09) for this, which 
adds a ":fcc" argument to vacation, but apparently it's not supported 
by Pigeonhole.


Actually, it is RFC already: https://tools.ietf.org/html/rfc8580 .


I have two questions:

Will ":fcc" be supported in the future?


I'd say yes, but it is not simple to implement, nor is it high up in our 
list of priorities.


In the meantime (or if supporting ":fcc" is not planned), is it 
possible to achieve the same result with the existing Sieve 
extensions?  (I can imagine a way to do this at the MTA level by 
checking the presence of the "Auto-Submitted:" header, but did not 
test it.)


I can't think of any means to do that within the Sieve language.

Regards,

Stephan.

Re: Panic: file smtp-client-connection.c: line 1212 (smtp_client_connection_established): assertion failed: (!conn->connect_succeeded)

[Stephan Bosch via dovecot]

On 11/10/2019 10:06, Hativ via dovecot wrote:

Hello Aki,

I have this problem just with 2.3.8, my self-compiled 2.3.3 works 
fine. I have previously tried to update from 2.3.3 to higher versions 
(possibly 2.3.5 or so), but always had this error, which is why I am 
always back to 2.3.3.


This bug was already known. It is tracked internally as DOV-3600. Looks 
like it will be put through somewhat soon.


Regards,

Stephan.


Configuration is exactly the same.

Here my output from "doveconf -n":

# 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.8 (b7b03ba2)
# OS: Linux 4.15.0-65-generic x86_64 Ubuntu 18.04.3 LTS
# Hostname: srv2
auth_mechanisms = plain login
default_login_user = dovecot-login
default_vsz_limit = 1 G
dict {
  quota = mysql:/etc/dovecot/dovecot-quota.conf.ext
}
doveadm_password = # hidden, use -P to show it
doveadm_port = 12345
haproxy_trusted_networks = 10.201.29.33 fec0:4f8:1c1c:3440::1 
10.46.163.91 fec0:4f8:1c17:4548::1 10.216.147.248 fec0:4f9:c010:bec::1

hostname = mda1-2.example.com
imap_capability = +XLIST
imap_id_send =
imap_idle_notify_interval = 30 mins
listen = 10.46.163.91, fec0:4f8:1c17:4548::1
lmtp_rcpt_check_quota = yes
mail_attribute_dict = file:%h/dovecot-attributes
mail_debug = yes
mail_gid = dovecot-mail
mail_home = /srv/mail/%d/%n
mail_location = sdbox:~/
mail_max_userip_connections = 20
mail_plugins = " acl mail_crypt mail_log notify quota quota_clone 
replication"

mail_uid = dovecot-mail
mail_vsize_bg_after_count = 100
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapsieve vnd.dovecot.imapsieve

namespace INBOX {
  inbox = yes
  location =
  mailbox Drafts {
auto = subscribe
special_use = \Drafts
  }
  mailbox Junk {
auto = subscribe
special_use = \Junk
  }
  mailbox Sent {
auto = subscribe
special_use = \Sent
  }
  mailbox Trash {
auto = subscribe
special_use = \Trash
  }
  prefix =
}
namespace Shared {
  list = children
  location = sdbox:%%h/:INDEXPVT=~/shares/%%u
  prefix = Shared/%%u/
  subscriptions = yes
  type = shared
}
passdb {
  args = /etc/dovecot/dovecot-auth.conf.ext
  driver = sql
}
plugin {
  acl = vfile
  acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
  imapsieve_mailbox1_before = file:/etc/dovecot/sieve/report-spam.sieve
  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox2_before = file:/etc/dovecot/sieve/report-ham.sieve
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_name = INBOX
  mail_crypt_curve = secp521r1
  mail_crypt_save_version = 0
  mail_log_events = copy delete expunge flag_change mailbox_create 
mailbox_delete mailbox_rename undelete save

  mail_log_fields = uid box msgid size from subject
  mail_replica = tcps:mda1-1.example.com
  quota = count
  quota_clone_dict = proxy::quota
  quota_rule = *:bytes=0
  quota_vsizes = yes
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_after = /etc/dovecot/sieve-after
  sieve_before = /etc/dovecot/sieve-before
  sieve_global_extensions = +vnd.dovecot.pipe
  sieve_pipe_bin_dir = /usr/bin/
  sieve_plugins = sieve_extprograms sieve_imapsieve
}
postmaster_address = postmas...@example.com 


protocols = imap lmtp sieve submission
service aggregator {
  fifo_listener replication-notify-fifo {
user = $mail_uid
  }
  unix_listener replication-notify {
user = $mail_uid
  }
}
service doveadm {
  inet_listener {
port = 12345
ssl = yes
  }
}
service imap-login {
  inet_listener imap {
port = 0
  }
  inet_listener imaps {
haproxy = yes
  }
}
service lmtp {
  inet_listener lmtp {
address = $listen
port = 24
  }
}
service managesieve-login {
  inet_listener sieve {
haproxy = yes
ssl = yes
  }
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
mode = 0600
user = $mail_uid
  }
}
service submission-login {
  inet_listener submission {
haproxy = yes
port = 465
ssl = yes
  }
}
ssl = required
ssl_alt_cert = ssl_cipher_list = 
TLS-CHACHA20-POLY1305-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:TLS-AES-256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:TLS-AES-128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256

ssl_client_ca_file = /etc/ssl/certs/ca-certificates.crt
ssl_curve_list = X25519:secp521r1:secp384r1
ssl_key = # hidden, use -P to show it
ssl_min_protocol = TLSv1.2
ssl_options = no_ticket
ssl_prefer_server_ciphers = yes
submission_client_workarounds = whitespace-before-path
submission_max_mail_size = 5 k
submission_relay_host = mta2.example.com
submission_relay_ssl = starttls
submission_relay_trusted = yes
userdb {
  driver = prefetch
}
userdb 

Re: Sieve replication - Debian 9.11 - Dovecot 2.2.7 - script changes not replicated

[Stephan Bosch via dovecot]

On 09/10/2019 18:07, Daniel Botting via dovecot wrote:

Hi,

I am trying to setup Sieve replication, it is almost working:

1/ Create a script - sync'd across

2/ Activate/de-activate a script - change sync'd

3/ Delete a script - change sync'd

But if I make a change to a script it is not replicated, unless I 
de-activate and activate again.


I am seeing in the logs the below during this process 
(de-activating/activating):


Oct  9 17:01:39 hostname dovecot: dsync-local(a_user): Debug: 
doveadm-sieve: Iterating Sieve mailbox attribute: 
vendor/vendor.dovecot/pvt/server/sieve/files/0910-active-1640


Oct  9 17:01:39 hostname dovecot: dsync-local(a_user): Debug: 
doveadm-sieve: Value missing for key 
`vendor/vendor.dovecot/pvt/server/sieve/default' (last change: 
2019-10-09 17:01:37)




Sieve replication bugs were solved recently. You're running a very old 
version. Try https://repo.dovecot.org/ instead.


Regards,

Stephan.


# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-3-amd64 x86_64 Debian 9.11
auth_debug = yes
auth_debug_passwords = yes
auth_verbose = yes
default_vsz_limit = 0
doveadm_password =  # hidden, use -P to show it
first_valid_gid = 8
first_valid_uid = 8
last_valid_gid = 8
last_valid_uid = 8
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_debug = yes
mail_gid = 8
mail_location = maildir:~/Maildir
mail_plugins = " notify replication"
mail_uid = 8
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext editheader imapflags

namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
}
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  mail_replica = tcps:replication_server_url:port_number
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_extensions = +editheader +imapflags
}
protocols = " imap sieve pop3"
replication_max_conns = 4
service aggregator {
  fifo_listener replication-notify-fifo {
    user = mail
  }
  unix_listener replication-notify {
    user = mail
  }
}
service auth {
  unix_listener /var/run/dovecot-exim-bridge {
    mode = 0660
    user = Debian-exim
  }
}
service doveadm {
  inet_listener {
    port = 12345
    ssl = yes
  }
}
service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_limit = 512
  process_min_avail = 4
  service_count = 1
}
service imap {
  process_limit = 1024
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  process_min_avail = 1
  service_count = 8
  vsz_limit = 256 M
}
service managesieve {
  process_limit = 1024
}
service replicator {
  process_min_avail = 1
  unix_listener replicator-doveadm {
    mode = 0666
  }
}
ssl = required
ssl_cert = 

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch via dovecot]

On 10/10/2019 04:05, Larry Rosenman via dovecot wrote:

Is there a way to get this as a patchset against 2.3.8?



https://github.com/stephanbosch/dovecot-core/tree/fix-solr-xml-parser-2.3.8

This hasn't seen any QA yet, so handle with care.

Regards,

Stephan.



On Tue, Oct 8, 2019 at 12:03 PM Stephan Bosch via dovecot 
mailto:dovecot@dovecot.org>> wrote:




On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:
>
>
> On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:
>> Hi,
>>
>>> Am 23.09.2019 um 09:36 schrieb Filip Hanes
mailto:filip.ha...@ringier.sk>
>>> <mailto:filip.ha...@ringier.sk <mailto:filip.ha...@ringier.sk>>>:
>>>
>>>
>>> št 19. 9. 2019 o 15:30 Fabian via dovecot mailto:dovecot@dovecot.org>
>>> <mailto:dovecot@dovecot.org <mailto:dovecot@dovecot.org>>>
napísal(a):
>>>
>>>     Are there any other hints or tips regarding this „invalid uid
>>>     ‚0‘"-message?
>>>
>>> from fts-solr source:
>>> case SOLR_XML_CONTENT_STATE_UID:
>>>
if (uint32_parse(str, len, >uid) < 0 || ctx->uid == 0) {
>>>
i_error("fts_solr: received invalid uid '%s'", t_strndup(str, 
len));

>>>
>>> It means that plugin tries to parse '0' when in state of
parsing uid
>>> in xml response from solr or there is a bug in xml response
parser.
>>> I would enable rawlog_dir in config and look there for unusual
>>> response or extra tags near usual uid in xml response. For
example:
>>> |plugin { fts = solr fts_solr = url=...
>>> rawlog_dir=/var/log/fts-solr/ }|
>>>
>>
>> Thanks for this tip! The rawlog_dir option is not yet supported in
>> our version 2.3.4. What we could provide now would be the
>> XML-Response from Solr. The corresponding search query sometimes
>> causes this error. But Solr's answer seems to always remain the
same
>> - in my opinion this already indicates a bug of Dovecot?
>>
>> The XML response can be downloaded here:
>> https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr
>
> Based on the XML response above, I investigated this problem
> thoroughly and determined that this is a pretty severe bug in
the Solr
> XML response parsing code. This occurs only when the response is
> rather large and the boundary between two read chunks falls in the
> middle of a numeric value (that happens to end in '0').
>
> Tracking internally as DOP-1470.
>
> I think this may also explain some of the weird reports about
invalid
> UIDs that we've seen in recent history on this mailing list.

Preliminary fix is here:
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser

It's big due to some code restructuring for making a proper unit
test. I
am not sure when this will be in a release.

Regards,

Stephan.



--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com 
<mailto:larry...@gmail.com>

US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch via dovecot]

On 08/10/2019 02:52, Stephan Bosch via dovecot wrote:



On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:

Hi,

Am 23.09.2019 um 09:36 schrieb Filip Hanes <mailto:filip.ha...@ringier.sk>>:



št 19. 9. 2019 o 15:30 Fabian via dovecot <mailto:dovecot@dovecot.org>> napísal(a):


    Are there any other hints or tips regarding this „invalid uid
    ‚0‘"-message?

from fts-solr source:
case SOLR_XML_CONTENT_STATE_UID:
if (uint32_parse(str, len, >uid) < 0 || ctx->uid == 0) {
i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len)); 

It means that plugin tries to parse '0' when in state of parsing uid 
in xml response from solr or there is a bug in xml response parser.
I would enable rawlog_dir in config and look there for unusual 
response or extra tags near usual uid in xml response. For example:
|plugin { fts = solr fts_solr = url=... 
rawlog_dir=/var/log/fts-solr/ }|




Thanks for this tip! The rawlog_dir option is not yet supported in 
our version 2.3.4. What we could provide now would be the 
XML-Response from Solr. The corresponding search query sometimes 
causes this error. But Solr's answer seems to always remain the same 
- in my opinion this already indicates a bug of Dovecot?


The XML response can be downloaded here: 
https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr


Based on the XML response above, I investigated this problem 
thoroughly and determined that this is a pretty severe bug in the Solr 
XML response parsing code. This occurs only when the response is 
rather large and the boundary between two read chunks falls in the 
middle of a numeric value (that happens to end in '0').


Tracking internally as DOP-1470.

I think this may also explain some of the weird reports about invalid 
UIDs that we've seen in recent history on this mailing list.


Preliminary fix is here: 
https://github.com/stephanbosch/dovecot-core/commits/fix-solr-xml-parser


It's big due to some code restructuring for making a proper unit test. I 
am not sure when this will be in a release.


Regards,

Stephan.

Re: fts_solr: Error: fts_solr: received invalid uid '0'

[Stephan Bosch via dovecot]

On 07/10/2019 15:13, Fabian Kuran via dovecot wrote:

Hi,

Am 23.09.2019 um 09:36 schrieb Filip Hanes >:



št 19. 9. 2019 o 15:30 Fabian via dovecot > napísal(a):


Are there any other hints or tips regarding this „invalid uid
‚0‘"-message?

from fts-solr source:
case SOLR_XML_CONTENT_STATE_UID:
if (uint32_parse(str, len, >uid) < 0 || ctx->uid == 0) {
i_error("fts_solr: received invalid uid '%s'", t_strndup(str, len));
It means that plugin tries to parse '0' when in state of parsing uid 
in xml response from solr or there is a bug in xml response parser.
I would enable rawlog_dir in config and look there for unusual 
response or extra tags near usual uid in xml response. For example:

|plugin { fts = solr fts_solr = url=... rawlog_dir=/var/log/fts-solr/ }|



Thanks for this tip! The rawlog_dir option is not yet supported in our 
version 2.3.4. What we could provide now would be the XML-Response 
from Solr. The corresponding search query sometimes causes this error. 
But Solr's answer seems to always remain the same - in my opinion this 
already indicates a bug of Dovecot?


The XML response can be downloaded here: 
https://owncloud.eideo.de/index.php/s/8oyJaf5HBK6zdbr


Based on the XML response above, I investigated this problem thoroughly 
and determined that this is a pretty severe bug in the Solr XML response 
parsing code. This occurs only when the response is rather large and the 
boundary between two read chunks falls in the middle of a numeric value 
(that happens to end in '0').


Tracking internally as DOP-1470.

I think this may also explain some of the weird reports about invalid 
UIDs that we've seen in recent history on this mailing list.


Regards,

Stephan.

Re: Sieve redirect is broken in 2.3.7.2 - signal 11

[Stephan Bosch via dovecot]

}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = 
{ip6 = {__in6_union = {
  __s6_addr = '\000' , __s6_addr16 = 
{0, 0, 0, 0, 0, 0, 0, 0}, __s6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr 
= 0}}}, local_port = 0,
  remote_port = 0, userdb_fields = 0x0, flags_override_add = 
(unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 
false, debug = false,

  conn_secured = false, conn_ssl_secured = false}
    user_source = 
    rcpt_to_source = 
    process_euid = 
    stderr_rejection = false
    ret = 
    c = 

On 2019. 10. 06. 23:28, Stephan Bosch via dovecot wrote:



On 06/10/2019 23:14, Demonhost wrote:

I think alpine does not have that

https://pkgs.alpinelinux.org/packages?name=dovecot*=v3.8=main=x86_64 



https://git.alpinelinux.org/aports/tree/main/dovecot/APKBUILD?h=3.8-stable 



What else would help? strace?


Do you have the opportunity to build Dovecot yourself, thereby 
including the debug symbols?


Regards,

Stephan.


On 2019. 10. 06. 23:05, Stephan Bosch via dovecot wrote:



On 06/10/2019 23:03, Demonhost wrote:

Hi Stephan,

I managed to get GDB, I hope this one helps. If not I can send the 
config too, but exactly same config works with 2.3.6.


You don't have the debug symbols for Dovecot installed. Usually, 
your system will have a separate package for that. On Debian, it is 
called dovecot-dbg.




92b9d3412dc8:/$ gdb /usr/libexec/dovecot/dovecot-lda -f 
root@ -d test@x

GNU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show 
copying"

and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Warning: //test@x: No such file or directory.
Reading symbols from /usr/libexec/dovecot/dovecot-lda...(no 
debugging symbols found)...done.

//root@: No such file or directory.
(gdb) r -f root@ -d test@x < message
Starting program: /usr/libexec/dovecot/dovecot-lda -f 
root@ -d test@x < message

process 359 is executing new program: /usr/bin/doveconf
process 359 is executing new program: 
/usr/libexec/dovecot/dovecot-lda


Program received signal SIGSEGV, Segmentation fault.
0x775b74f7 in p_strdup () from 
/usr/lib/dovecot/libdovecot.so.0

(gdb) bt full
#0  0x775b74f7 in p_strdup () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#1  0x7757dd1a in ssl_iostream_settings_init_from () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#2  0x77522545 in smtp_submit_session_init () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#3  0x7752265a in smtp_submit_init_simple () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#4  0x7687e923 in ?? () from 
/usr/lib/dovecot/lib90_sieve_plugin.so

No symbol table info available.
#5  0x765fe761 in sieve_smtp_start () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#6  0x765fe869 in sieve_smtp_start_single () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#7  0x7661c71e in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#8  0x7660e8a8 in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#9  0x766100d1 in sieve_result_execute () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#10 0x7661e9fc in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#11 0x7661f638 in sieve_multiscript_run () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#12 0x7687f7cb in ?? () from 
/usr/lib/dovecot/lib90_sieve_plugin.so

No symbol table info available.
#13 0x77b69960 in mail_deliver () from 
/usr/lib/dovecot/libdovecot-lda.so.0

No symbol table info available.
#14 0x6fd2 in main ()
No symbol table info available.

On 2019. 10. 04. 14:42, Stephan Bosch wrote:



Op 4-10-2019 om 12:10 schreef Demonhost via dovecot:

Hi,

If we use sieve redirect under dovecot 2.3.7.2 we end up with

Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 0605F207B0F36: to=, 
relay=127.0.0.1[127.0.0.1]:10024, delay=1.5

Re: Sieve redirect is broken in 2.3.7.2 - signal 11

[Stephan Bosch via dovecot]

On 06/10/2019 23:14, Demonhost wrote:

I think alpine does not have that

https://pkgs.alpinelinux.org/packages?name=dovecot*=v3.8=main=x86_64 



https://git.alpinelinux.org/aports/tree/main/dovecot/APKBUILD?h=3.8-stable 



What else would help? strace?


Do you have the opportunity to build Dovecot yourself, thereby including 
the debug symbols?


Regards,

Stephan.


On 2019. 10. 06. 23:05, Stephan Bosch via dovecot wrote:



On 06/10/2019 23:03, Demonhost wrote:

Hi Stephan,

I managed to get GDB, I hope this one helps. If not I can send the 
config too, but exactly same config works with 2.3.6.


You don't have the debug symbols for Dovecot installed. Usually, your 
system will have a separate package for that. On Debian, it is called 
dovecot-dbg.




92b9d3412dc8:/$ gdb /usr/libexec/dovecot/dovecot-lda -f 
root@ -d test@x

GNU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
<http://gnu.org/licenses/gpl.html>

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show 
copying"

and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Warning: //test@x: No such file or directory.
Reading symbols from /usr/libexec/dovecot/dovecot-lda...(no 
debugging symbols found)...done.

//root@: No such file or directory.
(gdb) r -f root@ -d test@x < message
Starting program: /usr/libexec/dovecot/dovecot-lda -f root@ 
-d test@x < message

process 359 is executing new program: /usr/bin/doveconf
process 359 is executing new program: /usr/libexec/dovecot/dovecot-lda

Program received signal SIGSEGV, Segmentation fault.
0x775b74f7 in p_strdup () from /usr/lib/dovecot/libdovecot.so.0
(gdb) bt full
#0  0x775b74f7 in p_strdup () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#1  0x7757dd1a in ssl_iostream_settings_init_from () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#2  0x77522545 in smtp_submit_session_init () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#3  0x7752265a in smtp_submit_init_simple () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#4  0x7687e923 in ?? () from 
/usr/lib/dovecot/lib90_sieve_plugin.so

No symbol table info available.
#5  0x765fe761 in sieve_smtp_start () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#6  0x765fe869 in sieve_smtp_start_single () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#7  0x7661c71e in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#8  0x7660e8a8 in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#9  0x766100d1 in sieve_result_execute () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#10 0x7661e9fc in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#11 0x7661f638 in sieve_multiscript_run () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#12 0x7687f7cb in ?? () from 
/usr/lib/dovecot/lib90_sieve_plugin.so

No symbol table info available.
#13 0x77b69960 in mail_deliver () from 
/usr/lib/dovecot/libdovecot-lda.so.0

No symbol table info available.
#14 0x6fd2 in main ()
No symbol table info available.

On 2019. 10. 04. 14:42, Stephan Bosch wrote:



Op 4-10-2019 om 12:10 schreef Demonhost via dovecot:

Hi,

If we use sieve redirect under dovecot 2.3.7.2 we end up with

Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 0605F207B0F36: to=, 
relay=127.0.0.1[127.0.0.1]:10024, delay=1.5, delays=0.36/0/0/1.1, 
dsn=2.0.0, status=sent (250 2.0.0 from 
MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 6FC89207B0F38)
Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 6FC89207B0F38: to=, 
relay=dovecot, delay=0.31, delays=0.05/0/0/0.26, dsn=4.3.0, 
status=deferred (Command died with signal 11: 
"/usr/libexec/dovecot/dovecot-lda")


Using docker with official alpine image.

/ # cat /etc/alpine-release
3.8.4

/ # apk info -v | grep dovecot
WARNING: Ignoring APKINDEX.adfa7ceb.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.efaa1f73.tar.gz: No such file or directory
dov

Re: Sieve redirect is broken in 2.3.7.2 - signal 11

[Stephan Bosch via dovecot]

On 06/10/2019 23:03, Demonhost wrote:

Hi Stephan,

I managed to get GDB, I hope this one helps. If not I can send the 
config too, but exactly same config works with 2.3.6.


You don't have the debug symbols for Dovecot installed. Usually, your 
system will have a separate package for that. On Debian, it is called 
dovecot-dbg.




92b9d3412dc8:/$ gdb /usr/libexec/dovecot/dovecot-lda -f root@ 
-d test@x

GNU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 


This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show 
copying"

and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Warning: //test@x: No such file or directory.
Reading symbols from /usr/libexec/dovecot/dovecot-lda...(no debugging 
symbols found)...done.

//root@: No such file or directory.
(gdb) r -f root@ -d test@x < message
Starting program: /usr/libexec/dovecot/dovecot-lda -f root@ -d 
test@x < message

process 359 is executing new program: /usr/bin/doveconf
process 359 is executing new program: /usr/libexec/dovecot/dovecot-lda

Program received signal SIGSEGV, Segmentation fault.
0x775b74f7 in p_strdup () from /usr/lib/dovecot/libdovecot.so.0
(gdb) bt full
#0  0x775b74f7 in p_strdup () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#1  0x7757dd1a in ssl_iostream_settings_init_from () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#2  0x77522545 in smtp_submit_session_init () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#3  0x7752265a in smtp_submit_init_simple () from 
/usr/lib/dovecot/libdovecot.so.0

No symbol table info available.
#4  0x7687e923 in ?? () from 
/usr/lib/dovecot/lib90_sieve_plugin.so

No symbol table info available.
#5  0x765fe761 in sieve_smtp_start () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#6  0x765fe869 in sieve_smtp_start_single () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#7  0x7661c71e in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#8  0x7660e8a8 in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#9  0x766100d1 in sieve_result_execute () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#10 0x7661e9fc in ?? () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#11 0x7661f638 in sieve_multiscript_run () from 
/usr/lib/dovecot/libdovecot-sieve.so.0

No symbol table info available.
#12 0x7687f7cb in ?? () from 
/usr/lib/dovecot/lib90_sieve_plugin.so

No symbol table info available.
#13 0x77b69960 in mail_deliver () from 
/usr/lib/dovecot/libdovecot-lda.so.0

No symbol table info available.
#14 0x6fd2 in main ()
No symbol table info available.

On 2019. 10. 04. 14:42, Stephan Bosch wrote:



Op 4-10-2019 om 12:10 schreef Demonhost via dovecot:

Hi,

If we use sieve redirect under dovecot 2.3.7.2 we end up with

Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 0605F207B0F36: to=, 
relay=127.0.0.1[127.0.0.1]:10024, delay=1.5, delays=0.36/0/0/1.1, 
dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 
250 2.0.0 Ok: queued as 6FC89207B0F38)
Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 6FC89207B0F38: to=, 
relay=dovecot, delay=0.31, delays=0.05/0/0/0.26, dsn=4.3.0, 
status=deferred (Command died with signal 11: 
"/usr/libexec/dovecot/dovecot-lda")


Using docker with official alpine image.

/ # cat /etc/alpine-release
3.8.4

/ # apk info -v | grep dovecot
WARNING: Ignoring APKINDEX.adfa7ceb.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.efaa1f73.tar.gz: No such file or directory
dovecot-2.3.7.2-r0
dovecot-fts-solr-2.3.7.2-r0
dovecot-ldap-2.3.7.2-r0
dovecot-pigeonhole-plugin-2.3.7.2-r0
dovecot-pigeonhole-plugin-ldap-2.3.7.2-r0

Sieve rule is:
# rule:[test]
if header :contains "subject" "test"
{
    redirect "testem...@gmail.com";
}

In host's dmesg we found the following:
[213184.882341] traps: dovecot-lda[6128] general protection 
ip:7ff6cf8144f7 sp:7fff5f800b30 error:0 in 
libdovecot.so.0.0.0[7ff6cf724000+36]


Sorry, not knowledgeable enough on how to do the GBD debug.
-
We went back to 

Re: Dovecot proxying to some backend using LOGIN proxy_mech

[Stephan Bosch via dovecot]

On 21/09/2019 12:32, Sébastien Riccio via dovecot wrote:


Hellolist,

I am currently testing a setup for a PoC wit this configuration.

- 1 x Frontend dovecot for proxying IMAP/POP3/LMTP/ManageSieve/Submission

- 2 x Backend dovecot with local mail storage

The frontend does the user authentification and communicate with the 
backends using a master password


The fronted accepts PLAIN and LOGIN auth mechanisms and talk with 
backends using PLAIN auth mechanism (with the master password).


This works very well, it's cool.

Then, I've added to this setup an existing mail server (running 
another mailserver software) as a 3rd backend, and would like the 
frontend to also proxy for this, but this time not using master password.


This is working for IMAP/POP3 when it uses PLAIN auth, but not for 
Submission. The backend server in this case only accepts LOGIN auth 
and not PLAIN on port 587.


So I found out there is a field in the passdb we can add (proxy_mech), 
and I've set it to LOGIN.


The frontend seems to now use LOGIN auth mech to talk to this 3rd 
backend, but the authentication still fails.


According to the backend logs, tt seems dovecot proxy is now using 
LOGIN auth, but sends garbage as username and/or password.


Also IMAP that was working when using PLAIN doesn't work anymore with 
LOGIN.




The Dovecot proxy is not strictly meant for proxying towards something 
other than Dovecot and you seem to have stumbled upon a bug/design 
flaw/protocol deviation in the client-side SASL LOGIN mechanism. An 
initial SASL response is sent (empty) even though the LOGIN mechanism 
doesn't define one.


Regards,

Stephan.




Might this be a bug or this setup shouldn't be supported at all ?

Passdb MySQL table on frontend:

mysql> describe users;

+-+--+--+-+-+---+

| Field   | Type | Null | Key | Default | Extra |

+-+--+--+-+-+---+

| user    | varchar(255) | NO   | PRI | NULL    |   |

| domain  | varchar(255) | NO   | PRI | NULL    |   |

| password    | varchar(255) | YES  | | NULL    |   |

| displayname | varchar(255) | YES  | | NULL    |   |

| host    | varchar(16)  | NO   | | NULL    |   |

| usemaster   | tinyint(1)   | NO   | | 1   |   |

| proxy_mech  | varchar(20)  | YES  | | NULL    |   |

| starttls    | varchar(20)  | YES  | | NULL    |   |

+-+--+--+-+-+---+

mailstore-01/02 are dovecot backends

otherhost is the other mailserver I also would like to proxy through 
dovecot proxy


+-+-+-++--+---++--+

| user    | domain  | password    
| displayname    | host | usemaster | proxy_mech | starttls |


+-+-+-++--+---++--+

| user1   | somedomain.com  | {SHA512-CRYPT}   
| Joe Black | mailstore-01 | 1 | NULL   | NULL |


| user2   | otherdomain.com | {SHA512-CRYPT}   
| Mr. Smith | mailstore-02 | 1 | NULL   | NULL |


| user3   | area13.com  | NULL    
| Donald Duck    | otherhost    | 0 | LOGIN  | any-cert |


| user4   | area13.com  | NULL    
| Thanks Obama   | otherhost    | 0 | LOGIN  | any-cert |


+-+-+-++--+---++--+

Passdb query :

password_query = \

  SELECT concat(user, '@', domain) AS user, password, host, 'Y' AS 
proxy, \


    IF(usemaster is TRUE, 'supermasteruserl', NULL) as master, \

    IF(usemaster is TRUE, 'supermasterpassword', pass) as pass, \

    IF(usemaster is FALSE, 'Y', NULL) as nopassword, \

    proxy_mech, \

    starttls \

    FROM users WHERE user = '%n' AND domain = '%d'

Some questions I'm asking myself:

Is proxying using LOGIN proxy_mech supported at all to auth on backends ?

Is dovecot proxying to "otherhost" supposed to decode original client 
auth (using PLAIN) and translate it to LOGIN format to auth with 
"otherhost" ?


# otherhost port 587 log

Without specifying proxy_mech

[2019.09.19] 09:27:22.167 [192.168.60.220][3070269] cmd: EHLO 
dovecotproxy.local.domain


[2019.09.19] 09:27:22.167 [192.168.60.220][3070269] rsp: 
250-dovecotproxy.local.domain Hello [192.168.60.220]250-SIZE 
52428800250-AUTH LOGIN CRAM-MD5250-STARTTLS250-8BITMIME250-DSN250 OK


[2019.09.19] 09:27:22.167 [192.168.60.220][3070269] cmd: AUTH PLAIN 



[2019.09.19] 09:27:22.167 [192.168.60.220][3070269] rsp: 504 
Unrecognized authentication type.


When using proxy_mech LOGIN

[2019.09.20] 08:24:45.201 [192.168.60.220][24148464] 

Re: imapsieve administrator scripts are not executed in the order they are defined

[Stephan Bosch via dovecot]

On 05/10/2019 00:43, frankagainstthemachine--- via dovecot wrote:

Dear dovecot developers,

I have an issue with the Pigeonhole IMAPSieve Plugin and the order in which
administrator scripts are executed.  Although I cannot find anything about the
order in which the scripts are executed, I would expect they are executed in
the order they are defined: the one defined by imapsieve_mailbox1_* before the
one defined by imapsieve_mailbox2_* in case both match.

I defined several administrator scripts; among them one that should be executed
when copying a mail to mailbox "Inbox", and one that should be executed when
copying a mail from a mailbox "*.Spam" to any other mailbox.  No matter in
which order I define the two scripts, the one related to "Inbox" is always
executed first when I move a mail from "blah.Spam" to "Inbox", see the snipped
from my logs below.

I guess there might be a bug in file imap-sieve-storage.c, function
imap_sieve_mailbox_rules_match(...): The comment in this function states
"Insert sorted by rule index", but in case the index of the rule to be
inserted is higher than any index already in the "rules" array, the rule is
inserted in position 0 instead of being appended to the end, because
"insert_idx" is initialized with 0.

Since the rule for "*.Spam" is a pattern rule, it is the first rule to be
inserted into "rules" (in imap_sieve_mailbox_rules_match_patterns(...)).
Afterwards the rule for "Inbox" is always inserted at position 0 (in
imap_sieve_mailbox_rules_match(...)), no matter if its rule index is higher or
lower than the index of the other rule.

The following change could fix this:
  - unsigned int insert_idx = 0;
  + unsigned int insert_idx = array_count(rules);


Confirmed. Tracking internally as DOP-1465.

Regards,

Stephan.


Best regards,
   Frank


- log -
[...]
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: imapsieve: mailbox INBOX: MOVE event
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: Pigeonhole version 0.5.7.2 (7372921a) initializing
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: Sieve Extprograms plugin for Pigeonhole version 0.5.7.2
(7372921a) loaded
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: imapsieve: Static mailbox rule [1]: mailbox=`*.Spam' from=`*'
causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/spamassassin-learn-
spam.sieve' after=(none)
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: imapsieve: Static mailbox rule [2]: mailbox=`*' from=`*.Spam'
causes=(COPY) => before=`file:/usr/lib/dovecot/sieve/spamassassin-learn-
ham.sieve' after=(none)
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: imapsieve: Static mailbox rule [3]: mailbox=`INBOX' from=`*'
causes=(COPY APPEND) => before=`file:~/.dovecot.sieve' after=(none)
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: imapsieve: Matched static mailbox rule [2]
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: imapsieve: Matched static mailbox rule [3]
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: file storage: Using Sieve script path: /var/spool/mail/
user/.dovecot.sieve
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: file script: Opened script `.dovecot' from `/var/spool/mail/
user/.dovecot.sieve'
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: file storage: Using Sieve script path: /usr/lib/dovecot/sieve/
spamassassin-learn-ham.sieve
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: file script: Opened script `spamassassin-learn-ham' from `/usr/
lib/dovecot/sieve/spamassassin-learn-ham.sieve'
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: Opening script 1 of 2 from `/var/spool/mail/user/.dovecot.sieve'
Oct  4 23:51:12 somehost dovecot[22918]: imap(user)<22927>:
Debug: sieve: Loading script /var/spool/mail/user/.dovecot.sieve
[...]

- dovecot -n -
# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 (7372921a)
# OS: Linux 5.1.18-gentoo x86_64 Gentoo Base System release 2.6 ext4
# Hostname: somehost.de
auth_username_format = %Ln
mail_debug = yes
mail_home = /var/spool/mail/%u
mail_location = maildir:/var/spool/mail/%u/mail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date index ihave duplicate
mime foreverypart extracttext vnd.dovec
ot.debug
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
}
passdb {
  args = scheme=CRYPT 

Re: LMTP-Process stays in RCPT TO state

[Stephan Bosch via dovecot]

On 05/10/2019 13:01, Pascal Christen via dovecot wrote:


Hi


We tried to update our Dovecot Director-Ring to 2.3 (latest version) 
and had one problem/bug. When we try to deliver an email from Exim to 
a receiving Dovecot (2.3 latest) via the LMTP-Proxy the LMTP process 
stays open forever and never closes. So after a few hours, we hit the 
LMPT-Process limit.


The problem only exists if the receiving Dovecot reports "Quota 
exceeded (mailbox for user is full)" back to the Exim. If this is the 
case the state on the LMPT process stays at "lmtp: [_IP_ RCPT TO] 
(lmtp) and never closes. The TCP-Connection to the Exim is then in 
CLOSE_WAIT state on the Proxy. On the receiving Dovecot we set the 
"lmtp_rcpt_check_quota=yes" option. For me, it sounds similar to this 
bug: https://dovecot.org/list/dovecot/2018-January/110697.html but 
without the SEGFAULT which makes it harder to debug it.


May you guys know why it doesn't change the state to IDLE or READY 
after reporting the quota issue or see something fishy in the commit 
above. If not I try to get a trace of the hanging process.



Greetings Pascal



So far, I haven't been able to reproduce anything weird at this end. Can 
you provide:


- Output from `dovecot -n'
- Protocol logs from the connections between Exim and Dovecot 
director/proxy and between Dovecot director/proxy and Dovecot backend 
(e.g. using ngrep when connections are plaintext or using the rawlog 
facility).
- Dovecot debug logs produced with `log_debug=category:lmtp' for both 
director/proxy and backend


Regards,

Stephan.

Re: Sieve redirect is broken in 2.3.7.2 - signal 11

[Stephan Bosch via dovecot]

Op 4-10-2019 om 12:10 schreef Demonhost via dovecot:

Hi,

If we use sieve redirect under dovecot 2.3.7.2 we end up with

Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 0605F207B0F36: to=, 
relay=127.0.0.1[127.0.0.1]:10024, delay=1.5, delays=0.36/0/0/1.1, 
dsn=2.0.0, status=sent (250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 
250 2.0.0 Ok: queued as 6FC89207B0F38)
Oct 04 03:30:31 dockerhost docker[12154]: 2019-10-04T03:30:31 
53ac2ae27650 postfix: 6FC89207B0F38: to=, relay=dovecot, 
delay=0.31, delays=0.05/0/0/0.26, dsn=4.3.0, status=deferred (Command 
died with signal 11: "/usr/libexec/dovecot/dovecot-lda")


Using docker with official alpine image.

/ # cat /etc/alpine-release
3.8.4

/ # apk info -v | grep dovecot
WARNING: Ignoring APKINDEX.adfa7ceb.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.efaa1f73.tar.gz: No such file or directory
dovecot-2.3.7.2-r0
dovecot-fts-solr-2.3.7.2-r0
dovecot-ldap-2.3.7.2-r0
dovecot-pigeonhole-plugin-2.3.7.2-r0
dovecot-pigeonhole-plugin-ldap-2.3.7.2-r0

Sieve rule is:
# rule:[test]
if header :contains "subject" "test"
{
    redirect "testem...@gmail.com";
}

In host's dmesg we found the following:
[213184.882341] traps: dovecot-lda[6128] general protection 
ip:7ff6cf8144f7 sp:7fff5f800b30 error:0 in 
libdovecot.so.0.0.0[7ff6cf724000+36]


Sorry, not knowledgeable enough on how to do the GBD debug.
-
We went back to previous version and issue disappeared.

8b45bb255b67:/# cat /etc/alpine-release
3.8.4
8b45bb255b67:/# apk info -v | grep dovecot
WARNING: Ignoring APKINDEX.adfa7ceb.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.efaa1f73.tar.gz: No such file or directory
dovecot-2.3.6-r0
dovecot-fts-solr-2.3.6-r0
dovecot-ldap-2.3.6-r0
dovecot-pigeonhole-plugin-2.3.6-r0
dovecot-pigeonhole-plugin-ldap-2.3.6-r0

Anyone else already had this problem or you know how to fix it?


I can't readily reproduce it with the information available so far. Do 
you have the opportunity to obtain a GDB backtrace? You can e.g. run LDA 
manually at the command line with the same arguments and message input 
Postfix provides and wrap it in GDB if you can reproduce the crash.


Your configuration may also help (output from `dovecot -n`)

Regards,

Stephan.

Re: Home Directory Creation

[Stephan Bosch via dovecot]

On 03/10/2019 22:43, Asai via dovecot wrote:


Tried your config here and it works. My debug log looks as follows 
for the first brief IMAP session:


Oct  3 22:28:56 drieka dovecot: imap-login: Login: 
user=, method=PLAIN, rip=::1, lip=::1, mpid=21949, 
secured, session=
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Added 
userdb setting: plugin/=yes
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Effective 
uid=1001, gid=1001, home=/vmail//diederik-jan/home
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Home dir 
not found: /vmail//diederik-jan/home
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Namespace 
inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes location=maildir:/vmail//diederik-jan
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: maildir++: 
root=/vmail//diederik-jan, index=, indexpvt=, control=, 
inbox=/vmail//diederik-jan, alt=
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Namespace 
: /vmail//diederik-jan doesn't exist yet, using default permissions
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Namespace 
: Using permissions from /vmail//diederik-jan: mode=0700 gid=default
Oct  3 22:29:02 drieka dovecot: imap(diederik-jan): Debug: INBOX: 
Mailbox opened because: SELECT
Oct  3 22:29:04 drieka dovecot: imap(diederik-jan): Logged out in=31 
out=747


Took me a few tries to get this working though. You need to configure 
the permissions of /vmail correctly. Are you sure you're looking in 
the correct log file for finding errors?


You can execute the following to find out:

$ sudo doveadm log find
Looking for log files from /var/log
Debug: /var/log/syslog
Debug: /var/log/mail.log
Info: /var/log/syslog
Info: /var/log/mail.info
Info: /var/log/mail.log
Warning: /var/log/mail.warn
Warning: /var/log/syslog
Warning: /var/log/mail.info
Warning: /var/log/mail.log
Error: /var/log/mail.warn
Error: /var/log/syslog
Error: /var/log/mail.info
Error: /var/log/mail.err
Error: /var/log/mail.log
Fatal: /var/log/mail.warn
Fatal: /var/log/syslog
Fatal: /var/log/mail.info
Fatal: /var/log/mail.err
Fatal: /var/log/mail.log



Thank you for your reply, Stephan.

But did it create the home directory in the new account's mail folder? 
e.g. /vmail/domain/user/home?


Because this debug message is exactly similar to what I'm getting: 
"Debug: Home dir not found: /vmail//diederik-jan/home"




Ah! I see what you mean now. No, indeed, that home directory is not 
created. IMAP doesn't really need it, so it doesn't fail. In contrast, 
ManageSieve does fail on that:


Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: Effective uid=1001, gid=1001, 
home=/vmail/example.com/pieter-joost/home
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: Home dir not found: /vmail/example.com/pieter-joost/home
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, 
hidden=no, list=yes, subscriptions=yes 
location=maildir:/vmail/example.com/pieter-joost
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: maildir++: root=/vmail/example.com/pieter-joost, index=, 
indexpvt=, control=, inbox=/vmail/example.com/pieter-joost, alt=
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: sieve: Pigeonhole version 0.4.25.alpha0 (3bbd9e9c) initializing
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: sieve: include: sieve_global is not set; it is currently not 
possible to include `:global' scripts.
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: sieve: file storage: Performing auto-detection
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Debug: sieve: file storage: access(/vmail/example.com/pieter-joost/home, 
rwx) failed: No such file or directory
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Error: sieve: file storage: Could not find storage root directory for 
write access; path was left unconfigured and autodetection failed
Oct  3 22:58:19 drieka dovecot: managesieve(pieter-jo...@example.com): 
Fatal: Failed to open Sieve storage.


Looks like this still old reply from TImo applies: 
https://dovecot.org/list/dovecot/2013-May/090397.html


The welcome plugin could maybe of help:

https://doc.dovecot.org/plugin-settings/welcome-plugin/

Regards,

Stephan.

Re: Dovecote IMAPSieve user scripts

[Stephan Bosch via dovecot]

On 27/09/2019 11:51, Gerald Galster via dovecot wrote:



I wonder how to configure IMAPSieve with user scripts. I can't find much
information on the internet.


try to enable managesieve:

service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}


https://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration

Roundcube or Thunderbird with Sieve plugin connect to your dovecot 
server on port 4190.
Users are authenticatd with the same credentials used for pop/imap and 
can upload their

scripts.


Yes, but then nothing.  No client I know of supports the IMAPSIEVE 
capability. The ones that support Sieve only support uploading scripts 
(through ManageSieve) for delivery Sieve scripts, not for Sieve scripts 
running on IMAP events.


Regardss,

Re: Dovecote IMAPSieve user scripts

[Stephan Bosch via dovecot]

On 27/09/2019 11:30, Alexander Mankuta via dovecot wrote:

Hello,

I wonder how to configure IMAPSieve with user scripts. I can't find much
information on the internet.

I have Sieve and IMAPSieve for spam configured as described in the wiki
and it works.

Documentation mentions that imapsieve_url has to be set to appropriate
ManageSieve server but no further explanation is given.


It is an IMAP capability. When that imapsieve_url setting is configured 
properly, the IMAP server will announce support for the IMAPSIEVe 
capability. However, if you wish to use this, your IMAP client needs to 
provide support. So far, I haven't seen any of those. Currently, the 
IMAPSieve facilities are mostly used for administrator-based filtering 
(e.g. Spam/Ham learning based on user activity).


The specification of the IMAP capability is here: 
https://tools.ietf.org/html/rfc6785



What I want to achieve: let users create their own sieve scripts and
execute them each time a message is being moved/copied between their
mailboxes.

I saw mentions of "association of user scripts with mailboxes" but I'm
not sure what that means or if it's supported by Thuderbird Sieve
extension (or any other software out there).


I haven't seen support like that in the Thunderbird add-on.

Regards,

Stephan.


Please advice.

--
Regards,
Alex

Re: Doveadm error during store - segfault clucene

[Stephan Bosch via dovecot]

On 02/10/2019 10:12, Domenico Pastore via dovecot wrote:



Hi,
during the store of the messages, I find the following doveadm errors:

2019-10-02 09:22:19 auth: Debug: 
ldap(test@test.xxx.it,10.11.47.60): result: 
mailMessageStore=1/1/2/0/8/7/2/1120872_91093 mailQuota=52428800; 
mailQuota,mailMessageStore unused
2019-10-02 09:22:19 auth: Debug: 
ldap(test@test.xxx.it,10.11.47.60): Finished userdb lookup
2019-10-02 09:22:19 auth: Debug: userdb out: USER    1 
test@test.xxx.it 
home=/store/mailbox/mailbox_test/1/1/2/0/8/7/2/1120872_91093 
quota_rule=*:bytes=52428800    mail_plugins+=
*2019-10-02 09:22:19 
doveadm(test@test.xxx.it)<180667>: Fatal: 
master: service(doveadm): child 180667 killed with signal 11 (core 
dumped)*
2019-10-02 09:22:19 auth: Debug: master in: USER    1 
test@test.xxx.it    service=doveadm lip=192.160.10.100    
lport=924    rip=10.11.47.60 rport=56515
2019-10-02 09:22:19 auth: Debug: 
ldap(test@test.xxx.it,10.11.47.60): Performing userdb lookup


This problem occurs only with the quota plugin enabled.

Can you help me?



The GDB trace tells me this is most likely a Lucene bug. There is very 
little Dovecot can mess up at library deinit like that.


Regards,

Stephan.


Best regards

Domenico

-

(gdb)
#0  clear (this=0x0) at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/util/VoidMap.h:123
#1  lucene::search::FieldSortedHitQueue::_shutdown () at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/search/FieldSortedHitQueue.cpp:60
#2  0x7f4b37a62e96 in _lucene_shutdown () at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/StdHeader.cpp:33
#3  0x7f4b3b2f2fbb in module_dir_deinit (modules=) 
at module-dir.c:578
#4  0x7f4b3b2f3001 in module_dir_unload 
(modules=modules@entry=0x7f4b3b8e1380 ) 
at module-dir.c:590
#5  0x7f4b3b5dd264 in mail_storage_service_deinit 
(_ctx=_ctx@entry=0x55ea409b7838) at mail-storage-service.c:1730
#6  0x55ea3f4a3d35 in doveadm_mail_cmd_exec 
(ctx=ctx@entry=0x55ea409b77f8, wildcard_user=wildcard_user@entry=0x0) 
at doveadm-mail.c:673
#7  0x55ea3f4a4bbd in doveadm_cmd_ver2_to_mail_cmd_wrapper 
(cctx=) at doveadm-mail.c:1144
#8  0x55ea3f4b5c98 in doveadm_cmd_run_ver2 (argc=argc@entry=1, 
argv=argv@entry=0x55ea40997150, cctx=cctx@entry=0x7fff9a1cb990) at 
doveadm-cmd.c:574
#9  0x55ea3f4ba020 in doveadm_cmd_server_run_ver2 
(cctx=0x7fff9a1cb990, argv=0x55ea40997150, argc=1, 
conn=0x55ea409ebbd8) at client-connection-tcp.c:178
#10 doveadm_cmd_handle (cctx=0x7fff9a1cb990, argv=0x55ea40997150, 
argc=1, cmd_name=, conn=0x55ea409ebbd8) at 
client-connection-tcp.c:342
#11 client_handle_command (args=, conn=0x55ea409ebbd8) 
at client-connection-tcp.c:415
#12 client_connection_tcp_input (conn=0x55ea409ebbd8) at 
client-connection-tcp.c:561
#13 0x7f4b3b2e855f in io_loop_call_io (io=0x55ea409c5b10) at 
ioloop.c:704
#14 0x7f4b3b2e9fa3 in io_loop_handler_run_internal 
(ioloop=ioloop@entry=0x55ea4099fcb0) at ioloop-epoll.c:221
#15 0x7f4b3b2e8666 in io_loop_handler_run 
(ioloop=ioloop@entry=0x55ea4099fcb0) at ioloop.c:756
#16 0x7f4b3b2e in io_loop_run (ioloop=0x55ea4099fcb0) at 
ioloop.c:729
#17 0x7f4b3b25b2d3 in master_service_run (service=0x55ea4099fb40, 
callback=callback@entry=0x55ea3f4bd260 ) at 
master-service.c:781

#18 0x55ea3f493421 in main (argc=1, argv=0x55ea4099f890) at main.c:151

(gdb) bt full
#0  clear (this=0x0) at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/util/VoidMap.h:123

No locals.
#1  lucene::search::FieldSortedHitQueue::_shutdown () at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/search/FieldSortedHitQueue.cpp:60

No locals.
#2  0x7f4b37a62e96 in _lucene_shutdown () at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/StdHeader.cpp:33

No locals.
#3  0x7f4b3b2f2fbb in module_dir_deinit (modules=) 
at module-dir.c:578

    _data_stack_cur_id = 4
    module = 0x55ea40a048b0
    rev = 0x55ea4099ac18
    i = 
    count = 6
#4  0x7f4b3b2f3001 in module_dir_unload 
(modules=modules@entry=0x7f4b3b8e1380 ) 
at module-dir.c:590

    module = 
    next = 
#5  0x7f4b3b5dd264 in mail_storage_service_deinit 
(_ctx=_ctx@entry=0x55ea409b7838) at mail-storage-service.c:1730

    ctx = 0x55ea409f0bf8
#6  0x55ea3f4a3d35 in doveadm_mail_cmd_exec 
(ctx=ctx@entry=0x55ea409b77f8, wildcard_user=wildcard_user@entry=0x0) 
at doveadm-mail.c:673

    cctx = 
    cli = 
    ret = 
    error = 0x7f4b3b22d2f7 <__x86_return_thunk+5> 
"\363\220\017\256\350\353\371H\215d$\b\303\350\a"
#7  0x55ea3f4a4bbd in doveadm_cmd_ver2_to_mail_cmd_wrapper 
(cctx=) at doveadm-mail.c:1144

    mctx = 0x55ea409b77f8
    wildcard_user = 
    fieldstr = 0x55ea40997310 "͒\t7K\177"
    pargv = {arr = {buffer = 0x55ea409b7a28, element_size = 8}, v 
= 0x55ea409b7a28, v_modifiable = 0x55ea409b7a28}
    full_args = {arr = {buffer = , 

Re: Home Directory Creation

[Stephan Bosch via dovecot]

On 03/10/2019 17:26, Asai via dovecot wrote:

Mr. Timo,

I would presume you are busy with many responsibilities, but might you 
have a few minutes to consider this problem?


Or is there any other Dovecot expert who may be able to offer advice 
on how to get the home directories created?  Thank you for those who 
have helped offer advice on this already.




Tried your config here and it works. My debug log looks as follows for 
the first brief IMAP session:


Oct  3 22:28:56 drieka dovecot: imap-login: Login: user=, 
method=PLAIN, rip=::1, lip=::1, mpid=21949, secured, 
session=
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Added userdb 
setting: plugin/=yes
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Effective 
uid=1001, gid=1001, home=/vmail//diederik-jan/home
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Home dir not 
found: /vmail//diederik-jan/home
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Namespace 
inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, 
subscriptions=yes location=maildir:/vmail//diederik-jan
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: maildir++: 
root=/vmail//diederik-jan, index=, indexpvt=, control=, 
inbox=/vmail//diederik-jan, alt=
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Namespace : 
/vmail//diederik-jan doesn't exist yet, using default permissions
Oct  3 22:28:56 drieka dovecot: imap(diederik-jan): Debug: Namespace : 
Using permissions from /vmail//diederik-jan: mode=0700 gid=default
Oct  3 22:29:02 drieka dovecot: imap(diederik-jan): Debug: INBOX: 
Mailbox opened because: SELECT

Oct  3 22:29:04 drieka dovecot: imap(diederik-jan): Logged out in=31 out=747

Took me a few tries to get this working though. You need to configure 
the permissions of /vmail correctly. Are you sure you're looking in the 
correct log file for finding errors?


You can execute the following to find out:

$ sudo doveadm log find
Looking for log files from /var/log
Debug: /var/log/syslog
Debug: /var/log/mail.log
Info: /var/log/syslog
Info: /var/log/mail.info
Info: /var/log/mail.log
Warning: /var/log/mail.warn
Warning: /var/log/syslog
Warning: /var/log/mail.info
Warning: /var/log/mail.log
Error: /var/log/mail.warn
Error: /var/log/syslog
Error: /var/log/mail.info
Error: /var/log/mail.err
Error: /var/log/mail.log
Fatal: /var/log/mail.warn
Fatal: /var/log/syslog
Fatal: /var/log/mail.info
Fatal: /var/log/mail.err
Fatal: /var/log/mail.log

Regards,

Stephan.

Asai

On 10/2/2019 5:08 PM, Asai via dovecot wrote:

Hello,

I spent the afternoon troubleshooting this, and after hours, I still 
can't see what I'm missing.


On the older version of Dovecot (2.2) it worked fine to create a new 
user, then Dovecot would create the home folder under the maildir 
folder as outlined in the documentation, simply by creating a new 
user in the database, and opening the new user account in the mail 
client.



# 2.3.7.2 (3c910f64b): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 (7372921a)
# Hostname: triata4
auth_debug = yes
auth_debug_passwords = yes
auth_mechanisms = plain login
auth_verbose = yes
auth_verbose_passwords = plain
debug_log_path = /var/log/dovecot-debug.log
deliver_log_format = msgid=%m: from=%f: %$
hostname = triata.globalchange.media
info_log_path = /var/log/dovecot-info.log
mail_debug = yes
mail_fsync = always
mail_home = /vmail/%d/%n/home
mail_location = maildir:/vmail/%d/%n
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext

namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix =
  separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-mysql.conf
  driver = sql
}
plugin {
  mail_log_fields = uid box msgid from flags
  sieve = 
file:/vmail/%d/%n/home/sieve;active=/vmail/%d/%n/home/.dovecot.sieve

  sieve_after = /etc/dovecot/sieve/junk-spam.sieve
  sieve_before = /etc/dovecot/sieve/junk-spam.sieve
  sieve_dir = /vmail/%d/%n/home/sieve
}
protocols = imap lmtp sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  unix_listener auth-master {
    group = vmail
    mode = 0660
    user = vmail
  }
  unix_listener auth-userdb {
    group = vmail
    mode = 0666
    user = vmail
  }
  user = root
}
service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
    ssl = yes
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    

Re: Lost mails with sieve "duplicate" extension in case of failure

[Stephan Bosch via dovecot]

On 01/10/2019 19:54, Stephan Bosch wrote:



On 01/10/2019 19:03, Hanno Stock via dovecot wrote:

RFC 7352 however explicitly states:

    Implementations MUST only update the internal duplicate-tracking 
list

    when the Sieve script execution finishes successfully.  If failing
    script executions add the unique ID to the duplicate-tracking list,
    all "duplicate" tests in the Sieve script would erroneously yield
    "true" for the next delivery attempt of the same message.

Dovecot seems not to honour this specification.

Maybe this is because the "before_script" is not the script that fails,
but the subsequent scripts fail?
Am I missing some configuration option that would propagate the
subsequent failure back to the before_script?
Or is this a general limitation of the before_script mechanism?

Any help appreciated how to alleviate this issue!


This is most definitely a bug.


Yes, it is a design flaw in our implementation. Tracking internally as 
DOP-1449.


Regards,

Stephan.






--

Filesystem: ext4

Dovecot config:

# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-11-amd64 x86_64 Debian 9.11
auth_cache_negative_ttl = 0
auth_cache_size = 1 M
auth_cache_ttl = 2 hours
auth_mechanisms = plain login
auth_username_format = %Ln
mail_gid = vmail
mail_home = /var/mail/vhosts/mail.COMPANY.com/%n
mail_location = maildir:~/.dovecot
mail_plugins = " acl"
mail_privileged_group = vmail
mail_uid = vmail
mailbox_list_index = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext editheader imapflags notify
namespace {
   hidden = yes
   list = no
   location = maildir:~/.dovecot-top-namespace
   prefix =
   separator = .
   subscriptions = yes
   type = private
}
namespace {
   location =
maildir:/var/mail/vhosts/mail.COMPANY.com/%%n/.dovecot:INDEXPVT=/var/mail/vhosts/mail.COMPANY.com/%n/.dovecot/shared/%%n 


   prefix = shared.%%u.
   separator = .
   subscriptions = no
   type = shared
}
namespace inbox {
   inbox = yes
   location =
   mailbox Drafts {
 special_use = \Drafts
   }
   mailbox Junk {
 special_use = \Junk
   }
   mailbox Sent {
 special_use = \Sent
   }
   mailbox "Sent Messages" {
 special_use = \Sent
   }
   mailbox Trash {
 special_use = \Trash
   }
   prefix = INBOX.
   separator = .
}
passdb {
   args = /etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}
plugin {
   acl = vfile
   acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db
   sieve =
file:/var/mail/vhosts/mail.COMPANY.com/%n/.sieve/sieve;active=/var/mail/vhosts/mail.COMPANY.com/%n/.sieve/.dovecot.sieve 


   sieve_after = /var/mail/sieve/global-after.sieve
   sieve_before = /var/mail/sieve/global-before.sieve
   sieve_default = /var/mail/sieve/global.sieve
   sieve_extensions = +imapflags +notify +editheader
   sieve_global = /var/mail/sieve
}
protocols = imap lmtp sieve
service auth-worker {
   user = vmail
}
service auth {
   unix_listener /var/spool/postfix/private/auth {
 group = postfix
 mode = 0600
 user = postfix
   }
   unix_listener auth-userdb {
 mode = 0600
 user = vmail
   }
   user = dovecot
}
service imap-login {
   inet_listener imap {
 port = 0
   }
   inet_listener imaps {
 port = 993
 ssl = yes
   }
   process_limit = 9000
}
service imap {
   process_limit = 9000
}
service lmtp {
   unix_listener /var/spool/postfix/private/dovecot-lmtp {
 group = postfix
 mode = 0600
 user = postfix
   }
}
service pop3-login {
   inet_listener pop3 {
 port = 0
   }
   inet_listener pop3s {
 port = 0
   }
}
ssl = required
ssl_cert = 

Re: Lost mails with sieve "duplicate" extension in case of failure

[Stephan Bosch via dovecot]

On 01/10/2019 19:03, Hanno Stock via dovecot wrote:

Hi!

we just experienced the following issue:

We have a global sieve_before script using the duplicate extension.

require ["duplicate", "imap4flags"];
if duplicate {
 discard;
}

Now the storage of our mailbox store got full and lmtp deliveries
started to fail.

This resulted in logs like this:

Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error:
o_stream_send_istream(/var/mail/vhosts/mail.COMPANY.com/USERNAME/.dovecot/tmp/1569420896.M612482P5820.mail-rc1)
failed: No space left on device
Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error:
write(/var/mail/vhosts/mail.COMPANY.com/USERNAME/.dovecot/tmp/1569420896.M612482P5820.mail-rc1)
failed: No space left on device
Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error:
KT9HJGB2i128FgAATXMPig: sieve:
msgid=:
failed to store into mailbox 'INBOX': Internal error occurred. Refer to
server log for more information. [2019-09-25 16:14:56]
Sep 25 16:14:56 mail-rc1 dovecot: lmtp(USERNAME): Error:
KT9HJGB2i128FgAATXMPig: sieve: Execution of script
/mnt/data/var/mail/vhosts/mail.COMPANY.com/USERNAME/.sieve/.dovecot.sieve was
aborted due to temporary failure (user logfile
/mnt/data/var/mail/vhosts/mail.COMPANY.com/USERNAME/.sieve/.dovecot.sieve.log
may reveal additional details)
Sep 25 16:14:56 mail-rc1 postfix/lmtp[5851]: 4D27690AC:
to=, orig_to=,
relay=mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp], delay=1.3,
delays=1.3/0/0/0.01, dsn=4.2.0, status=deferred (host
mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp] said: 451 4.2.0
 Internal error occurred. Refer to server log
for more information. [2019-09-25 16:14:56] (in reply to end of DATA
command))

So far, so good... A 451 response is what we wanted.

However on the next delivery attempt the following happens:

Sep 25 16:24:00 mail-rc1 postfix/lmtp[7040]: 4D27690AC:
to=, orig_to=,
relay=mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp], delay=545,
delays=545/0.09/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0
 DBcCEYB4i11qGwAATXMPig Saved)
Sep 25 16:24:00 mail-rc1 dovecot: lmtp(USERNAME):
DBcCEYB4i11qGwAATXMPig: sieve:
msgid=:
marked message to be discarded if not explicitly delivered (discard action)
Sep 25 16:24:00 mail-rc1 postfix/lmtp[7040]: 4D27690AC:
to=, orig_to=,
relay=mail-rc1.bvd.COMPANY.x[private/dovecot-lmtp], delay=545,
delays=545/0.09/0.02/0.01, dsn=2.0.0, status=sent (250 2.0.0
 DBcCEYB4i11qGwAATXMPig Saved)

As the duplicate extension has already seen the mail, it is now
immediately discarded.

RFC 7352 however explicitly states:

Implementations MUST only update the internal duplicate-tracking list
when the Sieve script execution finishes successfully.  If failing
script executions add the unique ID to the duplicate-tracking list,
all "duplicate" tests in the Sieve script would erroneously yield
"true" for the next delivery attempt of the same message.

Dovecot seems not to honour this specification.

Maybe this is because the "before_script" is not the script that fails,
but the subsequent scripts fail?
Am I missing some configuration option that would propagate the
subsequent failure back to the before_script?
Or is this a general limitation of the before_script mechanism?

Any help appreciated how to alleviate this issue!


This is most definitely a bug.

Regards,

Stephan.



Best regards

Hanno

---

Filesystem: ext4

Dovecot config:

# 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.16 (fed8554)
# OS: Linux 4.9.0-11-amd64 x86_64 Debian 9.11
auth_cache_negative_ttl = 0
auth_cache_size = 1 M
auth_cache_ttl = 2 hours
auth_mechanisms = plain login
auth_username_format = %Ln
mail_gid = vmail
mail_home = /var/mail/vhosts/mail.COMPANY.com/%n
mail_location = maildir:~/.dovecot
mail_plugins = " acl"
mail_privileged_group = vmail
mail_uid = vmail
mailbox_list_index = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date index ihave duplicate mime foreverypart
extracttext editheader imapflags notify
namespace {
   hidden = yes
   list = no
   location = maildir:~/.dovecot-top-namespace
   prefix =
   separator = .
   subscriptions = yes
   type = private
}
namespace {
   location =
maildir:/var/mail/vhosts/mail.COMPANY.com/%%n/.dovecot:INDEXPVT=/var/mail/vhosts/mail.COMPANY.com/%n/.dovecot/shared/%%n
   prefix = shared.%%u.
   separator = .
   subscriptions = no
   type = shared
}
namespace inbox {
   inbox = yes
   location =
   mailbox Drafts {
 special_use = \Drafts
   }
   mailbox Junk {
 special_use = \Junk
   }
   mailbox Sent {
 special_use = \Sent
   }
   mailbox "Sent Messages" {
 special_use = \Sent
   }
   mailbox Trash {
 special_use = \Trash
   }
   prefix = INBOX.
   separator = .
}
passdb {
   args = /etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}
plugin {
   acl = 

Re: CVE-2019-11500 and LMTP error

[Stephan Bosch via dovecot]

Op 9-9-2019 om 11:26 schreef Gabriele Nencioni via dovecot:

Hi all,
does the dovecot fixed version: 2.3.7.2, 2.2.36.4 (as the CVE-2019-11500
says) fix the LMTP error "Got unexpected reply" as well?

The LMTP error "Got unexpected reply" is described here:
https://dovecot.org/pipermail/dovecot/2018-August/112562.html
https://dovecot.org/pipermail/dovecot/2018-August/112666.html



This was tracked internally as DOV-2864, which was released as part of 
2.3.6.


Regards,

Stephan.

Re: Dovecot 2.3.7.2 lucene segfault and coredump

[Stephan Bosch via dovecot]

Op 16-9-2019 om 11:53 schreef Domenico Pastore via dovecot:


Hi,

I have other information.

I analyzed the core dump:

(gdb) core 22534
[New LWP 22534]
Reading symbols from /usr/libexec/dovecot/doveadm-server...Reading 
symbols from 
/usr/lib/debug/usr/libexec/dovecot/doveadm-server.debug...done.

done.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `dovecot-devel01/doveadm-server'.
Program terminated with signal 11, Segmentation fault.
#0  clear (this=0x0) at 
/usr/src/debug/clucene-core-2.3.3.4/src/core/CLucene/util/VoidMap.h:123

123            if ( dk || dv ){
(gdb)



Can you issue GDB command `bt full` at this point?

Regards,

Stephan.



Br,

Domenico

*Domenico Pastore*
Senior Cloud Engineer
*T* 06.98269600 | *M* 347.1474270 | *F* 06.98269680
*Par-Tec * | *beyond the IT domain*
Via Cristoforo Colombo 163, 00147 Roma
*CONFIDENZIALE*: Questo messaggio ed i suoi allegati sono di carattere 
confidenziale. È vietato l'inoltro non autorizzato a destinatari 
diversi da quelli indicati nel messaggio originale. Se ricevuto per 
errore si prega di informare il mittente e cancellarlo immediatamente.


Il 16/09/19 09:50, Domenico Pastore via dovecot ha scritto:


Hi,

do you have the same problem with lucene?

I collected a strace on the dovecot process during indexing:

[pid 31339] execve("/usr/libexec/dovecot/doveadm-server", 
["dovecot-devel01/doveadm-server"], [/* 18 vars */] 
[pid 30711] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, 
si_pid=30932, si_uid=0, si_status=SIGSEGV, si_utime=3, si_stime=1} ---

[pid 30711] write(62, "\0", 1)  = 1
[pid 30711] read(124, "8x\0\0008\0\0\0\251\3\0\0", 1020) = 12
[pid 30711] read(61, "\0", 64)  = 1
[pid 30711] open("/proc/sys/fs/suid_dumpable", O_RDONLY) = 227
[pid 30711] read(227, "0\n", 9) = 2
[pid 30711] write(65, "25 30932 FATAL service(doveadm): child 30932 
killed with signal 11 (core not dumped - 
https://dovecot.org/bugreport.html#coredumps - set 
/proc/sys/fs/suid_dumpable to 2)\n", 170) = 170

[pid 30711] write(14, "KILL\t30932\n", 11) = 11
[pid 30711] write(65, "25 30932 BYE\n", 13) = 13
[pid 31339] <... execve resumed> )  = 0
[pid 31339] open("/usr/lib64/dovecot/tls/x86_64/libz.so.1", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 31339] stat("/usr/lib64/dovecot/tls/x86_64", 0x7ffda7e3f1d0) = 
-1 ENOENT (No such file or directory)
[pid 31339] open("/usr/lib64/dovecot/tls/libz.so.1", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 31339] stat("/usr/lib64/dovecot/tls", 0x7ffda7e3f1d0) = -1 
ENOENT (No such file or directory)
[pid 31339] open("/usr/lib64/dovecot/x86_64/libz.so.1", 
O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
[pid 31339] stat("/usr/lib64/dovecot/x86_64", 0x7ffda7e3f1d0) = -1 
ENOENT (No such file or directory)
[pid 31339] open("/usr/lib64/dovecot/libz.so.1", O_RDONLY|O_CLOEXEC) 
= -1 ENOENT (No such file or directory)
[pid 31339] stat("/usr/lib64/dovecot", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}) = 0

[pid 31339] open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 4
[pid 31339] open("/lib64/libz.so.1", O_RDONLY|O_CLOEXEC) = 4


All missing libs are in "/usr/lib64" and no to "/usr/lib64/dovecot"

Can you help me?

Br,

Domenico



*Domenico Pastore*
Senior Cloud Engineer
*T* 06.98269600 | *M* 347.1474270 | *F* 06.98269680
*Par-Tec * | *beyond the IT domain*
Via Cristoforo Colombo 163, 00147 Roma
*CONFIDENZIALE*: Questo messaggio ed i suoi allegati sono di 
carattere confidenziale. È vietato l'inoltro non autorizzato a 
destinatari diversi da quelli indicati nel messaggio originale. Se 
ricevuto per errore si prega di informare il mittente e cancellarlo 
immediatamente.


Il 12/09/19 12:07, Domenico Pastore via dovecot ha scritto:


Hi,

i have a problem with libclucene-core.so.2.3.3.4.
I See segfault error to messages log, when dovecot store mail to INBOX.

set 12 13:41:21 box02.par-tec.it kernel: doveadm-server[20285]: 
*segfault at 38 ip 7f575b68fc42 sp 7ffc0272e750 error 4 in 
libclucene-core.so.2.3.3.4*[7f575b569000+188000]
set 12 13:41:22 box02.par-tec.it kernel: doveadm-server[18871]: 
segfault at 38 ip 7f9bea6d2c42 sp 7fff47bceae0 error 4 in 
libclucene-core.so.2.3.3.4[7f9bea5ac000+188000]


In the dovecot logs:

2019-09-12 13:45:09 auth: Debug: 
ldap(testclient@newsvil,10.0.8.128): Finished userdb lookup
2019-09-12 13:45:09 auth: Debug: userdb out: USER    1 
testclient@newsvil 
home=/pec_store/pec_mailbox/pec_mailbox_devel01/4/5/6/7/1/45671_18231 
quota_rule=*:bytes=1048576000    mail_plugins+=
2019-09-12 13:45:10  SKENO8Uvel0gbAAA5ZKy4w 
doveadm(testclient@newsvil): *Fatal: master: 
service(doveadm): child 27680 killed with signal 11 (core dumped)*
2019-09-12 13:45:10 auth: Debug: master in: USER    1 
testclient@newsvil    service=doveadm lip=192.160.10.3    
lport=924    

Re: Subscribe to a fileinto :create mailbox?

[Stephan Bosch via dovecot]

On 08/09/2019 04:16, Larry Rosenman via dovecot wrote:

Is it possible in pigeonhole?

I.E.  I do a fileinto :create "some/mail/box"
How can I make it autosubscribe?


You cannot control this from within Sieve (there is no :subscribe 
argument). However, you can  auto-subscribe to *all* mailboxes created 
in Sieve by setting lda_mailbox_autosubscribe=yes, which is usually 
adequate.


Regards,

Stephan.




doveconf -n:
# 2.3.7.2 (3c910f64b): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.7.2 (7372921a)
# OS: FreeBSD 12.0-STABLE amd64
# Hostname: thebighonker.lerctr.org 
auth_default_realm = lerctr.org 
auth_mechanisms = plain login
auth_realms = lerctr.org  thebighonker.lerctr.org 
 tbh.lerctr.org 
 thejonesonair.com  
thejonesonair.net  why.net 

default_vsz_limit = 1 G
deliver_log_format = msgid=%m: %$ (subject=%s from=%f size=%w)
doveadm_password = # hidden, use -P to show it
first_valid_gid = 0
first_valid_uid = 0
lda_mailbox_autocreate = yes
listen = 192.147.25.65, ::
login_access_sockets = tcpwrap
mail_attribute_dict = file:%h/mail/.imap/dovecot-mail-attributes
mail_location = mbox:~/mail:INBOX=~/mail/INBOX
mail_log_prefix = "%s(%u/%p): "
mail_plugins = " fts fts_solr notify virtual"
mail_privileged_group = mail
mail_server_admin = mailto:l...@lerctr.org 
mail_server_comment = LERCTR Mail Server
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext vacation-seconds editheader mboxmetadata servermetadata 
vnd.dovecot.debug imapsieve vnd.dovecot.imapsieve

namespace archive {
  hidden = no
  list = no
  location = mbox:~/MAIL-ARCHIVE
  prefix = ARCHIVE/
  separator = /
}
namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
    special_use = \Drafts
  }
  mailbox INBOX {
    auto = create
  }
  mailbox SENT {
    special_use = \Sent
  }
  mailbox SPAM {
    special_use = \Junk
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  mailbox virtual/Flagged {
    special_use = \Flagged
  }
  mailbox virtual/all {
    special_use = \All
  }
  prefix =
  separator = /
}
namespace virtual {
  hidden = no
  list = yes
  location = virtual:~/MAIL-VIRTUAL
  prefix = Virtual/
  separator = /
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
passdb {
  args = user=%Ln noauthenticate
  driver = static
  skip = authenticated
}
passdb {
  args = failure_show_msg=yes session=yes max_requests=20
  driver = pam
  override_fields = domain=lerctr.org 
  skip = authenticated
}
plugin {
  fts = solr
  fts_autoindex = yes
  fts_solr = url=http://thebighonker.lerctr.org:8983/solr/dovecot/
  fts_tika = http://localhost:9998/tika/
  imapsieve_mailbox1_before = 
file:/usr/local/share/dovecot-pigeonhole/sieve/report-spam.sieve

  imapsieve_mailbox1_causes = COPY
  imapsieve_mailbox1_name = SPAM
  imapsieve_mailbox2_before = 
file:/usr/local/share/dovecot-pigeonhole/sieve/report-ham.sieve

  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_from = SPAM
  imapsieve_mailbox2_name = *
  imapsieve_url = sieve://thebighonker.lerctr.org 

  mail_log_events = delete undelete expunge copy mailbox_delete 
mailbox_rename flag_change append

  mail_log_fields = uid box msgid size from subject vsize flags
  recipient_delimiter = +-_
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_execute_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve
  sieve_extensions = +editheader +vacation-seconds +mboxmetadata 
+servermetadata +vnd.dovecot.debug

  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.execute
  sieve_pipe_bin_dir = /usr/local/share/dovecot-pigeonhole/sieve
  sieve_plugins = sieve_imapsieve sieve_extprograms
}
postmaster_address = postmas...@lerctr.org 
protocols = imap pop3 lmtp sieve
recipient_delimiter = +-_
service anvil {
  unix_listener anvil {
    group = mail
    mode = 0666
  }
}
service auth {
  unix_listener auth-client {
    mode = 0666
  }
  unix_listener auth-master {
    mode = 0666
  }
}
service doveadm {
  inet_listener http {
    port = 8080
    ssl = yes
  }
}
service indexer-worker {
  drop_priv_before_exec = yes
}
service lmtp {
  inet_listener lmtp {
    address = 127.0.0.1
    port = 24
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
  inet_listener sieve_deprecated {
    port = 2000
  }
}
service stats {
  unix_listener stats-reader {
    group = mail
    mode = 0666
    user =
  }
  unix_listener stats-writer {
  

Re: Case sensitive problem for vacation with dovecot lmtp pigeonhole

[Stephan Bosch via dovecot]

On 09/06/2019 16:07, kadafax--- via dovecot wrote:

Hi list,

This issue was raised here by John Fawcett on 12/02/19 but got no 
response.

So, as I'm facing the same issue, I would like to try again:

When using the vacation sieve extension, if I send an email to 
user.n...@mydomain.com it will work, but not to 
user.n...@mydomain.com. Is there a way to make the user part case 
insensitive so that the vacation auto-reply is fired whatever the way 
user part is written ?


We're looking into this. Tracking internally as DOP-1424.

Regards,

Stephan.



doveconf -n output:

# 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.5 (2483b085)
# OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 
7.6.1810 (Core)  ext4

# Hostname: myserver.mydomain.com
auth_master_user_separator = *
auth_username_chars = 
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*

auth_verbose = yes
debug_log_path = /var/log/dovecot-debug.log
default_client_limit = 10640
default_process_limit = 2128
default_vsz_limit = 0
hostname = mydomain.com
info_log_path = /var/log/dovecot.log
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *
log_path = /var/log/dovecot-error.log
login_greeting = Welcome
mail_gid = vmail
mail_location = maildir:/var/vmail/%Ln
mail_plugins = quota fts fts_lucene
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext

namespace inbox {
  inbox = yes
  location = maildir:/var/vmail/%Ln
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix =
  type = private
}
passdb {
  args = /usr/local/etc/dovecot/deny-users
  deny = yes
  driver = passwd-file
}
passdb {
  args = /usr/local/etc/dovecot/master-users
  driver = passwd-file
  master = yes
  pass = yes
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars=@.
  quota = maildir:User quota
  quota_rule = *:storage=30GB
  quota_rule2 = Trash:storage=+200M
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = /var/sieve-scripts/%Ln.sieve
  sieve_dir = /var/vmail/%Ln/home/sieve
}
postmaster_address = postmas...@mydomain.com
protocols = imap pop3 lmtp sieve sieve
quota_full_tempfail = yes
service auth-worker {
  user = $default_internal_user
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0777
    user = vmail
  }
}
service imap-login {
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_min_avail = 10
}
service imap {
  process_limit = 2128
}
service lmtp {
  executable = lmtp -L
  process_min_avail = 15
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}
service pop3-login {
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  user = nobody
}
ssl_cert = 

Re: Sieve vacation :addresses match only case-sensitive?

[Stephan Bosch via dovecot]

On 02/09/2019 15:05, Philipp Faeustlin via dovecot wrote:
The recipients in the vacation :addresses String list are matched 
case-sensitive.


If the recipient gets a mail with wrong case, the message is discarded 
with:
"discarding vacation response for implicitly delivered message; no 
known (envelope) recipient address found in message headers"


I think the matching should be case-insensitive.

Is this a bug or do I miss some option?

I'm using Dovecot 2.3.7.2, Pigeonhole version 0.5.7.2.


We're looking into this. Tracking internally as DOP-1424.

Regards,

Stephan.

Re: [Bug] Sieve vacation :addresses match only,> case-sensitive?

[Stephan Bosch via dovecot]

On 11/09/2019 21:25, Klaus Steinberger via dovecot wrote:

I want to have it case-insensitive again, like in Pigeonhole version 0.4.24

i would also plea for making it case-insensitive again! We were hit by this too,
and I never saw any mail system in which the local Part ist Case sensitive!


We're looking into this. Tracking internally as DOP-1424.

Regards,

Stephan.

Re: Submission and TLS client certificate

[Stephan Bosch via dovecot]

On 18/08/2019 10:09, Christian Rößner via dovecot wrote:

Hi,

is there some configuration parameter in Dovecot, which sends a TLS 
client certificate to the SMTP server? I would need this to have 
XCLIENT enabled and TLS with Postfix. This way I could permit sending 
based on the certificates fingerprint.


Can you elaborate? Are you talking about the submission relay service or 
submission of outgoing messages from e.g. Sieve?


Regards,

Stephan.


Thanks in advance

Christian

Von unterwegs aus gesendet
--
Rößner-Network-Solutions
Karl-Bröger-Str. 10, 36304 Alsfeld 
Fax: +49 6631 78823409 , Mobil: +49 171 
9905345 
USt-IdNr.: DE225643613, https://roessner.website 


PGP Fingerprint: 8FB3 132F 85D8 C9C7 A9F1  9A3F 5183 D46C B885 897E

Re: exim+dovecot+sieve issue

[Stephan Bosch via dovecot]

On 17/08/2019 15:37, JM via dovecot wrote:

Hi,

We have mail server in production, with Exim + Dovecot + Mailman + 
OpenLDAP + Roundcube webmail server.
Openldap keeps mail accounts in db, exim uses lmtp transport to 
deliver messages to dovecot.
Mailman is responsible for mailing lists, Roundcube webmail has 
managsieve module to create sieve filters.


Exim router,transport for dovecot

localuser:
  driver = accept
  domains = +local_domains : +virt_domains
  check_local_user
  transport = dovecot_lmtp
  cannot_route_message = Unknown account
  no_more


dovecot_lmtp:
  driver = lmtp
  socket = /var/spool/dovecot/lmtp
  batch_max = 500
  rcpt_include_affixes


It worked perfectly, but after upgrade from Dovecot 2.2 to Dovecot 2.3 
major version, I've found a lot of strange errors in exim panic.log


mail exim[7821]  Failed to get user name for uid 1175
mail exim[7880]: Failed to get user name for uid 1173

etc
where uids are ldap accounts and recognized by system (id 1175 shows 
user's name and group)


Even worse, some clients start to receive notify email with strange 
content, when sending email to mailing lists


u...@domain.com 
    Delay reason: error in redirect data: malformed address:
someus...@domain.com  may not follow
someus...@domain.com   in
"someus...@domain.com  ..some crap
here.." , where someuser addresses are not related to recipient
address and unknown for sender.


After some investigation,I've found out that dovecot's sieve produces 
these errors (I suspect sieve vacation, when user creates vacation via 
Roundcube webmail).


Found out how? It doesn't look like a Pigeonhole or Dovecot error 
message to me.


If set "submission_host = exim ip" in dovecot.conf, exim doesn't 
produce errors in panic.log anymore, but clients still receive email 
with malformed address.
Please help me to find the root of the problem (some broken sieve 
filter in user's maildir, misconfiguration or breaking changes in new 
Dovecot version?)


Do  you have an example of a malformed address?

What is your exact Dovecot configuration (output from `dovecot -n`).

Regards,

Stephan.

Re: sieve scripts not synching for 2.3.5.1 pre-built

[Stephan Bosch via dovecot]

On 04/05/2019 22:17, Marcel Menzel via dovecot wrote:

Hi,

same here, this doesn't seeem to be fixed with latest pigeonhole and 
dovecot release.


Regards,
Marcel

Am 03.05.2019 um 08:20 schrieb Piper Andreas via dovecot:

Hello,

Am 02.04.19 um 21:51 schrieb Timo Sirainen via dovecot:

On 2 Apr 2019, at 22.37, Timo Sirainen via dovecot mailto:dovecot@dovecot.org>> wrote:

On 2 Apr 2019, at 17.03, Jan-Pieter Cornet via dovecot
mailto:dovecot@dovecot.org>> wrote:

Hi,

We're synching mailboxes, changing format from maildir to mdbox,
using doveadm backup/doveadm sync.

When still running 2.2.36, 'doveadm backup' also synched the sieve
scripts, without issues.

After the upgrade to 2.3.5.1, the sieve sync stopped working. We're
using the pre-built 2.3 packages from
https://repo.dovecot.org/ce-2.3-latest/debian/stretch

Looks like this is trivial to reproduce. It used to work still in
v2.3.1, but then something broke it. Tracking internally in DOP-1062.

Revertinghttps://github.com/dovecot/pigeonhole/commit/479c5e57046dec76078597df844daccbfc0eb75f
  fixes
this.


has this been fixed in 2.3.6 ?

In my first tests sieve scripts still seem not to be replicated with
dovecot-2.3.6


Should be fixed in 2.3.7.

Regards,

Stephan.

Re: Bug with latest GCC 9

[Stephan Bosch via dovecot]

On 25/04/2019 12:52, Martin Liška via dovecot wrote:

On 1/25/19 8:24 PM, Stephan Bosch wrote:


Op 25/01/2019 om 10:59 schreef Martin Liška:

Hi.

As mentioned here:
https://bugzilla.opensuse.org/show_bug.cgi?id=1123136

there's a new issue with GCC 9, it's related to:
https://gcc.gnu.org/gcc-9/porting_to.html#complit

in:
/* Not const! Never return this as a result directly! */
#define SMTP_ADDRESS_LITERAL(localpart, domain) \
 &((struct smtp_address){ (localpart), (domain) })

Thanks for reporting this. We are working on it (tracking internally as 
DOP-890).

Regards,

Stephan.

Hi.

Thanks for working on that. Is there any update please?


Fixed in 2.3.7.

Regards,

Stephan.

Re: IMAP copy stopped copying flags

[Stephan Bosch via dovecot]

On 20/01/2019 13:46, Stephan Bosch wrote:



Op 11/01/2019 om 12:42 schreef Stephan Bosch:


Op 11-1-2019 om 10:53 schreef Marc Weustink:

Hi all,

Andrzej A. Filip wrote:

Timo Sirainen  wrote:
On 9 Jul 2018, at 16.49, Andrzej A. Filip 
 wrote:


Is it intended behavior?


No.


It seems to be caused by upgrade to 1:2.3.2-2 on Debian/Testing.


What was the old version? What's your doveconf -n? How are you 
testing

that it's not working?


It seems that the problem occurred after Debian/Testing dovecot's 
upgrade

from 1:2.2.36-1 to 1:2.3.2-2.  I have fixed the problem by downgrading
dovecot to version provided by Debian/Stable (1:2.2.27-3+deb9u2).

I use dovecot in "server less" mode (IMAP via stdin+stdout) with 
maildir.


mail_location = maildir:~/Maildir:LAYOUT=fs




Same issue here on 2.3.4 (0ecbaf23d) as server.

When a message is moved, either through Thunderbird or doveadm move, 
the flags are lost.


Any update on this ?


We're working on it. We have reproduced the problem without 
difficulty and isolated the commit that caused it.


Tracking internally as DOP-842.


Fixed in 2.3.7.

Regards,

Stephan.

Re: submission configuration issues

[Stephan Bosch via dovecot]

On 27/07/2019 23:13, Stephan Bosch via dovecot wrote:



On 23/07/2019 17:13, Jean-Daniel Dupas via dovecot wrote:

Hello,

I'm having trouble configuring the submission proxy.

I have configured the submission service as follow:

submission_host = smtp.example.com
submission_relay_host = localhost
submission_relay_port = 8587
submission_relay_rawlog_dir = /var/log/dovecot/
submission_relay_trusted = yes

My main issue is that until I login, dovecot-submission won't connect 
to the backend and query the capabilities and so won't report the 
right capabilities.


That is true and expected. No connection to the relay server is made 
until the user is logged in.


That mean that the first EHLO message don't get the right 
capabilities list.


"
EHLO example.com

250-smtp.example.com
250-8BITMIME
250-AUTH PLAIN LOGIN
250-BURL imap
250-CHUNKING
250-ENHANCEDSTATUSCODES
250-SIZE
250 PIPELINING
"

This list don't contains VRFY, DNS, and SIZE is not specified (all of 
these is present in backend EHLO response).
After login, if I send an new EHLO command, everything is properly 
reported. The raw log shows that unlike what the documentation says,
dovecot don't try to connect to the backend until the user is 
properly logged.

Oh, then we need to adjust the documentation. This is normal behavior.

In my raw log I show that after I logged in dovecot-submission, the 
later open a connection to the backend and send a X-CLIENT command.



Now, if I try to force the capabilities by using:

submission_backend_capabilities = VRFY 8BITMIME DSN

dovecot properly reports all SMTP capabilities in the first EHLO 
response, but it completely stops emitting X-CLIENT command to the 
backend
and try to simply forward the command without authentication, which 
result in postfix rejecting the command with an unauthorized user error.


Yes, that is a bug. I have reproduced it here. We will look into it.


Tracking this bug as DOP-1323.

Regards,

Stephan.

Re: submission configuration issues

[Stephan Bosch via dovecot]

On 23/07/2019 17:13, Jean-Daniel Dupas via dovecot wrote:

Hello,

I'm having trouble configuring the submission proxy.

I have configured the submission service as follow:

submission_host = smtp.example.com
submission_relay_host = localhost
submission_relay_port = 8587
submission_relay_rawlog_dir = /var/log/dovecot/
submission_relay_trusted = yes

My main issue is that until I login, dovecot-submission won't connect to the 
backend and query the capabilities and so won't report the right capabilities.


That is true and expected. No connection to the relay server is made 
until the user is logged in.



That mean that the first EHLO message don't get the right capabilities list.

"
EHLO example.com

250-smtp.example.com
250-8BITMIME
250-AUTH PLAIN LOGIN
250-BURL imap
250-CHUNKING
250-ENHANCEDSTATUSCODES
250-SIZE
250 PIPELINING
"

This list don't contains VRFY, DNS, and SIZE is not specified (all of these is 
present in backend EHLO response).
After login, if I send an new EHLO command, everything is properly reported. 
The raw log shows that unlike what the documentation says,
dovecot don't try to connect to the backend until the user is properly logged.

Oh, then we need to adjust the documentation. This is normal behavior.


In my raw log I show that after I logged in dovecot-submission, the later open 
a connection to the backend and send a X-CLIENT command.


Now, if I try to force the capabilities by using:

submission_backend_capabilities = VRFY 8BITMIME DSN

dovecot properly reports all SMTP capabilities in the first EHLO response, but 
it completely stops emitting X-CLIENT command to the backend
and try to simply forward the command without authentication, which result in 
postfix rejecting the command with an unauthorized user error.


Yes, that is a bug. I have reproduced it here. We will look into it.

Regards,

Stephan.

Re: submission configuration issues

[Stephan Bosch via dovecot]

On 23/07/2019 17:13, Jean-Daniel Dupas via dovecot wrote:

Hello,

I'm having trouble configuring the submission proxy.

I have configured the submission service as follow:

submission_host = smtp.example.com
submission_relay_host = localhost
submission_relay_port = 8587
submission_relay_rawlog_dir = /var/log/dovecot/
submission_relay_trusted = yes

My main issue is that until I login, dovecot-submission won't connect to the 
backend and query the capabilities and so won't report the right capabilities.

That mean that the first EHLO message don't get the right capabilities list.

"
EHLO example.com

250-smtp.example.com
250-8BITMIME
250-AUTH PLAIN LOGIN
250-BURL imap
250-CHUNKING
250-ENHANCEDSTATUSCODES
250-SIZE
250 PIPELINING
"

This list don't contains VRFY, DNS, and SIZE is not specified (all of these is 
present in backend EHLO response).
After login, if I send an new EHLO command, everything is properly reported. 
The raw log shows that unlike what the documentation says,
dovecot don't try to connect to the backend until the user is properly logged.

In my raw log I show that after I logged in dovecot-submission, the later open 
a connection to the backend and send a X-CLIENT command.


Now, if I try to force the capabilities by using:

submission_backend_capabilities = VRFY 8BITMIME DSN

dovecot properly reports all SMTP capabilities in the first EHLO response, but 
it completely stops emitting X-CLIENT command to the backend
and try to simply forward the command without authentication, which result in 
postfix rejecting the command with an unauthorized user error.

What is wrong with my configuration ?
Thanks.


Can you send us your complete configuration (output from `dovecot -n`)?

Regards,

Stephan.

Re: Sieve question

[Stephan Bosch via dovecot]

On 03/07/2019 04:44, @lbutlr via dovecot wrote:

I have the following in my active sieve file, and there are no errors logged.


if header :contains "to" "+root" {
setflag "\\Seen";
fileinto :create "root";
stop;
}

The message is put in .root, bit is not marked as seen.

Is the default action to put mail in a folder matching the extension taking 
precedence?


That should work. What version is this (output from `dovecot -`n`)? 
There have been some bugs with flags in the recent history.


Regards,

Stephan.

Re: recipient delimiters

[Stephan Bosch via dovecot]

On 20/06/2019 01:20, @lbutlr via dovecot wrote:

On 18 Jun 2019, at 15:03, @lbutlr via dovecot  wrote:

I don’t see a way to tell dovecot what delimiters to use, and it appears it is 
still using a single delimiter only despite postfix having added support for 
more than one years ago.

Ideas?


There is the recipient_delimiter setting. For recent versions, this is a 
list of characters recognized as such.


Regards,

Stephan.

Re: RFC 8314 § 3.3 support at Dovecot submission proxy service?

[Stephan Bosch via dovecot]

On 10/06/2019 01:51, Robert Scheck via dovecot wrote:

Hello all,

I'm currently wondering whether Dovecot submission proxy service supports
RFC 8314 § 3.3 as well, which means implicit TLS for SMTP submission.

https://github.com/dovecot/core/blob/2cbbe9b4829adb184c83dbf780316f4144559054/doc/example-config/conf.d/10-master.conf#L48
from Git master just mentions port 587 like this:

service submission-login {
   inet_listener submission {
 #port = 587
   }
}

As per RFC 8314 § 3.3, I would expect the following to work (which I did
not try), and if it already works, I even would expect this snippet part
of the future Dovecot example/default configuration:

service submission-login {
   inet_listener submission {
 #port = 587
   }
   inet_listener submissions {
 #port = 465
 #ssl = yes
   }
}

And if it isn't supported yet, I would indeed like to raise this as feature
request. Note that 465 is named "submissions" as per RFC 8314 § 7.3.


That should work. Just try it.

Regards,

Stephan.

Re: Case sensitive problem for vacation with dovecot lmtp pigeonhole

[Stephan Bosch via dovecot]

On 09/06/2019 16:07, kadafax--- via dovecot wrote:

Hi list,

This issue was raised here by John Fawcett on 12/02/19 but got no 
response.

So, as I'm facing the same issue, I would like to try again:

When using the vacation sieve extension, if I send an email to 
user.n...@mydomain.com it will work, but not to 
user.n...@mydomain.com. Is there a way to make the user part case 
insensitive so that the vacation auto-reply is fired whatever the way 
user part is written ?




Hmm, I need to look into this.

Regards,

Stephan.


doveconf -n output:

# 2.3.5.1 (7ec6d0ade): /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.5 (2483b085)
# OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 
7.6.1810 (Core)  ext4

# Hostname: myserver.mydomain.com
auth_master_user_separator = *
auth_username_chars = 
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*

auth_verbose = yes
debug_log_path = /var/log/dovecot-debug.log
default_client_limit = 10640
default_process_limit = 2128
default_vsz_limit = 0
hostname = mydomain.com
info_log_path = /var/log/dovecot.log
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *
log_path = /var/log/dovecot-error.log
login_greeting = Welcome
mail_gid = vmail
mail_location = maildir:/var/vmail/%Ln
mail_plugins = quota fts fts_lucene
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext

namespace inbox {
  inbox = yes
  location = maildir:/var/vmail/%Ln
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  prefix =
  type = private
}
passdb {
  args = /usr/local/etc/dovecot/deny-users
  deny = yes
  driver = passwd-file
}
passdb {
  args = /usr/local/etc/dovecot/master-users
  driver = passwd-file
  master = yes
  pass = yes
}
passdb {
  args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars=@.
  quota = maildir:User quota
  quota_rule = *:storage=30GB
  quota_rule2 = Trash:storage=+200M
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = /var/sieve-scripts/%Ln.sieve
  sieve_dir = /var/vmail/%Ln/home/sieve
}
postmaster_address = postmas...@mydomain.com
protocols = imap pop3 lmtp sieve sieve
quota_full_tempfail = yes
service auth-worker {
  user = $default_internal_user
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0777
    user = vmail
  }
}
service imap-login {
  inet_listener imaps {
    port = 993
    ssl = yes
  }
  process_min_avail = 10
}
service imap {
  process_limit = 2128
}
service lmtp {
  executable = lmtp -L
  process_min_avail = 15
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}
service pop3-login {
  inet_listener pop3s {
    port = 995
    ssl = yes
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  user = nobody
}
ssl_cert = 

Re: failed to pipe to program sa-learn-spam.sh

[Stephan Bosch via dovecot]

Op 30-5-2019 om 9:46 schreef @lbutlr via dovecot:

Error: program `/usr/lib/dovecot/sieve/sa-learn-spam.sh' terminated abnormally, 
signal 11
Error: sieve: pipe action: failed to pipe message to program 
`sa-learn-spam.sh': refer to server log for more information. [2019-05-29 
17:59:12]

What server log? This is all that get logged.


Output from dovecot -n is needed to answer this one. Does it crash when 
you invoke sa-learn-spam.sh manually from shell?


Regards,

Stephan.


  # pwd
/usr/lib/dovecot/sieve

  # ls -lsa

total 96
8 drwxrwxrwx  3 root wheel  512 Jun 18  2018 .
8 drwxr-xr-x  3 root wheel  512 May 17 17:54 ..
8 -rw-r--r--  1 root wheel   86 Jun 15  2018 default.sieve
8 drwxr-xr-x  2 root wheel  512 Jun 13  2018 global
8 -rw-r--r--  1 root wheel   62 May 14  2018 mark-read.sieve
8 -rw-r--r--  1 kremels  wheel  192 Jun 17  2018 mark-read.svbin
8 -rwxr-xr-x  1 root wheel  314 Feb 12  2018 report-ham.sieve
8 -rw-r--r--  1 lbutler  wheel  448 Jun 18  2018 report-ham.svbin
8 -rwxr-xr-x  1 root wheel  199 Feb 12  2018 report-spam.sieve
8 -rw-r--r--  1 kremewheel  354 Jun 15  2018 report-spam.svbin
8 -rwxr-xr-x  1 root wheel  131 Jun 18  2018 sa-learn-ham.sh
8 -rwxr-xr-x  1 root wheel   54 Feb 12  2018 sa-learn-spam.sh

  # cat sa-learn-spam.sh

#!/bin/sh
exec /usr/local/bin/sa-learn -u ${1} --spam

  #  ls -ls /usr/local/bin/sa-learn

96 -rwxr-xr-x  1 root  wheel  45939 May 29 09:59 /usr/local/bin/sa-learn

report-spam.sieve:

require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.user" "*" {
   set "username" "${1}";
}

pipe :copy "sa-learn-spam.sh" [ "${username}" ];

in doveconf I have:


managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave

plugin {
   imapsieve_mailbox1_before = file:/usr/lib/dovecot/sieve/report-spam.sieve
   imapsieve_mailbox1_causes = COPY
   imapsieve_mailbox1_name = Junk
   …

It WAS working.

Re: regression with lmtp_hdr_delivery_address in 2.3

[Stephan Bosch via dovecot]

On 30/04/2019 07:47, dovecot-ffe7--- via dovecot wrote:

Hi,

In my LMTP config, I have

lmtp_hdr_delivery_address = none.

Under 2.2, Received headers looked like this:

Received: from smtp.example.com
    by lmtp.example.com with LMTP id +IVdKS81cFwWagAAGhmRSA
    ; Fri, 22 Feb 2019 17:45:19 +

After the update to 2.3, they look like this:

Received: from smtp.example.com
    by lmtp.example.com with LMTP
    id MJbKAMFEclyKCQAAGhmRSA
    (envelope-from )
    for ; Sun, 24 Feb 2019 07:16:17 +

In both cases I *do not* have a "Delivered-To" header

I do see a check for rcpt_to == NULL in the 
smtp_server_transaction_write_trace_record function, but it looks like 
the lmtp_local_add_headers function does not update the transaction, 
so it doesn't get pulled in. Also, the LMTP settings struct isn't 
exposed in that function currently, and I'm not sure how to add it.


In 2.2, both the "Delivered-To" and "Received" headers were handled by 
client_get_added_headers.


Please CC me in replies, as I am not subscribed to the list.


I'll give this a look.

Regards,

Stephan.

Re: 2.3.6 lib-smtp test failure in CentOS 6

[Stephan Bosch via dovecot]

On 02/05/2019 06:52, Peter via dovecot wrote:

After applying the patches in my previous message...

I'm getting the following when building dovecot for CentOS 6 (but not 
for CentOS 7):



lmtp payload - normal: parallel pipelining ssl ... 
: ok
CLIENT: Panic: file smtp-client-connection.c: line 1309 
(smtp_client_connection_established): assertion failed: 
(!conn->connect_succeeded)
CLIENT: Error: Raw backtrace: 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x4547ea] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x454891] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(_start+0) 
[0x4193f8] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x41fb95] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x4203f0] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x421d77] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(smtp_client_command_input_reply+0x17d) 
[0x42ffdd] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x42145b] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(io_loop_call_io+0x61) 
[0x461d81] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(io_loop_handler_run_internal+0xd6) 
[0x463c16] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(io_loop_handler_run+0x59) 
[0x461e79] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(io_loop_run+0x38) 
[0x4620a8] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x41a619] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x41aaa8] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x41ab7d] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x44beae] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(test_run+0x11) 
[0x44c011] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload(main+0x115) 
[0x419d65] -> /lib64/libc.so.6(__libc_start_main+0x100) 
[0x7f3850d4ad20] -> 
/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp/.libs/lt-test-smtp-payload() 
[0x419421]

/bin/sh: line 1: 28384 Aborted ./$bin
make[3]: Leaving directory 
`/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp'

make[3]: *** [check-local] Error 1
make[2]: *** [check-am] Error 2
make[2]: Leaving directory 
`/builddir/build/BUILD/dovecot-2.3.6/src/lib-smtp'

make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/dovecot-2.3.6/src'
make: *** [check-recursive] Error 1
error: Bad exit status from /var/tmp/rpm-tmp.9PlfRJ (%check)


Can you get a GDB backtrace for this?

Regards,

Stephan.

Re: http-lib test failures when building dovecot-2.3.5 and later in mock builder

[Stephan Bosch via dovecot]

On 02/05/2019 22:54, Tuomo Soini via dovecot wrote:

There is random failure in test-http-payload when building rpm package
from 2.3.6. I couldn't reproduce that in normal system but that happens
something like every second try in mock chroot build envirnoment. Other
tests don't have issues so it looks like test is not very reliable.
Building 2.3.4 didn't yet have this issue.

./test-http-payload -D output attached.


Yes, we have seen this before. It is being looked at.

Regards,

Stephan.

Re: haproxy + submission services -> postfix failure

[Stephan Bosch via dovecot]

On 19/04/2019 19:58, Chris Thomas via dovecot wrote:

Hi,

I have a nginx server which is using the proxy protocol to forward tcp
connections to dovecot. Dovecot is configured to be a submission
service for email to be sent. Then postfix should send the email
itself which is also using the ha proxy protocol. There are a few
moving parts in this problem so I'm not sure where the problem is. But
I want to ask if somebody can validate my dovecot configuration
somehow so I can start to tick off some things from the list.

Sending email fails, seems to get to postfix, then die
Receiving emails succeeds and I don't have any problem to pick them up.

I've figured out some stuff, like lmtp shouldn't use haproxy when
talking between postfix -> dovecot for receiving emails. If I enable
the protocol on lmtp, I can't receive any emails at all.

In order to get postfix to accept emails, I enabled haproxy protocol
and enabled postscreen and then postfix could access the source ip and
stop my server from being an open relay.

I've got tls certificates installed on dovecot and postfix, all
created by letsencrypt and I don't appear to have any problems with
them.

I will try to give as much information about the config as I can, I'm
not sure what other parts are good to have, but let me know if you are
missing something or want to check a value.


You enabled haproxy both for nginx<->dovecot and dovecot<->postfix. 
Dovecot does not support haproxy for outgoing connections, so the latter 
wont work.


Regards,

Stephan.




10-master.conf:

service submission-login {
   inet_listener submission {
 port = 587
 haproxy = yes
   }
}

service lmtp {
   inet_listener lmtp {
 port = 24
 haproxy = no
   }
}



20-submission.conf

submission_relay_host = postfix.mail-server
submission_relay_port = 25
submission_relay_ssl = starttls
submission_relay_ssl_verify = yes

Then because it might help to give the other side of the connection
configuration for postfix, here is the relevant information:


master.cf:

smtp  inet  n   -   -   -   1   postscreen
smtpd pass  -   -   -   -   -   smtpd


main.cf

postscreen_upstream_proxy_protocol = haproxy
postscreen_upstream_proxy_timeout = 10s

That's it. I don't know what other information could be useful.

There are some logs, they are like this (I've got logging turned on
for pretty much every option I have:

Dovecot logs:

Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
Added userdb setting: plugin/quota_rule=*:bytes=0
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
Effective uid=8, gid=8, home=/mail/__DOMAIN_COM__/__USER__
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no,
list=yes, subscriptions=yes
location=maildir:/mail/__DOMAIN_COM__/__USER__
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
maildir++: root=/mail/__DOMAIN_COM__/__USER__, index=, indexpvt=,
control=, inbox=/mail/__DOMAIN_COM__/__USER__, alt=
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-server: conn __IP_ADDR_1__:31217 [0]: Connection created
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connection created
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Looking up IP address
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: DNS lookup successful;
got 1 IPs
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connecting to
10.104.211.161:25
Apr 19 17:54:47 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connected
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Received greeting from
server: 421 4.3.2 No system resources
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Connection failed: 421
4.3.2 No system resources
Apr 19 17:54:57 submission(__EMAIL__)<497>: Error:
Failed to establish relay connection: 421 4.3.2 No system resources
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-client: conn postfix.mail-server:25 [0]: Disconnected
Apr 19 17:54:57 submission(__EMAIL__)<497>: Info:
Disconnect from __IP_ADDR_1__: Failed to establish relay connection
in=0 out=22 (state=GREETING)
Apr 19 17:54:57 submission(__EMAIL__)<497>: Debug:
smtp-server: conn __IP_ADDR_1__:31217 [0]: Disconnected: Failed to
establish relay connection

Postfix Logs:
postfix/postscreen[525]: warning: haproxy read: time limit exceeded

If anybody could help out, I'd be grateful because I just can't see
what the problem is.

Chris

Re: lmtp_save_to_detail_mailbox override

[Stephan Bosch via dovecot]

On 12/04/2019 17:06, Andreas Thienemann via dovecot wrote:

Hi,

I have a dovecot 2.3.2.1 instance running here that is getting userdb 
and passdb data from a SQL database. This was a cyrus server migrated 
over and all users are happy.


All users except one: Said user asked if it would be possible to 
disable subaddressing (localuser+fol...@example.com) again.


The setting controlling that is AFAIK the lmtp_save_to_detail_mailbox 
item.


My idea was to override that setting for the specific user. I changed 
the user_query to return no for the user in question and all seems well:


# doveadm user test
field    value
user    t...@example.com
uid    vmail
gid    vmail
home    /var/lib/imap/example.com/test
mail    mdbox:/var/spool/imap/user/example.com/test
lmtp_save_to_detail_mailbox    no
#

Looks good, doesn't work though. Mail is still delivered to the 
+folder. Is this per user setting not taken into account?


No, unfortunately that setting is global and read at client connection 
before the user is looked up for the RCPT command.


Regards,

Stephan.

Re: pigeonhole tests crashing in deleteheader.svtest

[Stephan Bosch via dovecot]

On 18/04/2019 15:01, Michal Hlavinka via dovecot wrote:

On 4/12/19 12:48 AM, Stephan Bosch wrote:



On 29/03/2019 10:23, Michal Hlavinka via dovecot wrote:

On 3/28/19 6:41 PM, Aki Tuomi via dovecot wrote:


On 28 March 2019 19:40 Michal Hlavinka via dovecot 
 wrote:


  Hi,

when trying to build dovecot 2.3.5.1 pigeonhole testsuite crashes in



Which version of pigeonhole are you using?


latest available - 0.5.5



Hmm, what platform are you compiling this on and what compiler are 
you using?


Fedora Linux, x86_64, kernel 5.0.6, glibc 2.29.9000, gcc 9.0.1
Looking at the backtrace, it seems it's probably related to
https://dovecot.org/pipermail/dovecot/2019-January/114494.html



Indeed, tracking as DOP-890.

Regards,

Stephan.

Re: pigeonhole tests crashing in deleteheader.svtest

[Stephan Bosch via dovecot]

On 29/03/2019 10:23, Michal Hlavinka via dovecot wrote:

On 3/28/19 6:41 PM, Aki Tuomi via dovecot wrote:


On 28 March 2019 19:40 Michal Hlavinka via dovecot 
 wrote:


  Hi,

when trying to build dovecot 2.3.5.1 pigeonhole testsuite crashes in



Which version of pigeonhole are you using?


latest available - 0.5.5



Hmm, what platform are you compiling this on and what compiler are you 
using?


Regards,

Stephan.

Re: Dovecot 2.3.4 crash

[Stephan Bosch via dovecot]

Op 01/12/2018 om 00:45 schreef azu...@pobox.sk:


Citát azu...@pobox.sk:


Citát azu...@pobox.sk:


Citát Timo Sirainen :


On 29 Nov 2018, at 15.46, azu...@pobox.sk wrote:


Hi,

is this a know problem? Newest Dovecot 2.3.4 package from 
repo.dovecot.org , Debian Stretch (fully upgraded).




Nov 29 14:25:11 server00 dovecot: lmtp(16854): Panic: file 
ostream-dot.c: line 208 (o_stream_dot_sendv): assertion failed: 
((size_t)ret == sent + added)


Is this proxying LMTP traffic?





Yes, it is. The main server is, currently, doing standard 
IMAP/POP3/LMTP for some clients and also proxying IMAP/POP3/LMTP for 
other clients (proxying to other servers). But this one error is 
related to one message which is stucked in postfix queue and cannot 
be delivered - the error appears everytime i do 'postfix flush' 
(error in postfix queue is 'lost connection with 
[private/dovecot-lmtp] while sending 
end of data -- message may be sent more than once').


There are no errors logged on proxy backend side.
Just to clarify this - that one message is to be proxied to other 
server.


Anything new to this? Thank you.


Fix is included in 2.3.5.

Regards,

Stephan.

Re: Crash in director for secure lmtp proxy connections

[Stephan Bosch via dovecot]

Op 21/11/2018 om 14:21 schreef Marcus Fenner:

Hi,

After updating some of our director servers from 2.2.36 to 2.3.3
(dcead646b) we experience crashes in about 1/5000 mails in lmtp delivery:

Nov 13 12:23:36 Fatal: lmtp(113620): master: service(lmtp): child 113620
killed with signal 6 (core dumped)
Nov 13 12:23:36 Panic: lmtp(113623): file ostream-dot.c: line 208
(o_stream_dot_sendv): assertion failed: ((size_t)ret == sent + added)
Nov 13 12:23:36 Error: lmtp(113623): Raw backtrace:
/usr/lib64/dovecot/libdovecot.so.0(+0xe34fb) [0x7f31d89134fb] ->
/usr/lib64/dovecot/libdovecot.so.0(+0xe3597) [0x7f31d8913597] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x51207) [0x7f31d8881207] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x4eeef) [0x7f31d887eeef] ->
/usr/lib64/dovecot/libdovecot.so.0(+0x107cf8) [0x7f31d8937cf8] ->
/usr/lib64/dovecot/libdovecot.so.0(o_stream_sendv+0x32) [0x7f31d89382c2]
-> /usr/lib64/dovecot/libdovecot.so.0(io_stream_copy+0x5a)
[0x7f31d8938b6a] ->
/usr/lib64/dovecot/libdovecot.so.0(o_stream_send_istream+0x53)
[0x7f31d8938873] ->
/usr/lib64/dovecot/libdovecot.so.0(smtp_client_command_send_more+0x145)
[0x7f31d8890805] -> /usr/lib64/dovecot/libdovecot.so.0(+0x660dd)
[0x7f31d88960dd] ->
/usr/lib64/dovecot/libssl_iostream_openssl.so(+0xa7e1) [0x7f31d864a7e1]
-> /usr/lib64/dovecot/libdovecot.so.0(+0x10a558) [0x7f31d893a558] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x73)
[0x7f31d892a203] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x136)
[0x7f31d892b8b6] ->
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x50)
[0x7f31d892a2b0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x48)
[0x7f31d892a418] ->
/usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x17)
[0x7f31d88a95a7] -> dovecot/lmtp(main+0x245) [0x555b8b59cea5] ->
/lib64/libc.so.6(__libc_start_main+0xf3) [0x7f31d867c413] ->
dovecot/lmtp(_start+0x2e) [0x555b8b59cfbe]
Nov 13 12:23:36 Fatal: lmtp(113623): master: service(lmtp): child 113623
killed with signal 6 (core dumped)

On the backend servers the only message is
lmtp(9571): Disconnect from 192.168.1.56: Connection closed (in DATA)

This happens for every server with this version every time postfix tries
to resend the mail.

After removing "starttls=y" from the static passdb entry, the mail is
sent successfully. Starting with version 2.3.2 the lmtp proxy uses
starttls as well, therefore the error seems to be there. I've not been
able to figure out, if this has to do with the content of the mails,
because they are from many different sources.


Fix is included in 2.3.5.

Regards,

Stephan.

Re: lmtp 2.3.2.1 segfault with backtrace

[Stephan Bosch via dovecot]

Hi,

Op 16/08/2018 om 00:53 schreef Stephan Bosch:

Hi,

I have reproduced this problem and I am working on a fix.


Fix is included in 2.3.5.

Regards,

Stephan.


Op 14/08/2018 om 11:44 schreef Tom Sommer:

lmtp on Director crash with 2.3.2.1

# gdb /usr/libexec/dovecot/lmtp /var/core/60174
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 


This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show 
copying"

and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/libexec/dovecot/lmtp...Reading symbols from 
/usr/lib/debug/usr/libexec/dovecot/lmtp.debug...done.

done.
[New Thread 60174]
Reading symbols from /usr/lib64/dovecot/libdovecot-lda.so.0...Reading 
symbols from 
/usr/lib/debug/usr/lib64/dovecot/libdovecot-lda.so.0.0.0.debug...done.

done.
Loaded symbols for /usr/lib64/dovecot/libdovecot-lda.so.0
Reading symbols from 
/usr/lib64/dovecot/libdovecot-storage.so.0...Reading symbols from 
/usr/lib/debug/usr/lib64/dovecot/libdovecot-storage.so.0.0.0.debug...done. 


done.
Loaded symbols for /usr/lib64/dovecot/libdovecot-storage.so.0
Reading symbols from /usr/lib64/dovecot/libdovecot.so.0...Reading 
symbols from 
/usr/lib/debug/usr/lib64/dovecot/libdovecot.so.0.0.0.debug...done.

done.
Loaded symbols for /usr/lib64/dovecot/libdovecot.so.0
Reading symbols from /lib64/libc.so.6...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/librt.so.1...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libdl.so.2...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging 
symbols found)...done.

Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols 
found)...done.

[Thread debugging using libthread_db enabled]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from 
/usr/lib64/dovecot/libssl_iostream_openssl.so...Reading symbols from 
/usr/lib/debug/usr/lib64/dovecot/libssl_iostream_openssl.so.debug...done.

done.
Loaded symbols for /usr/lib64/dovecot/libssl_iostream_openssl.so
Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols 
found)...done.

Loaded symbols for /usr/lib64/libssl.so.10
Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging 
symbols found)...done.

Loaded symbols for /usr/lib64/libcrypto.so.10
Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging 
symbols found)...done.

Loaded symbols for /lib64/libgssapi_krb5.so.2
Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libkrb5.so.3
Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libcom_err.so.2
Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libk5crypto.so.3
Reading symbols from /lib64/libz.so.1...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libz.so.1
Reading symbols from /lib64/libkrb5support.so.0...(no debugging 
symbols found)...done.

Loaded symbols for /lib64/libkrb5support.so.0
Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libkeyutils.so.1
Reading symbols from /lib64/libresolv.so.2...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libresolv.so.2
Reading symbols from /lib64/libselinux.so.1...(no debugging symbols 
found)...done.

Loaded symbols for /lib64/libselinux.so.1
Core was generated by `dovecot/lmtp'.
Program terminated with signal 11, Segmentation fault.
#0  smtp_client_command_set_replies (cmd=0x0, replies=1) at 
smtp-client-command.c:401

401 i_assert(cmd->replies_expected == 1 ||
Missing separate debuginfos, use: debuginfo-install 
glibc-2.12-1.212.el6.x86_64 keyutils-libs-1.4-5.el6.x86_64 
krb5-libs-1.10.3-65.el6.x86_64 libcom_err-1.41.12-24.el6.x86_64 
libselinux-2.0.94-7.el6.x86_64 openssl-1.0.1e-57.el6.x86_64 
zlib-1.2.3-29.el6.x86_64

(gdb) bt full
#0  smtp_client_command_set_replies (cmd=0x0, replies=1) at 
smtp-client-command.c:401

    __func__ = "smtp_client_command_set_replies"
#1  0x7f5d14f40f3f in smtp_client_transaction_data_cb 
(reply=0x7ffe5cd19650, trans=0x7f5d176ae2b8) at 
smtp-client-transaction.c:658

    conn = 0x7f5d176ade80
    rcpt = 0x7f5d176ae560
    i = 
    count = 1
#2  0x7f5d14f3e941 in smtp_client_command_fail_reply (_cmd=optimized out>, reply=0x7ffe5cd19650) at smtp-client-command.c:299

    cmd = 0x7f5d17600d18
    tmp_cmd = 
    conn = 

Re: 2.3.3: Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion failed

[Stephan Bosch via dovecot]

Op 02/10/2018 om 11:26 schreef Tom Sommer:

I see this in my logs after 2.3.3:

using zlib plugin, ofc.

Oct 02 10:01:39 imap(u...@example.com)<50643>: 
Panic: file ostream-zlib.c: line 37 (o_stream_zlib_close): assertion 
failed: (zstream->ostream.finished || 
zstream->ostream.ostream.stream_errno != 0 || 
zstream->ostream.error_handling_disabled)
Oct 02 10:01:39 imap(u...@example.com)<50643>: 
Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xce56a) 
[0x7f442487556a] -> /usr/lib64/dovecot/libdovecot.so.0(+0xce5b1) 
[0x7f44248755b1] -> /usr/lib64/dovecot/libdovecot.so.0(+0x3d941) 
[0x7f44247e4941] -> /usr/lib64/dovecot/lib20_zlib_plugin.so(+0x5cdf) 
[0x7f44233c4cdf] -> /usr/lib64/dovecot/libdovecot.so.0(+0xf4b46) 
[0x7f442489bb46] -> 
/usr/lib64/dovecot/libdovecot.so.0(o_stream_destroy+0x13) 
[0x7f442489be83] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(maildir_save_finish+0x173) 
[0x7f4424b93973] -> 
/usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_save_cancel+0x36) 
[0x7f4424b68696] -> dovecot/imap [u...@example.com X.X.X.X 
APPEND](+0xede9) [0x55a9d7e86de9] -> dovecot/imap [u...@example.com 
X.X.X.X APPEND](command_exec+0x65) [0x55a9d7e956e5] -> dovecot/imap 
[u...@example.com X.X.X.X APPEND](+0xf9e6) [0x55a9d7e879e6] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x55) 
[0x7f442488c275] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xbf) 
[0x7f442488e13f] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x55) 
[0x7f442488c365] -> 
/usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f442488c588] 
-> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f4424808053] -> dovecot/imap [u...@example.com X.X.X.X 
APPEND](main+0x32d) [0x55a9d7ea20dd] -> 
/lib64/libc.so.6(__libc_start_main+0x100) [0x7f4424431d20] -> 
dovecot/imap [u...@example.com X.X.X.X APPEND](+0xe419) [0x55a9d7e86419]


No clue how to reproduce


Fix is included in 2.3.5.

Regards,

Stephan.

Re: submission-login: Fatal: master: service(submission-login):

[Stephan Bosch via dovecot]

Op 11/03/2019 om 12:53 schreef Marcelo Coelho via dovecot:

Hi everyone!

I’m using dovecot 2.3.5. submission-login is crashing many times in a day:

Here is a sample error message:

dovecot: submission-login: Fatal: master: service(submission-login): child 
34247 killed with signal 11 (core not dumped - 
https://dovecot.org/bugreport.html#coredumps - set service submission-login { 
drop_priv_before_exec=yes })

After I added drop_priv_before_exec, I got these error messages:

submission-login: Error: master-service: cannot get filters: 
net_connect_unix(/var/run/dovecot/config) failed: Permission denied
dovecot: master: Error: service(submission-login): command startup failed, 
throttling for 2 secs

How to get submission core dumps properly?

Thank you.


We got a fix for this bug in off-list discussions.

Tracking as DOP-987.

Regards,

Stephan.

Re: Sieve scripts not triggered on IMAP inbound messages using IMAPC

[Stephan Bosch via dovecot]

Op 08/03/2019 om 15:12 schreef Fred via dovecot:


Good morning.

I am using Dovecot as an IMAP proxy -- using IMAPC -- and the system 
is working great. Thank you for contributing such great software to 
the community!


I’m using Dovcot 2.3.5 and Pigeonhole 0.5.5.

My server is located behind a content-filtering firewall that will 
mark incoming messages by adding “[SPAM]” to the subject line. I’m 
trying to use Sieve to identify those messages and move them into the 
Spam folder. I have a basic Sieve script that looks like this:


   require ["fileinto"];

   if anyof (header :contains "Subject" "[SPAM]",

     header :contains "X-Spam-Flag" "Yes")

   {

   fileinto "Spam";

   stop;

   }

I can see in the logs that Sieve is loaded, and also the imapsieve_plugin

   Debug: sieve: Pigeonhole version 0.5.5 (2483b085) initializing

   Debug: sieve: Sieve imapsieve plugin for Pigeonhole version 0.5.5 
(2483b085) loaded


Sieve detects outbound messages (from the log: Debug: imapsieve: 
mailbox INBOX.Sent: APPEND event) but nothing registers for inbound, 
and my spam_rule is never triggered. Any suggestions?




https://wiki.dovecot.org/Pigeonhole/Sieve/Troubleshooting


Here is the output from “Dovecot -n”:

# 2.3.5 (513208660): /etc/dovecot/dovecot.conf

# Pigeonhole version 0.5.5 (2483b085)

# OS: Linux 3.10.0-957.5.1.el7.x86_64 x86_64 CentOS Linux release 
7.6.1810 (Core)  xfs


# Hostname: 

auth_master_user_separator = *

auth_mechanisms = PLAIN LOGIN

auth_verbose_passwords = plain

deliver_log_format = from=%{from}, envelope_sender=%{from_envelope}, 
subject=%{subject}, msgid=%m, size=%{size}, %$


dict {

  acl = pgsql:/etc/dovecot/dovecot-share-folder.conf

  quotadict = pgsql:/etc/dovecot/dovecot-used-quota.conf

}

first_valid_uid = 2000

imapc_features = fetch-bodystructure fetch-headers

imapc_host = 

imapc_port = 993

imapc_ssl = imaps

imapc_ssl_verify = no

last_valid_uid = 2000

listen = * [::]

mail_debug = yes

mail_gid = 2000

mail_home = /var/mail/imapc/home/%d/%n

mail_location = imapc:/var/mail/imapc/%d/%n

mail_plugins = mailbox_alias mail_log notify

mail_uid = 2000

managesieve_notify_capability = mailto

managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate mime foreverypart 
extracttext imapsieve vnd.dovecot.imapsieve


namespace inbox {

  inbox = yes

  list = yes

  location =

  prefix =

  separator = .

  type = private

}

passdb {

  args = host=

  default_fields = userdb_imapc_user=%u 
userdb_imapc_password=#hidden_use-P_to_show#


  driver = imap

}

passdb {

  args = /etc/dovecot/dovecot-master-users

  driver = passwd-file

  master = yes

}

plugin {

  imapsieve_default = /var/vmail/sieve/global/spam_rule.sieve

  mail_log_events = delete undelete expunge mailbox_delete mailbox_rename

  mail_log_fields = uid box msgid size from subject

  mailbox_alias_new = Sent Messages

  mailbox_alias_new2 = Sent Items

  mailbox_alias_old = Sent

  mailbox_alias_old2 = Sent

  sieve_global_dir = /var/vmail/sieve/global/

  sieve_max_redirects = 30

  sieve_plugins = sieve_imapsieve

  sieve_trace_debug = yes

  sieve_trace_dir = /var/log/dovecot

  sieve_trace_level = matching

}

protocols = imap sieve

service auth {

  unix_listener /var/spool/postfix/private/dovecot-auth {

    group = postfix

    mode = 0666

    user = postfix

  }

  unix_listener auth-master {

    group = vmail

    mode = 0666

    user = vmail

  }

  unix_listener auth-userdb {

    group = vmail

    mode = 0660

    user = vmail

  }

}

service dict {

  unix_listener dict {

    group = vmail

    mode = 0660

    user = vmail

  }

}

service imap-login {

  process_limit = 500

  service_count = 1

}

service lmtp {

  executable = lmtp -L

  inet_listener lmtp {

    address = 127.0.0.1

    port = 24

  }

  process_min_avail = 5

  unix_listener /var/spool/postfix/private/dovecot-lmtp {

    group = postfix

    mode = 0600

    user = postfix

  }

  user = vmail

}

service managesieve-login {

  inet_listener sieve {

    address = 127.0.0.1

    port = 4190

  }

}

ssl = required

ssl_cert = ssl_cipher_list = 
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5


ssl_client_ca_file = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

ssl_dh = # hidden, use -P to show it

ssl_key = # hidden, use -P to show it

ssl_prefer_server_ciphers = yes

syslog_facility = local5

userdb {

  driver = prefetch

}

protocol imap {

  imap_client_workarounds = tb-extra-mailbox-sep

  mail_max_userip_connections = 30

  mail_plugins = mailbox_alias mail_log notify imap_sieve

}

protocol sieve {

  info_log_path = /var/log/dovecot/sieve-info.log

  log_path = /var/log/dovecot/sieve-err.log

  managesieve_implementation_string = 

Re: dovecot 2.3.5 - tests fail: http payload echo (ssl)

[Stephan Bosch via dovecot]

Op 07/03/2019 om 23:03 schreef Helmut K. C. Tessarek:

Thank you very much for getting back to me.

On 2019-03-07 04:23, Stephan Bosch via dovecot wrote:

If that
doesn't fail for some reason, can you run it within valgrind?

I ran the following command:

valgrind --log-file=valgrind_test-http-payload.out
src/lib-http/test-http-payload -D

But I guess the valgrind.log itself is useless to you, and the debug output is
printed to the terminal, so I kindliy ask you to be a bit more specific. What
exact output do you need or better yet, which command do you want me to run.


Since you're compiling it anyway, maybe you should first try to increase 
the CLIENT_PROGRESS_TIMEOUT in src/lib-http/test-http-payload.c. It is 
currently 10 seconds.


Regards,

Stephan.

Re: dovecot 2.3.5 - tests fail: http payload echo (ssl)

[Stephan Bosch via dovecot]

Op 08/03/2019 om 23:49 schreef Helmut K. C. Tessarek:


On 2019-03-07 17:03, Helmut K. C. Tessarek via dovecot wrote:

Does this work?:

NOVALGRIND=yes make -C src/lib-http check

Yes, no errors.


Does this mean dovecot will run properly?


Yes. This is test is just a bit unreliable it seems.

Regards,

Stephan.

Re: dovecot 2.3.5 - tests fail: http payload echo (ssl)

[Stephan Bosch via dovecot]

Op 07/03/2019 om 01:02 schreef Helmut K. C. Tessarek via dovecot:

Hi,

I was womdering, if anyone has experienced the same issues. When I run the
tests after compiling 2.3.5, the following 4 tests fail:

http payload echo (ssl): sequential .. : ok
http payload echo (ssl): pipeline  : ok
http payload echo (ssl): parallel  :
FAILED: Test is hanging
http payload echo (ssl) .. : FAILED
http payload echo (ssl; unknown size): sequential  : ok
http payload echo (ssl; unknown size): pipeline .. : ok
http payload echo (ssl; unknown size): parallel .. :
FAILED: Test is hanging
http payload echo (ssl; unknown size)  : FAILED
4 / 114 tests failed
Failed to run: ./test-http-payload

The tests ran fine for 2.3.4.1. Is there any way, to debug the issue any
further? Do the tests write log files?

It just doesn't make amy sense. Nothing has changed on the OS since I compiled
and tested 2.3.4.1.


SSL tests were added here for 2.3.5. Is the test taking a very long 
time? Is this a very slow system?


Does this work?:

NOVALGRIND=yes make -C src/lib-http check

Can you make a log of the output of src/lib-http/test-http-payload -D? 
If that doesn't fail for some reason, can you run it within valgrind?


Regards,

Stephan.

Re: Unexpected behavior with sieve_vacation_to_header_ignore_envelope = yes

[Stephan Bosch via dovecot]

Op 27/02/2019 om 13:30 schreef Matthias Petermann via dovecot:

Hello,

we face an issue with the vacation extension of pigeonhole. Our 
upstream mail server (from which we pull our E-Mails with POP3) uses 
SPF and performs a SRS re-write of the Return Path. Once the mail is 
processed via our local postfix, it is delivered via dovecot deliver 
(using the lda). When the recipient has a vacation rule enabled in 
sieve, the out-of-office note cannot be sent, as pigeonhole tries to 
use the Return-Path (which is not accepted as sender by the mail server).


We are using the following dovecot / pigeonhole versions:

dovecot-2.3.2.1
dovecot-pigeonhole-0.5.2

As of my understanding, the problem was addressed in pigeonhole 
version v0.4.21 by introducing the 
sieve_vacation_to_header_ignore_envelope setting. Anyway - even when 
this is set to "yes" pigeonhole still uses the Return-Path instead of 
the "From:" header which I had expected:


Feb 27 13:22:26 mail dovecot: 
lda(customer)<48383><3o+yOQGBdlz/vAAAZU03Dg>: Error: sieve: 
msgid=: failed 
to send vacation response to 
SRS0=gwGXg6ad=RC=petermann-it.de=matth...@example.com: program returned error> (temporary error)


Are my expectations wrong or did I miss some precondition?


That setting is only about the composition of the "To:" header for the 
outgoing vacation message (it's in the name). It has nothing to do with 
the RCPT TO envelope address used for that message. Why exactly do the 
mail servers reject the vacation message (I am not too familiar with SRS).


Regards,

Stephan.




Best regards,
Matthias

1) This is the email which I use to test (I replaced the actual domain 
with example.com):


-->8-- 


Return-Path: 
Delivered-To: customer[[@example.com]]
To: Customer 
From: Matthias Petermann 
Subject: Test
Message-ID: 
Date: Wed, 27 Feb 2019 05:21:45 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
 Thunderbird/60.5.1

Test message
--8<-- 



2) The related excerpt of the configuration:

-->8-- 

mail /usr/local/etc/dovecot/conf.d >doveconf | grep -A 10 -B 10 
to_header_ignore_envelope

  result_failure = continue
  result_internalfail = continue
  result_success = return-ok
  skip = never
  username_filter =
}
plugin {
  acl = vfile
  acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes
  sieve = file:/srv/vmail/%u/sieve;active=/srv/vmail/%u/active.sieve
  sieve_vacation_to_header_ignore_envelope = yes
}
pop3_client_workarounds =
pop3_delete_type = default
pop3_deleted_flag =
pop3_enable_last = no
pop3_fast_size_lookups = no
pop3_lock_session = no
pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s
pop3_no_flag_updates = no
pop3_reuse_xuidl = no
--8<-- 

Re: dovecot Buch 2014 vs 2016

[Stephan Bosch via dovecot]

Hallo,

Op 22-2-2019 om 13:08 schreef Heiko Schlittermann via dovecot:

Moin,

Es geht um das Dovecot-Buch. Ich nehme an, daß hier mindestens einer,
der sich auskennt, mitliest:

Gibt es - außer der Sprache und dem Preis - einen Unterschied zwischen
der ersten (und einzigen?) deutschen Auflage von 2014 und der englischen
Auflage von 2016?


Hier wird leider nur Englisch gesprochen. Sie können diese Frage dort 
nochmal versuchen: dove...@listen.jpberlin.de. Das ist auch die korrekte 
Adresse für Fragen über das Dovecot Buch.


Gruß,

Stephan.

Re: sieve filter not working

[Stephan Bosch via dovecot]

Op 20-2-2019 om 9:30 schreef subin ks via dovecot:
I've Dovecot and dovecot-sieve v 2.2.27 installed on a Debian 9.6. I'm 
trying to set a Sieve filter which will redirect all emails from 
`info` (i.e. .info) TLD to another email. This is the filter:


require ["regex"];
# rule:[test]
if header :regex "from" "info$"
{
redirect "su...@domain.com ";
}

It's not being honored; all emails from .info TLD ends up in the inbox 
and none are redirected. Let me know what I'm doing wrong.


You should use the "address" test instead. This parses the header for 
addresses. Using the "header" test this way is unreliable. For example, 
an address wrapped in <...> is not handled well by your attempt.


So, this is better:

if address :matches :domain "from" "*.info" {
redirect "su...@domain.com ";
}

Regards,

Stephan.

Re: assertion failed: (srcleft <= CHARSET_MAX_PENDING_BUF_SIZE)

[Stephan Bosch via dovecot]

Op 9-2-2019 om 19:27 schreef Giacomo via dovecot:

I got a core file this morning.

opening it with gdb I get this:

(gdb) core imap.core
Core was generated by `dovecot/imap'.
Program terminated with signal 6, Aborted.
#0  0x11c1347a in ?? ()
(gdb) bt
#0  0x11c1347a in ?? ()
#1  0x11c13444 in ?? ()
#2  0x00018dee in ?? ()
#3  0x306db139575f3b0d in ?? ()
#4  0x7fffc144 in ?? ()
#5  0x in ?? ()

Doesn't look clean at all.


That is not that helpful. Did you install debug symbols for Dovecot? 
This is usually a separate package. It all depends on your OS. I am not 
familiar with FreeBSD.


Regards,

Stephan.



let me know if I should upload the file somewhere.

Thanks

Il giorno lun 21 gen 2019 alle ore 08:52 Giacomo 
mailto:giacomo1...@gmail.com>> ha scritto:


I've just enabled core dumps on the involved FreeBSD system. Let's
see if it dumps something..

Il giorno dom 20 gen 2019 alle ore 19:16 Stephan Bosch
mailto:step...@rename-it.nl>> ha scritto:

Hi Giacomo,

Op 21/12/2018 om 16:16 schreef Giacomo:
> The bug happens not very often, it might need a week to get
a core file.

Any luck getting a core file?

Regards,

Stephan.
>
>
> Il giorno ven 21 dic 2018 alle ore 15:18 Aki Tuomi
> mailto:aki.tu...@open-xchange.com>
>> ha
> scritto:
>
>
>     On 21.12.2018 14.49, Giacomo wrote:
>     > I'm running the following:
>     >
>     > # 2.3.4 (0ecbaf23d): /usr/local/etc/dovecot/dovecot.conf
>     > # OS: FreeBSD 11.2-RELEASE-p4 amd64
>     > # FS: ZFS
>     >
>     > auth_mechanisms = plain login
>     > auth_username_format = %Ln
>     > listen = *
>     > mail_location = maildir:~/Maildir
>     > namespace inbox {
>     >   inbox = yes
>     >   location =
>     >   mailbox Drafts {
>     >     auto = subscribe
>     >     special_use = \Drafts
>     >   }
>     >   mailbox Junk {
>     >     auto = subscribe
>     >     special_use = \Junk
>     >   }
>     >   mailbox Sent {
>     >     special_use = \Sent
>     >   }
>     >   mailbox Trash {
>     >     auto = subscribe
>     >     special_use = \Trash
>     >   }
>     >   prefix =
>     > }
>     > passdb {
>     >   driver = pam
>     > }
>     > protocols = imap lmtp
>     > service auth {
>     >   unix_listener /var/spool/postfix/private/auth {
>     >     group = postfix
>     >     mode = 0666
>     >     user = postfix
>     >   }
>     >   user = dovecot
>     > }
>     > service imap-login {
>     >   inet_listener imap {
>     >     port = 0
>     >   }
>     > }
>     > service lmtp {
>     >   unix_listener /var/spool/postfix/private/dovecot-lmtp {
>     >     group = postfix
>     >     mode = 0600
>     >     user = postfix
>     >   }
>     > }
>     > service stats {
>     >   unix_listener stats-reader {
>     >     group =
>     >     mode = 0600
>     >     user =
>     >   }
>     >   unix_listener stats-writer {
>     >     group = $default_internal_group
>     >     mode = 0666
>     >     user =
>     >   }
>     >   user = $default_internal_user
>     > }
>     > ssl_cert =      > ssl_key = # hidden, use -P to show it
>     > userdb {
>     >   driver = passwd
>     > }
>     >
>     > dmesg shows:
>     >
>     > pid 74441 (imap), uid 1002: exited on signal 6
>     > pid 74784 (imap), uid 1002: exited on signal 6
>     > pid 21471 (imap), uid 1002: exited on signal 6
>     > pid 21470 (imap), uid 1002: exited on signal 6
>     > pid 21464 (imap), uid 1002: exited on signal 6
>     > pid 21731 (imap), uid 1002: exited on signal 6
>     >
>     > Sometimes the dovecot process serving an user crashes.
He runs
>     mostly
>     > airmail on macOS and samsung mail on Android.
>     >
>     > Happens only to that particular user.
>     >
>     > The log shows the following: (maillog)
>     >
>     > Dec 16 14:52:35 mail dovecot: imap-login: Login:
user=,
>     > method=PLAIN, rip=79.12.201.157, lip=192.168.1.28,
mpid=74779, TLS,
>     > session=
>     > Dec 16 14:52:39 mail dovecot:

Re: Dovecot v2.2.36.1 released (Pigeonhole 0.4.24.1)

[Stephan Bosch via dovecot]

Op 05/02/2019 om 20:27 schreef Michael Marley via dovecot:

On 2019-02-05 13:07, Stephan Bosch via dovecot wrote:

Hi,

Here is the associated release for Pigeonhole:

https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.1.tar.gz 

https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.1.tar.gz.sig 


Binary packages included in https://repo.dovecot.org/

Is there going to be an equivalent 0.5.4.1 release with the same 
functionality but for Dovecot 2.3.x?




No. The current plan is to release a 2.3.5/0.5.5 later this month.

Regards,

Stephan.

Re: [Dovecot-news] Dovecot v2.2.36.1 released (Pigeonhole 0.4.24.1)

[Stephan Bosch via Dovecot-news]

Hi,

Here is the associated release for Pigeonhole:

https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.1.tar.gz
https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.1.tar.gz.sig
Binary packages included in https://repo.dovecot.org/

+ imapsieve: Added imapsieve_expunge_discarded setting which causes
  discarded messages to be expunged immediately.
- Sieve scripts running in IMAPSIEVE or IMAP FILTER=SIEVE context that
  modify the message, store the message a second time, rather than
  replacing the originally stored unmodified message.
- imapsieve: Fix crash when COPYing mails from a virtual mailbox when
  the source messages originate from more than a single real mailbox
- imap_filter_sieve plugin: Implement the missing UID FILTER command.
- imap_filter_sieve plugin: Fix FILTER to work with pipelining


Regards,

Stephan.

Op 5-2-2019 om 14:01 schreef Aki Tuomi:

https://dovecot.org/releases/2.2/dovecot-2.2.36.1.tar.gz
https://dovecot.org/releases/2.2/dovecot-2.2.36.1.tar.gz.sig

     * CVE-2019-3814: If imap/pop3/managesieve/submission client has
       trusted certificate with missing username field
       (ssl_cert_username_field), under some configurations Dovecot
       mistakenly trusts the username provided via authentication instead
       of failing.
     * ssl_cert_username_field setting was ignored with external SMTP AUTH,
       because none of the MTAs (Postfix, Exim) currently send the
       cert_username field. This may have allowed users with trusted
       certificate to specify any username in the authentication. This bug
       didn't affect Dovecot's Submission service.

     - pop3_no_flag_updates=no: Don't expunge RETRed messages without QUIT
     - director: Kicking a user assert-crashes if login process is very slow
     - lda/lmtp: Fix assert-crash with some Sieve scripts when
       mail_attachment_detection_options=add-flags-on-save
     - fs-compress: Using maybe-gz assert-crashed when reading 0 sized file
     - Snippet generation crashed with invalid Content-Type:multipart


---

Aki Tuomi
Open-Xchange Oy




___
Dovecot-news mailing list
Dovecot-news@dovecot.org
https://dovecot.org/mailman/listinfo/dovecot-news

Re: Dovecot v2.2.36.1 released (Pigeonhole 0.4.24.1)

[Stephan Bosch via dovecot]

Hi,

Here is the associated release for Pigeonhole:

https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.1.tar.gz
https://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.24.1.tar.gz.sig
Binary packages included in https://repo.dovecot.org/

+ imapsieve: Added imapsieve_expunge_discarded setting which causes
  discarded messages to be expunged immediately.
- Sieve scripts running in IMAPSIEVE or IMAP FILTER=SIEVE context that
  modify the message, store the message a second time, rather than
  replacing the originally stored unmodified message.
- imapsieve: Fix crash when COPYing mails from a virtual mailbox when
  the source messages originate from more than a single real mailbox
- imap_filter_sieve plugin: Implement the missing UID FILTER command.
- imap_filter_sieve plugin: Fix FILTER to work with pipelining


Regards,

Stephan.

Op 5-2-2019 om 14:01 schreef Aki Tuomi:

https://dovecot.org/releases/2.2/dovecot-2.2.36.1.tar.gz
https://dovecot.org/releases/2.2/dovecot-2.2.36.1.tar.gz.sig

     * CVE-2019-3814: If imap/pop3/managesieve/submission client has
       trusted certificate with missing username field
       (ssl_cert_username_field), under some configurations Dovecot
       mistakenly trusts the username provided via authentication instead
       of failing.
     * ssl_cert_username_field setting was ignored with external SMTP AUTH,
       because none of the MTAs (Postfix, Exim) currently send the
       cert_username field. This may have allowed users with trusted
       certificate to specify any username in the authentication. This bug
       didn't affect Dovecot's Submission service.

     - pop3_no_flag_updates=no: Don't expunge RETRed messages without QUIT
     - director: Kicking a user assert-crashes if login process is very slow
     - lda/lmtp: Fix assert-crash with some Sieve scripts when
       mail_attachment_detection_options=add-flags-on-save
     - fs-compress: Using maybe-gz assert-crashed when reading 0 sized file
     - Snippet generation crashed with invalid Content-Type:multipart


---

Aki Tuomi
Open-Xchange Oy