Logstash pattern (GROK, KV, ...) to parse dovecot logs anyone?

2015-12-01 Thread Christian Rohmann 
Hello dovecot-users,

I am currently playing with Elastics ELK stack and was kind of surprised to NOT
yet find a good set of GROK or KV pattern to parse dovecots lush and information
rich logs.
The last post regarding this endeavor was in 2014
(http://www.dovecot.org/list/dovecot/2014-June/096589.html), which "only"
extracts the key->value pairs but not other parts of the log lines. 
One finds the occasional attempt here and there on GitHub, like
https://github.com/PCextreme/logstash-grok-patterns/blob/master/mail .
But nothing in comparison to the simply amazingly good patterns there are for
Postfix from whyscream (https://github.com/whyscream/postfix-grok-patterns). He
even added some "I don't understand this yet" rule to learn where the parsing
lags.

I was wondering if anyone here is running logstash and does already have a set
of GROK or KV configuration and is willing to share that with the world?
A joint effort might get us to a complete extraction of key->values and all
other interesting fields for dovecot quickly I hope.


Regards

Christian

Re: Questions about hardlinks, alternate storage and compression]

2015-12-01 Thread Gionatan Danti 

Il 30-11-2015 23:23 Timo Sirainen ha scritto:

On 30 Nov 2015, at 17:48, Gionatan Danti  wrote:


Hi Timo,
glad to know it is in your TODO list ;)


It's been for many years.


Any rough ETA on that?


Right now it doesn't seem likely to be developed anytime soon.



Thank you anyway :)

It let me wonder if I am the only one caring about hardlinks...
Maybe I overstate the space savings?

Thanks.

--
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.da...@assyoma.it - i...@assyoma.it
GPG public key ID: FF5F32A8

Re: mail_log & doveadm

2015-12-01 Thread Timo Sirainen 
On 01 Dec 2015, at 18:27, Larry Rosenman  wrote:
> 
> Is there any way to have doveadm NOT log using mail_log, but have the
> daemons do it?
> (or have doveadm's log go to syslogd)?
> 
> Thanks!
> 
> ex:
> doveadm(ler): Info: copy from SA/FN: box=#ARCHIVE/2015/11/SA/FN, uid=1,
> msgid=, size=107835, vsize=110376,
> from="Alley Boost"  , subject=Alley Boost: VIP Mixer
> TONIGHT, Startup Expo TOMORROW, & More! :), flags=(\Seen)

One possibility is to include mail_log plugin for all except the doveadm:

protocol !doveadm {
  mail_plugins = $mail_plugins mail_log
}

mail_log plugin and uid=error in log file

2015-12-01 Thread Alessio Cecchi 

Hi,

I enabled notify and mail_log plugins to log the message uid for new 
emails delivered via LMTP. All works fine except that the uid is not 
always logged (see uid=error instead of uid=NUMBER)


from dovecot.log:

Dec 01 10:03:08 lmtp(ales...@domain.com): Info: copy from : 
box=INBOX, uid=error, msgid=<14489...@domain.com>, from="WordPress" 
, subject=Enquiry from Joy


Dec 01 10:03:09 lmtp(ales...@domain.com): Info: copy from : 
box=INBOX, uid=92, msgid=<03c4e...@www.domain.com>, from="WordPress" 
, subject=Enquiry from Joy



When the uid number is missing the message have is uid in 
dovecot-uidlist file but is represented in different format:


92 G1448960589.M630964P41853.qb-dev 
:1448960589.M630964P41853.qb-dev,S=35341,W=35856

93 :1448960586.M423202P41802.qb-dev,S=35378,W=35890

It seems that the uid is determined after the log is written ...

I'm running Dovecot 2.2.19, Maildir as storage format and LMTP for delivery.

Is a bug or a "feature"? Can be fixed?

Thanks

this is my configuration:

# dovecot -n
# 2.2.19: /etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.9
# OS: Linux 2.6.32-573.7.1.el6.x86_64 x86_64 CentOS release 6.7 (Final)
auth_master_user_separator = *
dict {
  expire = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
  sqlquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext
}
first_valid_gid = 89
first_valid_uid = 89
last_valid_gid = 89
last_valid_uid = 89
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
log_path = /var/log/dovecot/dovecot.log
mail_fsync = always
mail_location = maildir:~/Maildir
mail_plugins = quota zlib fts fts_lucene acl mail_log notify
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate

namespace inbox {
  inbox = yes
  location =
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix =
  separator = /
}
passdb {
  args = /etc/dovecot/dovecot-deny-sql.conf.ext
  deny = yes
  driver = sql
}
passdb {
  args = /etc/dovecot/extra/master-users
  driver = passwd-file
  master = yes
  pass = yes
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  acl = vfile:/etc/dovecot/global-acls:cache_secs=300
  fts = lucene
  fts_lucene = whitespace_chars=@.
  mail_log_events = copy save
  mail_log_fields = uid box msgid from subject
  quota = dict:User quota::noenforcing:proxy::sqlquota
  sieve_default = /etc/dovecot/sieve/default.sieve
  zlib_save = gz
  zlib_save_level = 6
}
protocols = imap sieve lmtp
service auth {
  unix_listener auth-userdb {
group = vchkpw
mode = 0660
user = vmail
  }
}
service dict {
  unix_listener dict {
group = vchkpw
mode = 0660
user = vmail
  }
}
service imap-postlogin {
  executable = script-login /etc/dovecot/scripts/imap-postlogin.sh
  unix_listener imap-postlogin {
group = vchkpw
mode = 0660
user = vmail
  }
  user = vmail
}
service imap {
  executable = imap imap-postlogin
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0660
user = postfix
  }
}
ssl_cert =   mail_plugins = quota zlib fts fts_lucene acl mail_log notify 
imap_quota imap_zlib imap_acl

}

--
Alessio Cecchi
Postmaster @ http://www.qboxmail.it
https://www.linkedin.com/in/alessice

Looks like a bug to me: Dovecot ignores Maildir/new timestamp

2015-12-01 Thread Fredrik Roubert 
Hello!

I run Debian jessie and have an issue with the interaction between
Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218)
IMAP client, when using Maildir:

I use Dovecot LMTP for local delivery into Maildir and access these
files both through Dovecot IMAP from the Android Gmail IMAP client and
also directly through the filesystem by running Mutt locally on the mail
server.

When a new mail arrives, Dovecot LMTP will add a new file in Maildir/new
and also update the Dovecot index cache. If I then open Mutt and just
delete this new message, then Mutt will just delete this file from
Maildir/new and the Dovecot index cache will be out-of-sync.

(For testing, Mutt can be cut out of the equation entirely by instead
just running "rm Maildir/new/*" from the command line to mimic what Mutt
does when deleting a new message, but I mention the use of Mutt here to
show why this is a real use-case for me.)

If I after this open the Gmail app and pull down the message index to
refresh from the Dovecot IMAP server, it'll get the cached version of
the index, in which the new message is still present (but only the
headers, no body will be loaded and the app will show no body snippet).
Pulling down the message index again to force a second refresh will
cause the new/deleted message to disappear again.

This seems to be caused by Dovecot not looking at the Maildir/new
timestamp to determine whether the index cache is up-to-date, but
instead only looking at the Maildir/cur timestamp.

If I instead repeat all these testing steps from the beginning, sending
a new mail message, deleting it with "rm Maildir/new/*" but then also
running "touch Maildir/cur" before opening the Gmail app and doing the
refresh, then Dovecot does update the cache and does not send any ghost
messages to the IMAP client.

It does seem reasonable to me that Mutt doesn't touch Maildir/cur when
just deleting new messages and I suspect that it might be a bug in
Dovecot that it timestamp of Maildir/new isn't checked and the index
cache updated if that timestamp is newer than the last index update done
by the Dovecot LMTP service.

Does this analysis seem sound? Did I miss anything?

(Output of "doveconf -n" attached.)

Cheers // Fredrik Roubert

-- 
Forsterstrasse 64  |  +41 78 8170377
CH-8044 Zürich |  https://roubert.name/fredrik/
# 2.2.13: /etc/dovecot/dovecot.conf
# OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.2 
auth_username_chars = 
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@/
auth_username_format = %Ln
first_valid_uid = 1000
last_valid_uid = 65533
listen = 5.148.172.24, 2a02:418:1003:7::1
mail_location = maildir:~/Maildir
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave editheader 
vnd.dovecot.duplicate
mbox_read_locks = flock
mbox_write_locks = flock
namespace inbox {
  inbox = yes
  location = 
  mailbox Drafts {
special_use = \Drafts
  }
  mailbox Junk {
special_use = \Junk
  }
  mailbox Sent {
special_use = \Sent
  }
  mailbox "Sent Messages" {
special_use = \Sent
  }
  mailbox Trash {
special_use = \Trash
  }
  prefix = 
}
passdb {
  args = scheme=SHA512-CRYPT /etc/dovecot/passwd-imap
  driver = passwd-file
}
passdb {
  args = scheme=SHA512-CRYPT /etc/dovecot/passwd-smtp
  driver = passwd-file
}
plugin {
  sieve = ~/.dovecot.sieve
  sieve_before = /var/lib/dovecot/sieve/spam.sieve
  sieve_dir = ~/.sieve
  sieve_extensions = +editheader +vnd.dovecot.duplicate
}
protocols = " imap lmtp sieve"
service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
group = postfix
mode = 0666
user = postfix
  }
}
service imap-login {
  inet_listener imap {
port = 143
  }
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0666
user = postfix
  }
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
}
ssl = required
ssl_cert =

Dovecot doesn't sent rejection message user overquota

2015-12-01 Thread Antonello Cioffi 

Hi

I'm using postfix+dovecot (2.2.18).

The problem is that dovecot silently discard message when user has its 
own mailbox full without sending rejection message to the sender.


Here a sample log:

Dec  1 14:54:23 posta2 postfix/smtp[21478]: B315111C00B: 
to=, 
relay=192.168.241.110[192.168.241.110]:25, delay=0.4, 
delays=0.2/0/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 
C27BF1244DD)
Dec  1 14:54:23 posta2 dovecot: lda(antonen): Error: sieve: 
msgid=<565da68e.8060...@uniparthenope.it>: failed to store into mailbox 
'INBOX': Quota exceeded (mailbox for user is full)
Dec  1 14:54:23 posta2 dovecot: lda(antonen): sieve: Execution of script 
/usr/local/etc/dovecot/sieve/default.sieve failed with unsuccessful 
implicit keep
Dec  1 14:54:23 posta2 dovecot: lda(antonen): 
msgid=<565da68e.8060...@uniparthenope.it>: rejected: Quota exceeded 
(mailbox for user is full)
Dec  1 14:54:23 posta2 postfix/pipe[21020]: 111FB11C00B: 
to=, relay=dovecot, delay=0.14, 
delays=0.05/0.01/0/0.09, dsn=2.0.0, status=sent (delivered via dovecot 
service)


This is dovecot configuration:

posta2:/usr/local/etc/dovecot # doveconf -n
# 2.2.18: /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.8 (0c4ae064f307+)
# OS: Linux 3.0.101-0.46-default x86_64 SUSE Linux Enterprise Server 11 
(x86_64)

auth_mechanisms = plain login
auth_username_format = %Ln
auth_verbose = yes
default_internal_user = vmail
default_login_user = nobody
disable_plaintext_auth = no
first_valid_uid = 100
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
mail_gid = 100
mail_location = maildir:%h
mail_plugins = " quota"
mail_uid = 1002
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date index ihave duplicate spamtest spamtestplus

passdb {
  args = /usr/local/etc/dovecot/dovecot-people-ldap.conf.ext
  driver = ldap
}
plugin {
  last_login_dict = redis:host=127.0.0.1:port=6379
  mail_log_events = delete undelete expunge copy mailbox_delete 
mailbox_rename

  mail_log_fields = uid box msgid size
  quota = maildir:User quota
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_before = /usr/local/etc/dovecot/sieve/
  sieve_dir = ~/.sieve
  sieve_extensions = +spamtest +spamtestplus +relational 
+comparator-i;ascii-numeric

}
postmaster_address = postmas...@uniparthenope.it
protocols = imap pop3 sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
  }
}
service imap-login {
  inet_listener imap {
port = 143
  }
  inet_listener imaps {
port = 993
ssl = yes
  }
  service_count = 0
  vsz_limit = 256 M
}
service managesieve-login {
  inet_listener sieve {
port = 4190
  }
  service_count = 1
  vsz_limit = 64 M
}
service pop3-login {
  inet_listener pop3 {
port = 110
  }
  inet_listener pop3s {
port = 995
ssl = yes
  }
  service_count = 0
  vsz_limit = 256 M
}
service quota-status {
  client_limit = 1
  executable = quota-status -p postfix
  inet_listener {
port = 12340
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
user = vmail
  }
  user = vmail
}
ssl_ca =   imap_client_workarounds = delay-newmail tb-extra-mailbox-sep 
tb-lsub-flags

  mail_max_userip_connections = 100
  mail_plugins = " quota imap_quota last_login"
}
protocol pop3 {
  mail_max_userip_connections = 100
  mail_plugins = " quota last_login"
  pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s, %u
  pop3_save_uidl = no
  pop3_uidl_format = %08Xu%08Xv
}

while in master.cf

dovecotunix  -   n   n   -   -   pipe
  flags=DRhu user=vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f 
${sender} -d ${recipient}


Is there someone who could help me?

Regards

--
Dott. Antonello Cioffi
Ufficio Servizi Informatici
Università degli Studi di Napoli Parthenope
Tel. 081/5475292 - Fax. 081/5475180

Re: Dovecot doesn't sent rejection message user overquota

2015-12-01 Thread Timo Sirainen 
On 01 Dec 2015, at 17:10, Antonello Cioffi  
wrote:
> 
> Hi
> 
> I'm using postfix+dovecot (2.2.18).
> 
> The problem is that dovecot silently discard message when user has its own 
> mailbox full without sending rejection message to the sender.
> 
> Here a sample log:
> 
> Dec  1 14:54:23 posta2 postfix/smtp[21478]: B315111C00B: 
> to=, relay=192.168.241.110[192.168.241.110]:25, 
> delay=0.4, delays=0.2/0/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued 
> as C27BF1244DD)
> Dec  1 14:54:23 posta2 dovecot: lda(antonen): Error: sieve: 
> msgid=<565da68e.8060...@uniparthenope.it>: failed to store into mailbox 
> 'INBOX': Quota exceeded (mailbox for user is full)
> Dec  1 14:54:23 posta2 dovecot: lda(antonen): sieve: Execution of script 
> /usr/local/etc/dovecot/sieve/default.sieve failed with unsuccessful implicit 
> keep
> Dec  1 14:54:23 posta2 dovecot: lda(antonen): 
> msgid=<565da68e.8060...@uniparthenope.it>: rejected: Quota exceeded (mailbox 
> for user is full)
> Dec  1 14:54:23 posta2 postfix/pipe[21020]: 111FB11C00B: 
> to=, relay=dovecot, delay=0.14, 
> delays=0.05/0.01/0/0.09, dsn=2.0.0, status=sent (delivered via dovecot 
> service)

Dovecot should have sent a bounce message at that time. I guess that doesn't 
work for some reason. By default Dovecot uses:

sendmail_path = /usr/sbin/sendmail

Also you can instead have it send it via SMTP by setting submission_host 
setting. And finally if you give -e parameter to dovecot-lda, it won't send a 
bounce itself but instead will exit with 77 causing Postfix to send the bounce.

Re: mail_log plugin and uid=error in log file

2015-12-01 Thread Timo Sirainen 
On 01 Dec 2015, at 11:36, Alessio Cecchi  wrote:
> 
> Hi,
> 
> I enabled notify and mail_log plugins to log the message uid for new emails 
> delivered via LMTP. All works fine except that the uid is not always logged 
> (see uid=error instead of uid=NUMBER)
> 
> from dovecot.log:
> 
> Dec 01 10:03:08 lmtp(ales...@domain.com): Info: copy from : 
> box=INBOX, uid=error, msgid=<14489...@domain.com>, from="WordPress" 
> , subject=Enquiry from Joy
> 
> Dec 01 10:03:09 lmtp(ales...@domain.com): Info: copy from : 
> box=INBOX, uid=92, msgid=<03c4e...@www.domain.com>, from="WordPress" 
> , subject=Enquiry from Joy

Does this help? http://hg.dovecot.org/dovecot-2.2/rev/25d63d9c7f5a

Re: lmtp panic

2015-12-01 Thread Stephan Bosch 



Op 10-11-2015 om 13:46 schreef Sergey Schwartz:

Gents,

I've just upgraded to the latest build of dovecot , now lmtp delivery 
process panics for just one user


Hmm, this could very well be a Sieve issue. Can you obtain a GDB 
backtrace for this problem (http://www.dovecot.org/bugreport.html)?


Can you reproduce it using the sieve-test tool with the involved Sieve 
script and some example e-mail?


Regards,

Stephan.







Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasil...@bgoperator.com): 
Panic: file str.c: line 22 (str_new_const): assertion failed: 
(str[len] == '\0')
Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasil...@bgoperator.com): 
Panic: file str.c: line 22 (str_new_const): assertion failed: 
(str[len] == '\0')
Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasil...@bgoperator.com): 
Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) 
[0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) 
[0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) 
[0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) 
[0x7f50e380ddbd] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) 
[0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) 
[0x7f50e380f555] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f50e37eb2e7] 
-> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) 
[0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) 
[0x7f50e37fe6ea] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) 
[0x7f50e37ff277] -> 
/usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) 
[0x7f50e3a62b81] -> 
/usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) 
[0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> 
dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) 
[0x7f50e597ebbc] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) 
[0x7f50e597ffb1] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) 
[0x7f50e597ec45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) 
[0x7f50e597ede8] -> 
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) 
[0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223]
Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasil...@bgoperator.com): 
Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) 
[0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) 
[0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) 
[0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) 
[0x7f887d2b0dbd] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) 
[0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) 
[0x7f887d2b2555] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f887d28e2e7] 
-> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) 
[0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) 
[0x7f887d2a16ea] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) 
[0x7f887d2a2277] -> 
/usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) 
[0x7f887d505b81] -> 
/usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) 
[0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> 
dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) 
[0x7f887f421bbc] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) 
[0x7f887f422fb1] -> 
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) 
[0x7f887f421c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) 
[0x7f887f421de8] -> 
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) 
[0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> 
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) 
[0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223]

Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local
Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local
Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasil...@bgoperator.com): 
Panic: file str.c: line 22 (str_new_const): assertion failed: 
(str[len] == '\0')
Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasil...@bgoperator.com): 
Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) 
[0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) 
[0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) 
[0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) 
[0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) 
[0x7fd26aef5dbd] -> 
/usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) 
[0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) 
[0x7fd26aef7555] ->

Re: Let lmtp create target directories

2015-12-01 Thread Timo Sirainen 
On 26 Nov 2015, at 04:12, martin f krafft  wrote:
> 
> Hello,
> 
> we're using vmm¹ to manage our postfix+dovecot virtual mail setup,
> which allows us to give every virtual user a separate EUID and every
> domain a separate EGID for additional security (vs. handling all
> virtual mail with a single "vmail" user).
> 
> As a consequence, however, vmm must itself create the user
> directories with the appropriate owners, and to do so, it requires
> root rights.
> 
> I am trying to investigate getting rid of this need³. Since Dovecot
> quite happily creates ~/Maildir when necessary, couldn't it also
> create parents? The home directory should be trivial (same
> EUID/EGID), but grandparents etc. might need a different policy
> (e.g. 0/EGID for the grandparent, 0/0 for great-grandparents, etc.).

Dovecot already creates all the parent directories. What to set to the 
permissions are of course a problem. 
http://wiki2.dovecot.org/SharedMailboxes/Permissions explains how it works 
right now. See especially "Permissions to new /domain/user directories" and 
"Permissions to new user home directories (v2.2+)".

> Is this something that could fall within the realm of Dovecot's
> lmtp? Or is the lmtp invoked as the user and doesn't actually drop
> root? If so, might there be another way?

That's the even bigger issue. The home dir creation is done with the user's 
privileges, not as root. But the +t bit might help.

Re: CentOS rpm dovecot 2.2.10 auth/db-ldap.c TLS bug/patch

2015-12-01 Thread Timo Sirainen 
On 25 Nov 2015, at 15:42, Andrey Fesenko  wrote:
> 
> Hello,
> CentOS rpm dovecot 2.2.10 сontains bug auth/db-ldap.c TLS (not connect
> LDAP+TLS server ldaps://), exist bug/patch
> https://bugs.centos.org/view.php?id=8267
> 
> As far as the correct patch in upstream dovecot quite a lot of changes
> at this point if there is a correct patch?

http://hg.dovecot.org/dovecot-2.2/rev/727acba74cbf

Re: Looks like a bug to me: Dovecot ignores Maildir/new timestamp

2015-12-01 Thread Marcus Rueckert 
On 2015-12-01 11:30:10 +0100, Fredrik Roubert wrote:
> I run Debian jessie and have an issue with the interaction between
> Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218)
> IMAP client, when using Maildir:

As a fellow mutt user. I use mutt with dovecot. mutt headercache was
much slower for me than dovecot's caching. if you dont want to go via
the tcp port, you can set /usr/lib/dovecot/imap as a tunnel in your
muttrc.

hth

darix

-- 
   openSUSE - SUSE Linux is my linux
   openSUSE is good for you
   www.opensuse.org

Re: Looks like a bug to me: Dovecot ignores Maildir/new timestamp

2015-12-01 Thread Timo Sirainen 

> On 01 Dec 2015, at 12:30, Fredrik Roubert  wrote:
> 
> Hello!
> 
> I run Debian jessie and have an issue with the interaction between
> Dovecot (2.2.13), Mutt (1.5.23) and the Android Gmail (5.8.105868218)
> IMAP client, when using Maildir:
> 
> I use Dovecot LMTP for local delivery into Maildir and access these
> files both through Dovecot IMAP from the Android Gmail IMAP client and
> also directly through the filesystem by running Mutt locally on the mail
> server.
> 
> When a new mail arrives, Dovecot LMTP will add a new file in Maildir/new
> and also update the Dovecot index cache. If I then open Mutt and just
> delete this new message, then Mutt will just delete this file from
> Maildir/new and the Dovecot index cache will be out-of-sync.
> 
> (For testing, Mutt can be cut out of the equation entirely by instead
> just running "rm Maildir/new/*" from the command line to mimic what Mutt
> does when deleting a new message, but I mention the use of Mutt here to
> show why this is a real use-case for me.)
> 
> If I after this open the Gmail app and pull down the message index to
> refresh from the Dovecot IMAP server, it'll get the cached version of
> the index, in which the new message is still present (but only the
> headers, no body will be loaded and the app will show no body snippet).
> Pulling down the message index again to force a second refresh will
> cause the new/deleted message to disappear again.
> 
> This seems to be caused by Dovecot not looking at the Maildir/new
> timestamp to determine whether the index cache is up-to-date, but
> instead only looking at the Maildir/cur timestamp.

Dovecot does check Maildir/new timestamp, but it'll only perform a "partial 
sync" then, which means it adds any newly found mails but it doesn't delete any 
mails because it can't be sure that they weren't moved to cur/. One simple fix 
for this would be to scan cur/ also whenever new/ changes, but that would 
unnecessarily make the performance worse for most people. Another fix would be 
to remember that the mail was in new/ and when cur's timestamp hadn't changed 
and the mail wasn't in new/ anymore it would be removed from index. But this 
would require remembering that the mail is in new/, which isn't currently done 
and adding it would probably cause extra disk writes.

So I don't really see a way of fixing this in any nice way.

Re: Sieve, shared folders, different daemons

2015-12-01 Thread Frido Otten 
Op 01-12-15 om 08:41 schreef Steffen Kaiser:
> On Mon, 30 Nov 2015, Frido Otten wrote:
>
> > In our dovecot setup we use 3 different config files. One for
> > maildropping(sieve), one for dovecot/imap on standard ports with ssl
> > config and one for dovecot/imap on standard ports+1 with a different ssl
> > certificate. Currently the client using the standard port+1 config wants
> > to make use of shared folders, but the thing is that the current
> > namespace separator in all configs is '.'. This is conflicting with
> > shared folders.
>
> How does '.' conflict with shared folders?
>

Sorry for the confusion, I forgot to mention that we have usernames with
a '.' in it. The separator and those usernames together with shared
folders, won't work.

> >Can we simply run one of the dovecot/imap daemons
> > configured with a different separator and shared folders without having
> > impact on sieve and the current connected users to the standard ports?
>
> I hate all questions and answers with "simple" or "simply" in it.
Sorry, I'll use easily then... ;)
> But yes, the IMAP separator does not have no impact on the filesystem
> _usually_. But mayhap you have a strange config anyway (with 3
> configs) and the separator does influence the storage.
>
Double denial there.. ;)
Why 3 different configs? It's for historic reasons, and probably they
can be combined into one.

So if I understand this right, the separator as a '.' and sieve rules
like this with a dot in the foldername:..

if allof (header :contains ["Return-path"] ["@dovecot.org"]) {
fileinto "Maillist.Dovecot";
}

won't be affected with only one of both imap configs having '/' as a
separator.

Regards,
Frido



signature.asc
Description: OpenPGP digital signature

Re: mail_log plugin and uid=error in log file

2015-12-01 Thread Gilles Chauvin 

On 01/12/2015 16:31, Timo Sirainen wrote:


Does this help? http://hg.dovecot.org/dovecot-2.2/rev/25d63d9c7f5a



Hi Timo,

Sorry about the thread hijacking but, speaking about the mail_log 
plugin, what do you think about the ability to add the session number to 
the log lines produced by this plugin?


This could be a useful information to have too, especially on a large 
traffic mailhost.



Regards,
Gilles



smime.p7s
Description: S/MIME Cryptographic Signature

Re: mail_log plugin and uid=error in log file

2015-12-01 Thread Marcus Rueckert 
On 2015-12-01 17:08:01 +0100, Gilles Chauvin wrote:
> Sorry about the thread hijacking but, speaking about the mail_log plugin,
> what do you think about the ability to add the session number to the log
> lines produced by this plugin?
> 
> This could be a useful information to have too, especially on a large
> traffic mailhost.

mail_log_prefix = "%s(%u): %{session} "

only downside is some lines will then end up with the session ID twice.
When i asked Timo said he doesnt want to change that behavior in 2.2.
but in 2.3 it might be an option to unify that.

hth

 darix

-- 
   openSUSE - SUSE Linux is my linux
   openSUSE is good for you
   www.opensuse.org

mail_log & doveadm

2015-12-01 Thread Larry Rosenman 
Is there any way to have doveadm NOT log using mail_log, but have the
daemons do it?
(or have doveadm's log go to syslogd)?

Thanks!

ex:
doveadm(ler): Info: copy from SA/FN: box=#ARCHIVE/2015/11/SA/FN, uid=1,
msgid=, size=107835, vsize=110376,
from="Alley Boost"  , subject=Alley Boost: VIP Mixer
TONIGHT, Startup Expo TOMORROW, & More! :), flags=(\Seen)



-- 
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larry...@gmail.com
US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961


dc.n
Description: Binary data

Re: mail_log plugin and uid=error in log file

2015-12-01 Thread Gilles Chauvin 

On 01/12/2015 17:14, Marcus Rueckert wrote:

mail_log_prefix = "%s(%u): %{session} "

only downside is some lines will then end up with the session ID twice.
When i asked Timo said he doesnt want to change that behavior in 2.2.
but in 2.3 it might be an option to unify that.



Hi Marcus,

I will try this soon.

Thanks for the tip.


Regards,
Gilles



smime.p7s
Description: S/MIME Cryptographic Signature