Re: Dovecot 2.1.7 randomly lost emails

2016-05-12 Thread Steffen Kaiser

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Thu, 12 May 2016, Paolo Rinaldo wrote:


the Postfix mail.log
May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: 
from=, size=11822371, nrcpt=2 (queue active)
May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: to=, 
relay=mail4.robinson.it[private/dovecot-lmtp], delay=0.64,  delays=0.45/0/0/0.19, 
dsn=2.0.0, status=sent (250 2.0.0  0vNzHyptMlezHgAAk+lh+g Saved)
May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: to=user-...@domain.com>, 
relay=mail4.robinson.it[private/dovecot-lmtp], delay=0.74,  delays=0.45/0/0/0.29, 
dsn=2.0.0, status=sent (250 2.0.0 user-...@domain.com> 0vNzHyptMlezHgAAk+lh+g 
Saved)
May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: removed

the dovecot-info.log
May 11 01:26:55 lmtp(7859, user-...@domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: sieve: 
msgid=<003a2d931fff0e48a2314a1adc87e23e8e4ba...@t49-ms-exmb02.intranet.saksroot.saksinc.com>:
 stored mail into mailbox 'INBOX'
May 11 01:26:55 lmtp(7859, user-...@domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: sieve: 
msgid=<003a2d931fff0e48a2314a1adc87e23e8e4ba...@t49-ms-exmb02.intranet.saksroot.saksinc.com>:
 stored mail into mailbox 'INBOX'

I cannot find the email 0vNzHyptMlezHgAAk+lh+g  for 
user-...@domain.com, nor I cannot suppose problems 
on the client, nothing changed on the back-end.


try the Maillog plugin first:
http://wiki2.dovecot.org/Plugins/MailLog


- -- 
Steffen Kaiser

-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBVzVumHz1H7kL/d9rAQI3RQf/VhrIbiQd8tlrtWkPA5Awr2ZBN8QcclsP
OoMGM6GAgs3ElSmGJx8VYv4nQMycjcmcvRbA0czGGoHvlO8RyL13MH3UKvLOv2wQ
zotVdm4VFMM4z0jxZzoO9CWdeC0QZ0NrqQknaBxG/SoOPPlfIlmTUfG+o8O/2WNM
DxU8GdxLL+vbLqsQ5Gmg1wsm2FPGlZ89CoFYR1STu5AmR1lB1SgmF+qyZ/g2ARov
bU9kK7fmthQdTWamRXcw2RgASkXks11GMrb9WbEF8awdI0dCfFgNdIMmsP3xrDv6
Evi+HnQImnowS0s19B8TJkv3KMJlKPHZalTZBeI10Qw1axgxPFA3PQ==
=3m11
-END PGP SIGNATURE-


Dovecot 2.1.7 randomly lost emails

2016-05-12 Thread Paolo Rinaldo
Hi
Dovecot 2.1.7 Postfix 2.9.6 Maildir backend, emails filtered by a Postfix 
frontend (Sqlgrey, Amavis).
Lmtp delivering emails using a simple sieve script.
Ubuntu 12.04 VMware virtual server.

When email are CCed to other recipients on the same domain (the same backend), 
sometimes they are lost.
We hosts approx. 1500 mailboxes, the problem occurs one or two times a day.

Recently I changed the frontend, the previous one was delivering one email for 
each recipient, we had a couple of error per year, we thought it was due to 
client-side error.
The frequency increased with the new frontend, no more “splitting” messages for 
more than one recipient.

The following sample is from a client using a POP3 connection, leaving emails 
on server.

the Postfix mail.log
May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: 
from=, size=11822371, nrcpt=2 (queue active)
May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: 
to=, relay=mail4.robinson.it[private/dovecot-lmtp], 
delay=0.64,  delays=0.45/0/0/0.19, dsn=2.0.0, status=sent (250 2.0.0 
 0vNzHyptMlezHgAAk+lh+g Saved)
May 11 01:26:55 lmail01 postfix/lmtp[3106]: 96E9CA6628: 
to=user-...@domain.com>, relay=mail4.robinson.it[private/dovecot-lmtp], 
delay=0.74,  delays=0.45/0/0/0.29, dsn=2.0.0, status=sent (250 2.0.0 
user-...@domain.com> 0vNzHyptMlezHgAAk+lh+g Saved)
May 11 01:26:55 lmail01 postfix/qmgr[7054]: 96E9CA6628: removed

the dovecot-info.log
May 11 01:26:55 lmtp(7859, user-...@domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: 
sieve: 
msgid=<003a2d931fff0e48a2314a1adc87e23e8e4ba...@t49-ms-exmb02.intranet.saksroot.saksinc.com>:
 stored mail into mailbox 'INBOX'
May 11 01:26:55 lmtp(7859, user-...@domain.com): Info: 0vNzHyptMlezHgAAk+lh+g: 
sieve: 
msgid=<003a2d931fff0e48a2314a1adc87e23e8e4ba...@t49-ms-exmb02.intranet.saksroot.saksinc.com>:
 stored mail into mailbox 'INBOX'

I cannot find the email 0vNzHyptMlezHgAAk+lh+g  for 
user-...@domain.com, nor I cannot suppose problems 
on the client, nothing changed on the back-end.

Please let me know if you need some more info or config dump.

Any help will be appreciated

Paolo Rinaldo

Dovecot config:
# 2.1.7: /etc/dovecot/dovecot.conf
# OS: Linux 3.11.0-15-generic x86_64 Ubuntu 12.04.4 LTS
auth_mechanisms = plain login
debug_log_path = /var/log/dovecot/dovecot-debug.log
disable_plaintext_auth = no
info_log_path = /var/log/dovecot/dovecot-info.log
log_path = /var/log/dovecot/dovecot.log
mail_plugins = " quota"
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date ihave
namespace inbox {
  inbox = yes
  location =
  prefix = INBOX.
  separator = .
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf
  driver = sql
}
plugin {
  autocreate = INBOX.Trash
  autocreate2 = INBOX.Spam
  autocreate3 = INBOX.Sent
  autocreate4 = INBOX.Drafts
  autosubscribe = INBOX.Trash
  autosubscribe2 = INBOX.Spam
  autosubscribe3 = INBOX.Sent
  autosubscribe4 = INBOX.Drafts
  quota = maildir:User quota
  quota_exceeded_message = Impossibile consegnare il messaggio: spazio 
insufficiente sulla casella di destinazione
  quota_grace = 10%%
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_global_path = /var/lib/dovecot/sieve/spam.sieve
}
protocols = imap pop3 lmtp sieve
service auth-worker {
  user = vmail
}
service auth {
  unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0660
user = postfix
  }
  unix_listener auth-userdb {
group = vmail
mode = 0600
user = vmail
  }
  user = dovecot
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
user = vmail
  }
  user = vmail
}
ssl_cert = 

Re: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox)

2016-05-12 Thread Jon Kristensen

Jon Kristensen  writes:


I'm assuming you are referring to this:

https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt


Is it only me, or this not documented at 
?


If it's not documented, should a bug be filed somewhere?

--
Jon Kristensen (jonkri)
PGP fingerprint: 0377 F488 8908 0237 A2AE C484 387E C31D DA60 1350


Re: name of mailbox of implicit keep (was Re: Sieve extension for testing for mailbox)

2016-05-12 Thread Jon Kristensen

Steffen Kaiser  writes:


On Tue, 10 May 2016, Jon Kristensen wrote:

Which Pigeonhole Sieve extension should I use to process/test a 
message 
depending on whether the message is in a particular mailbox? 
(Is it the 
mboxmetadata extension by any chance? If so, I would appreciate 
if 
someone could tell me what the reason is for it not being 
enabled by 
default.)


The reason I want to perform this test is that I have 
configured my MTA 
to deliver "outgoing" e-mails to my "Sent" IMAP folder, and 
these 
messages arrive unread. I would like to mark them as read using 
Sieve.


er, I guess, "message is in a particular mailbox" is meaning it 
to be 
spooled, or something like that, right?


Do you use subaddressing (aka +addressing, +detail addressing)? 
Then look 
here: 
http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples#Plus_Addressed_mail_filtering


If you want to resolve the question in the second paragraphe, 
search the 
mailing list archive about how to get the name of the mailbox, 
the 
implicit keep stores the message in. I'm pretty sure, that there 
was such 
question lately.


If you actually want to resolve the question in the first 
paragraphe, to 
check whether or not the same message has been spooled in a 
folder already 
(aka duplicate check), check out the duplicate extension with 
":handle" 
argument. But you probably need to know the target mailbox, too.


I apologize for not expressing my problem in clear enough terms.

My e-mail server will put sent e-mail to the ~/.Maildir/.Sent 
(which is also a Maildir directory); naturally, the e-mail is 
first written into ~/.Maildir/.Sent/tmp, and then it's moved to 
~/.Maildir/.Sent/new.


After this has happened, I would like Dovecot to run a Sieve 
program that will discover that the e-mail is in the "Sent" 
folder, and mark it as seen.


To explain what I'm imagining with code:

if folder :is "Sent" {
   setflag "\\Seen";
}

So I don't use subaddressing. I always know the target mailbox, 
but it's always "Sent" (this is configured in the MTA).


I guess knowing the name of the mailbox that implicit keep stores 
the message in would solve my problem.


I'm assuming you are referring to this:

https://raw.githubusercontent.com/dovecot/pigeonhole/master/doc/rfc/spec-bosch-sieve-dovecot-environment.txt

And in particular this:

  "vnd.dovecot.default-mailbox"
The default mailbox that is used by the (implicit) keep 
action.

Normally, this will always yield "INBOX", unless this is
changed in the configuration.

It says that this normally will yield "INBOX". Do you, given my 
explanation above, think that it would yield "Sent" in my case? If 
not, how would I be able to use this?


Thanks a million!

--
Jon Kristensen (jonkri)
PGP fingerprint: 0377 F488 8908 0237 A2AE C484 387E C31D DA60 1350


Re: How do I share folders ?

2016-05-12 Thread chaouche yacine
There's also no need to create the shared namespace, so this part below is also 
unnecessary

namespace partage {
location = maildir:%%h:INDEX=~/%%u
prefix = shared.%%u.
separator = .
type = shared
}

I've commented it out, restarted dovecot, sharing still works.

Yassine.



- Original Message -
From: chaouche yacine 
To: "dovecot@dovecot.org" 
Sent: Thursday, May 12, 2016 1:35 PM
Subject: Re: How do I share folders ?

Thanks Steffen. I went the quick'n'dirty way described at : 
http://wiki2.dovecot.org/SharedMailboxes/Symlinks and I must say it works fine 
as far as I have tested.

Here's what I did : 


$ cd /var/vmail/algerian-radio.dz/a.chaouche/

$ ln -s /var/vmail/backup.algerian-radio.dz/a.chaouche/ 
.shared.a_chaouche@backup_algerian-radio_dz  < change "." to "_"

If I didn't change the dots they would appear as subfolder separators to 
dovecot (Maildir++ specs ? )

If you do this then a.chaou...@algerian-radio.dz can do whatever he wants with 
a.chaou...@backup.algerian-radio.dz mailbox. We don't want this. We want him to 
only read the mail, not delete it. For this, he needs two permissions : reading 
of course and listing, because he needs to list the emails in order to choose 
which one he wants to read. If he doesn't have the listing permission then the 
MUA would get an error when it tries to list all the messages in the inbox.


We do this by adding a dovecot-acl file in the mailbox we want to share : 


root@messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # 
cat dovecot-acl
user=a.chaou...@algerian-radio.dz lr
root@messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # 

This says that user a.chaou...@algerian-radio.dz has the "l" right (listing) 
and the "r" right (reading).

You can make sure that it's setup as expected with the acl tool : 


root@messagerie[10.10.10.20] ~ # doveadm acl get  -u 
a.chaou...@algerian-radio.dz INBOX.shared.a_chaouche@backup_algerian-radio_dz 
IDGlobal Rights 
user=a.chaou...@algerian-radio.dzlookup read 
root@messagerie[10.10.10.20] ~ # 


1. No need for SQL tables
2. No need for dictionary mappings
3. No need for doveadm acl commands

Yassine.




From: Steffen Kaiser 
To: chaouche yacine  
Cc: "dovecot@dovecot.org" 
Sent: Wednesday, May 11, 2016 3:09 PM
Subject: Re: How do I share folders ?


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, 11 May 2016, chaouche yacine wrote:

> namespace inbox {

>  prefix = INBOX.
>  separator = .



>  type = private
> }
> namespace partage {
>  hidden = no
>  location = maildir:%%h:INDEX=~/%%u
>  prefix = shared.%%u.

^

>  separator = .
>  type = shared
> }

looks fine for me,

> I don't know if it's possible to share folders with the doveadm acl command ?
> root@messagerie[10.10.10.20] ~ # doveadm acl debug  -u 
> a.chaou...@algerian-radio.dz shared/a.chaou...@backup.algerian-radio.dz
> doveadm(a.chaou...@algerian-radio.dz): Error: Can't open mailbox 
> shared/a.chaou...@backup.algerian-radio.dz: Mailbox doesn't exist: 
> shared/a.chaou...@backup.algerian-radio.dz

you are using the dot as separator, that is:

shared/a.chaouche@backup. should spell


shared.a.chaouche@backup.

However, because there are dots in the username, too, this probably won't 
work, because Dovecot probably picks up "a" as username.

- -- 
Steffen Kaiser
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBVzM9HHz1H7kL/d9rAQIGmwgAi7ga1a1Vv8HfvcAc6mIqxkBHdI5jL3MC
YfMVwwuj7NJMjS8l/ouYxJCXdMn5k+sbRifwR/jyBcZtsvz3/ijkMx3RQyoq+LwZ
r01Mxb4YR7ub5sNUIeIRtRcR/aNfDLNZb86PpDCHs2WKLU2atS5vVLmJWNiMaJNV
MSG72OFKoLUJuRoSM26c2AH/f33WjuEinjjGMk481ezNBmZIuSWU2lJjmUXjfox9
BPzGRUkUKtlV9tkQjJUVjuwcfmvs+ChmTI3O2F37KWswP8zw0/EYXdyRpWCJso7F
8Ecrbu14cSHjisaerTQ4xko+ossPGoEE+Rh7XHuv8VDdNruur5AeVg==
=DH0D
-END PGP SIGNATURE-


Re: How do I share folders ?

2016-05-12 Thread chaouche yacine
Thanks Steffen. I went the quick'n'dirty way described at : 
http://wiki2.dovecot.org/SharedMailboxes/Symlinks and I must say it works fine 
as far as I have tested.

Here's what I did : 


$ cd /var/vmail/algerian-radio.dz/a.chaouche/

$ ln -s /var/vmail/backup.algerian-radio.dz/a.chaouche/ 
.shared.a_chaouche@backup_algerian-radio_dz  < change "." to "_"

If I didn't change the dots they would appear as subfolder separators to 
dovecot (Maildir++ specs ? )

If you do this then a.chaou...@algerian-radio.dz can do whatever he wants with 
a.chaou...@backup.algerian-radio.dz mailbox. We don't want this. We want him to 
only read the mail, not delete it. For this, he needs two permissions : reading 
of course and listing, because he needs to list the emails in order to choose 
which one he wants to read. If he doesn't have the listing permission then the 
MUA would get an error when it tries to list all the messages in the inbox.


We do this by adding a dovecot-acl file in the mailbox we want to share : 


root@messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # 
cat dovecot-acl
user=a.chaou...@algerian-radio.dz lr
root@messagerie[10.10.10.20] /var/vmail/backup.algerian-radio.dz/a.chaouche # 

This says that user a.chaou...@algerian-radio.dz has the "l" right (listing) 
and the "r" right (reading).

You can make sure that it's setup as expected with the acl tool : 


root@messagerie[10.10.10.20] ~ # doveadm acl get  -u 
a.chaou...@algerian-radio.dz INBOX.shared.a_chaouche@backup_algerian-radio_dz 
IDGlobal Rights 
user=a.chaou...@algerian-radio.dzlookup read 
root@messagerie[10.10.10.20] ~ # 


1. No need for SQL tables
2. No need for dictionary mappings
3. No need for doveadm acl commands

Yassine.



From: Steffen Kaiser 
To: chaouche yacine  
Cc: "dovecot@dovecot.org" 
Sent: Wednesday, May 11, 2016 3:09 PM
Subject: Re: How do I share folders ?


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Wed, 11 May 2016, chaouche yacine wrote:

> namespace inbox {

>  prefix = INBOX.
>  separator = .



>  type = private
> }
> namespace partage {
>  hidden = no
>  location = maildir:%%h:INDEX=~/%%u
>  prefix = shared.%%u.

^

>  separator = .
>  type = shared
> }

looks fine for me,

> I don't know if it's possible to share folders with the doveadm acl command ?
> root@messagerie[10.10.10.20] ~ # doveadm acl debug  -u 
> a.chaou...@algerian-radio.dz shared/a.chaou...@backup.algerian-radio.dz
> doveadm(a.chaou...@algerian-radio.dz): Error: Can't open mailbox 
> shared/a.chaou...@backup.algerian-radio.dz: Mailbox doesn't exist: 
> shared/a.chaou...@backup.algerian-radio.dz

you are using the dot as separator, that is:

shared/a.chaouche@backup. should spell


shared.a.chaouche@backup.

However, because there are dots in the username, too, this probably won't 
work, because Dovecot probably picks up "a" as username.

- -- 
Steffen Kaiser
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEVAwUBVzM9HHz1H7kL/d9rAQIGmwgAi7ga1a1Vv8HfvcAc6mIqxkBHdI5jL3MC
YfMVwwuj7NJMjS8l/ouYxJCXdMn5k+sbRifwR/jyBcZtsvz3/ijkMx3RQyoq+LwZ
r01Mxb4YR7ub5sNUIeIRtRcR/aNfDLNZb86PpDCHs2WKLU2atS5vVLmJWNiMaJNV
MSG72OFKoLUJuRoSM26c2AH/f33WjuEinjjGMk481ezNBmZIuSWU2lJjmUXjfox9
BPzGRUkUKtlV9tkQjJUVjuwcfmvs+ChmTI3O2F37KWswP8zw0/EYXdyRpWCJso7F
8Ecrbu14cSHjisaerTQ4xko+ossPGoEE+Rh7XHuv8VDdNruur5AeVg==
=DH0D
-END PGP SIGNATURE-


Re: [PATCH 0/7] Fixes for lib-mail message-address

2016-05-12 Thread Pali Rohár
On Thursday 21 April 2016 14:24:09 Timo Sirainen wrote:
> On 11 Apr 2016, at 13:44, Pali Rohár  wrote:
> > 
> > On Thursday 07 April 2016 16:11:12 Timo Sirainen wrote:
> >> On 06 Apr 2016, at 17:29, Pali Rohár  wrote:
> >>> 
> >>> On Monday 04 April 2016 19:35:22 Pali Rohár wrote:
>  On Monday 04 April 2016 19:31:06 aki.tu...@dovecot.fi wrote:
> > Can you please send them directly to me, please?
> > ---
> > Aki Tuomi
>  
>  Sent.
> >>> 
> >>> Received? Are patches OK now?
> >> 
> >> It would be nice if each commit was accompanied with the corresponding 
> >> unit test change in test-message-address.c. Now that the unit test changes 
> >> are done in a separate commit I'm not really sure which test change is 
> >> testing which commit or if some tests are missing. At least the "Quote and 
> >> escape strings if needed" patch needs more tests for different kinds of 
> >> escapes/atext/non-atext chars and =? in strings.
> >> 
> > 
> > Ok. If you really need that I can rework my patches. But I'm thinking
> > what to do with tests... I have written lot of examples and tests
> > scenarios in perl for that new perl module. But dovecot test framework
> > is not easy to use as perl Test::More/Test::Spec and such complicated
> > tests which I have are really hard to write directory for dovecot...
> 
> Although it would be nice to have a lot of unit tests for everything, I think 
> it would be enough to have just some to show what the change actually fixes. 
> Maybe even just in the git commit message. I remember that by looking at the 
> patches I didn't completely understand what all of the changes did.

Hi! Here are examples/test cases which patches fixes. Patches 3 and 4
should be self-explained... It is enough? Or do you need more detailed
description about problems in parser/formatter?

PATCH 1:

struct:
{ { name = NULL, mailbox = "group", domain = NULL}, { name = NULL, mailbox = 
NULL, domain = NULL } }

should be formatted as string:

group:;

before patch it was:

group;



PATCH 2:

struct:
{ { name = NULL, mailbox = "", domain = NULL}, { name = NULL, mailbox = NULL, 
domain = NULL } }

should be formatted as string:

"":;

before patch it was:

:;



PATCH 5:

input string:

 test

will be parsed as struct:

{ name = "test", mailbox = NULL, domain = NULL }

before patch it was struct:

{ name = NULL, mailbox = "test", domain = NULL }



PATCH 6:

struct:
{ name = "test\"test", mailbox = "user", domain = "host" }

should be formatted as string:

"test\"test" 

before patch it was:

test"test 


-- 
Pali Rohár
pali.ro...@gmail.com


Re: Bug Report - Crash on imap with core dump - Alternate storage related

2016-05-12 Thread Aki Tuomi
Ah, sorry, missed it =)

Aki

On 12.05.2016 12:53, Daniel van Ham Colchete wrote:
> Hi Aki!
>
> It was already there: http://pastebin.com/2xA2xuRw
>
> Best,
> Daniel Colchete
>
> On Thu, May 12, 2016 at 3:03 AM, Aki Tuomi  wrote:
>
>>
>> On 11.05.2016 22:45, Daniel van Ham Colchete wrote:
>>> I can confirm that this bug also happens on the latest version available:
>>>
>>> root@dovecot:/# dovecot --version
>>> 2.2.devel (ebf2e93)
>>> root@dovecot:/# dpkg -l|grep dovecot
>>> ii  dovecot-core   2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - core files
>>> ii  dovecot-dbg2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - debug symbols
>>> ii  dovecot-imapd  2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - IMAP daemon
>>> ii  dovecot-lmtpd  2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - LMTP server
>>> ii  dovecot-lucene 2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - Lucene support
>>> ii  dovecot-mysql  2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - MySQL support
>>> ii  dovecot-pop3d  2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - POP3 daemon
>>> ii  dovecot-sieve  2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - Sieve filters support
>>> ii  dovecot-solr   2:2.2.24-1~auto+23amd64
>>>  secure POP3/IMAP server - Solr support
>>>
>>>
>>> On Wed, May 11, 2016 at 4:38 PM, Daniel van Ham Colchete <
>>> daniel.colch...@gmail.com> wrote:
>>>
 I would like to report a bug we are seeing on an new deployment here:

 Linux: Debian Jessie, inside a Docker container
 Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1
 doveconf -n: http://pastebin.com/2xA2xuRw
 Filesystem: CephFS - Jewel

 After migrating our users from our old server to our new one we can't
 access a few mailboxes. Backtrace follows:

 root@dovecot:/# gdb --args /usr/lib/dovecot/imap -u
 daniel.colch...@mav.com.br
 GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
 Copyright (C) 2014 Free Software Foundation, Inc.
 License GPLv3+: GNU GPL version 3 or later <
 http://gnu.org/licenses/gpl.html>
 This is free software: you are free to change and redistribute it.
 There is NO WARRANTY, to the extent permitted by law.  Type "show
>> copying"
 and "show warranty" for details.
 This GDB was configured as "x86_64-linux-gnu".
 Type "show configuration" for configuration details.
 For bug reporting instructions, please see:
 .
 Find the GDB manual and other documentation resources online at:
 .
 For help, type "help".
 Type "apropos word" to search for commands related to "word"...
 Reading symbols from /usr/lib/dovecot/imap...Reading symbols from

>> /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done.
 done.
 (gdb) run
 Starting program: /usr/lib/dovecot/imap -u daniel.colch...@mav.com.br
 * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
>> ENABLE
 IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
>> THREAD=ORDEREDSUBJECT
 MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
 LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
>> WITHIN
 CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as
 daniel.colch...@mav.com.br
 a select INBOX

 Program received signal SIGSEGV, Segmentation fault.
 strrchr () at ../sysdeps/x86_64/strrchr.S:32
 32  ../sysdeps/x86_64/strrchr.S: No such file or directory.
 (gdb) where
 #0  strrchr () at ../sysdeps/x86_64/strrchr.S:32
 #1  0x77b0a261 in sdbox_file_move (file=file@entry
>> =0x557e6530,
 alt_path=alt_path@entry=true) at sdbox-file.c:312
 #2  0x77b0cd11 in dbox_sync_file_move_if_needed
 (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x557e6530) at
 sdbox-sync.c:30
 #3  sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120,
 seq=819, ctx=0x557e32e0) at sdbox-sync.c:58
 #4  sdbox_sync_add (sync_rec=0x7fffe570, ctx=0x557e32e0) at
 sdbox-sync.c:96
 #5  sdbox_sync_index (ctx=0x557e32e0) at sdbox-sync.c:131
 #6  sdbox_sync_begin (mbox=mbox@entry=0x557ce500, flags=flags@entry
>> =(unknown:
 0), ctx_r=ctx_r@entry=0x7fffe680) at sdbox-sync.c:234
 #7  0x77b0d07d in sdbox_sync (mbox=mbox@entry=0x557ce500,
 flags=flags@entry=(unknown: 0)) at sdbox-sync.c:297
 #8  0x77b0d108 in sdbox_storage_sync_init (box=0x557ce500,
 flags=(MAILBOX_SYNC_FLAG_FULL_READ |
>> MAILBOX_SYNC_FLAG_FIX_INCONSISTENT))
 at sdbox-sync.c:323
 #9  0x77aea85b in mailbox_sync_init (box=0x557ce500,
 flags=(MAILBOX_SY

Re: Bug Report - Crash on imap with core dump - Alternate storage related

2016-05-12 Thread Daniel van Ham Colchete
Hi Aki!

It was already there: http://pastebin.com/2xA2xuRw

Best,
Daniel Colchete

On Thu, May 12, 2016 at 3:03 AM, Aki Tuomi  wrote:

>
>
> On 11.05.2016 22:45, Daniel van Ham Colchete wrote:
> > I can confirm that this bug also happens on the latest version available:
> >
> > root@dovecot:/# dovecot --version
> > 2.2.devel (ebf2e93)
> > root@dovecot:/# dpkg -l|grep dovecot
> > ii  dovecot-core   2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - core files
> > ii  dovecot-dbg2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - debug symbols
> > ii  dovecot-imapd  2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - IMAP daemon
> > ii  dovecot-lmtpd  2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - LMTP server
> > ii  dovecot-lucene 2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - Lucene support
> > ii  dovecot-mysql  2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - MySQL support
> > ii  dovecot-pop3d  2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - POP3 daemon
> > ii  dovecot-sieve  2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - Sieve filters support
> > ii  dovecot-solr   2:2.2.24-1~auto+23amd64
> >  secure POP3/IMAP server - Solr support
> >
> >
> > On Wed, May 11, 2016 at 4:38 PM, Daniel van Ham Colchete <
> > daniel.colch...@gmail.com> wrote:
> >
> >> I would like to report a bug we are seeing on an new deployment here:
> >>
> >> Linux: Debian Jessie, inside a Docker container
> >> Dovecot version: 2.2.23 (806d709) / 2.2.23-1~bpo8+1
> >> doveconf -n: http://pastebin.com/2xA2xuRw
> >> Filesystem: CephFS - Jewel
> >>
> >> After migrating our users from our old server to our new one we can't
> >> access a few mailboxes. Backtrace follows:
> >>
> >> root@dovecot:/# gdb --args /usr/lib/dovecot/imap -u
> >> daniel.colch...@mav.com.br
> >> GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
> >> Copyright (C) 2014 Free Software Foundation, Inc.
> >> License GPLv3+: GNU GPL version 3 or later <
> >> http://gnu.org/licenses/gpl.html>
> >> This is free software: you are free to change and redistribute it.
> >> There is NO WARRANTY, to the extent permitted by law.  Type "show
> copying"
> >> and "show warranty" for details.
> >> This GDB was configured as "x86_64-linux-gnu".
> >> Type "show configuration" for configuration details.
> >> For bug reporting instructions, please see:
> >> .
> >> Find the GDB manual and other documentation resources online at:
> >> .
> >> For help, type "help".
> >> Type "apropos word" to search for commands related to "word"...
> >> Reading symbols from /usr/lib/dovecot/imap...Reading symbols from
> >>
> /usr/lib/debug/.build-id/e2/488f839ac2b10504ef74b1b2f2146891ca2f62.debug...done.
> >> done.
> >> (gdb) run
> >> Starting program: /usr/lib/dovecot/imap -u daniel.colch...@mav.com.br
> >> * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
> ENABLE
> >> IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS
> THREAD=ORDEREDSUBJECT
> >> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS
> >> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES
> WITHIN
> >> CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in as
> >> daniel.colch...@mav.com.br
> >> a select INBOX
> >>
> >> Program received signal SIGSEGV, Segmentation fault.
> >> strrchr () at ../sysdeps/x86_64/strrchr.S:32
> >> 32  ../sysdeps/x86_64/strrchr.S: No such file or directory.
> >> (gdb) where
> >> #0  strrchr () at ../sysdeps/x86_64/strrchr.S:32
> >> #1  0x77b0a261 in sdbox_file_move (file=file@entry
> =0x557e6530,
> >> alt_path=alt_path@entry=true) at sdbox-file.c:312
> >> #2  0x77b0cd11 in dbox_sync_file_move_if_needed
> >> (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, file=0x557e6530) at
> >> sdbox-sync.c:30
> >> #3  sdbox_sync_file (type=SDBOX_SYNC_ENTRY_TYPE_MOVE_TO_ALT, uid=13120,
> >> seq=819, ctx=0x557e32e0) at sdbox-sync.c:58
> >> #4  sdbox_sync_add (sync_rec=0x7fffe570, ctx=0x557e32e0) at
> >> sdbox-sync.c:96
> >> #5  sdbox_sync_index (ctx=0x557e32e0) at sdbox-sync.c:131
> >> #6  sdbox_sync_begin (mbox=mbox@entry=0x557ce500, flags=flags@entry
> =(unknown:
> >> 0), ctx_r=ctx_r@entry=0x7fffe680) at sdbox-sync.c:234
> >> #7  0x77b0d07d in sdbox_sync (mbox=mbox@entry=0x557ce500,
> >> flags=flags@entry=(unknown: 0)) at sdbox-sync.c:297
> >> #8  0x77b0d108 in sdbox_storage_sync_init (box=0x557ce500,
> >> flags=(MAILBOX_SYNC_FLAG_FULL_READ |
> MAILBOX_SYNC_FLAG_FIX_INCONSISTENT))
> >> at sdbox-sync.c:323
> >> #9  0x77aea85b in mailbox_sync_init (box=0x557ce500,
> >> flags=(MAILBOX_SYNC_FLAG_FULL_READ |
> MAILBOX_SYNC_FLAG_FIX_INCONSISTENT))
> >> at mail-storage.c:1664
> >> #10 0x7f

Re: Dovecot + libsodium

2016-05-12 Thread Aki Tuomi


On 11.05.2016 14:50, Andreas Meyer wrote:
> 2016-05-06 23:15 GMT+02:00 Timo Sirainen :
>
>> On 06 May 2016, at 13:14, Andreas Meyer  wrote:
>>> Hi,
>>>
>>> Thank you very much for creating and maintaining dovecot!
>>>
>>> In my scenario, I want to use the password hash algorithms provided by
>>> libsodium: https://download.libsodium.org/doc/
>>>
>>> So my difficulty is to have dovecot support libsodium's hash algorithms,
>>> particularly: crypto_pwhash_scryptsalsa208sha256_str
>>>
>>> On the sodium maillinglist I asked for help and received an adjusted
>>> dovecot code, which exactly does what I need. You find it here:
>>> https://github.com/jedisct1/core/tree/scrypt-argon2
>>>
>>> Obviously I need to apply these changes everytime I upgrade to a new
>>> dovecot version now.
>>>
>>> So my question ist, what do I need to do so that you will include
>> libsodium
>>> support in future versions of dovecot?
>> You could also change it to be a plugin to avoid patching. This is a
>> pretty old example, but it probably still works, at least with minor
>> changes:
>> http://dovecot.org/patches/password-scheme-lmpass.c
>>
>> Although it's still a good idea to recompile the plugin after a new
>> version since sometimes the ABI changes.
>>
>>
> Hi Timo,
>
>
> thank you very much for your reply. Creating a plugin is an option. Though
> I don't possess the right abilities to do that right away.
>
> Nevertheless I want to re-ask my initial question: What is required to get
> libsodium support into the dovecot core?
> Or are there concerns about supporting it or is there simply no interest in
> doing so?
>
> As I understand, security is a relevant concern when developing Dovecot.
> The sodium crypto library focuses on: "... provide all of the core
> operations needed to build higher-level cryptographic tools."
> I am sure, utilizing this library by default can be of great benefit for
> Dovecot. It will help to easily support the latest password hashing
> algorithms, currently Scrypt and Argon2.
> And if used for additional cryptographic purposes, it also provides easy to
> use cryptographically secure pseudo random data, secret-key authenticated
> encryption and of course secure memory allocations, just to name three
> features.
>
>
> Thank you very much,
>
> Andreas

Hi!

At the moment we do not want to add libsodium as permanent core
dependency. If you are willing to make a plugin for sodium, that
provides f.ex. Scrypt and Argon2 support, and make a pull request out of
that, we are happy to include it as plugin.

Due to how dovecot works adding new libraries to certain places is not
really wanted, since the memory requirements on large installations will
increase quite a lot.

We already are able to handle most of the things sodium would provide
with current code.

Aki