Re: UNSUBSCRIBE

[Stanislav via dovecot]

UNSUBSCRIBE

Gianfranco Bartolini писал 2018-11-30 10:52:

UNSUBSCRIBE


-Original Message-
From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf
Of Michael A. Peters
Sent: Friday, November 30, 2018 8:21 AM
To: dovecot@dovecot.org
Subject: Re: DMARC policies

On 11/29/2018 11:13 PM, Aki Tuomi wrote:
> Hi!
>
> It seems we accidentically had a high amount of subscribers
temporarily
> disabled due to DMARC on some sender's host. We have now
taken actions
> to prevent this in the future and all temporarily disabled
members have
> been restored.
>
> Aki
>
>

I've seen that happen on several lists.

I disabled DMARC on my mail servers. I really like the
concept but way
way way too many false positives.

Mostly lists, but also what sometimes happens - primary MX for
whatever.com is down so mail goes to their backup with then
relays it to
their primary when primary back up, but their backup MX
obviously isn't
in my SPF record and they have things mis-configured causing a DMARC
trigger.

I like the concept but it seems to have too many problems in
implementation.

Re: UNSUBSCRIBE

[Aki Tuomi]

Please visit

https://dovecot.org/mailman/listinfo/dovecot

to unsubscribe

Aki

On 30.11.2018 9.52, Gianfranco Bartolini wrote:
> UNSUBSCRIBE 
>
>> -Original Message-
>> From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf 
>> Of Michael A. Peters
>> Sent: Friday, November 30, 2018 8:21 AM
>> To: dovecot@dovecot.org
>> Subject: Re: DMARC policies
>>
>> On 11/29/2018 11:13 PM, Aki Tuomi wrote:
>>> Hi!
>>>
>>> It seems we accidentically had a high amount of subscribers 
>> temporarily
>>> disabled due to DMARC on some sender's host. We have now 
>> taken actions
>>> to prevent this in the future and all temporarily disabled 
>> members have
>>> been restored.
>>>
>>> Aki
>>>
>>>
>> I've seen that happen on several lists.
>>
>> I disabled DMARC on my mail servers. I really like the 
>> concept but way 
>> way way too many false positives.
>>
>> Mostly lists, but also what sometimes happens - primary MX for 
>> whatever.com is down so mail goes to their backup with then 
>> relays it to 
>> their primary when primary back up, but their backup MX 
>> obviously isn't 
>> in my SPF record and they have things mis-configured causing a DMARC 
>> trigger.
>>
>> I like the concept but it seems to have too many problems in 
>> implementation.
>>

UNSUBSCRIBE

[Gianfranco Bartolini]

UNSUBSCRIBE 

> -Original Message-
> From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf 
> Of Michael A. Peters
> Sent: Friday, November 30, 2018 8:21 AM
> To: dovecot@dovecot.org
> Subject: Re: DMARC policies
> 
> On 11/29/2018 11:13 PM, Aki Tuomi wrote:
> > Hi!
> > 
> > It seems we accidentically had a high amount of subscribers 
> temporarily
> > disabled due to DMARC on some sender's host. We have now 
> taken actions
> > to prevent this in the future and all temporarily disabled 
> members have
> > been restored.
> > 
> > Aki
> > 
> > 
> 
> I've seen that happen on several lists.
> 
> I disabled DMARC on my mail servers. I really like the 
> concept but way 
> way way too many false positives.
> 
> Mostly lists, but also what sometimes happens - primary MX for 
> whatever.com is down so mail goes to their backup with then 
> relays it to 
> their primary when primary back up, but their backup MX 
> obviously isn't 
> in my SPF record and they have things mis-configured causing a DMARC 
> trigger.
> 
> I like the concept but it seems to have too many problems in 
> implementation.
> 

Re: DMARC policies

[Michael A. Peters]

On 11/29/2018 11:13 PM, Aki Tuomi wrote:

Hi!

It seems we accidentically had a high amount of subscribers temporarily
disabled due to DMARC on some sender's host. We have now taken actions
to prevent this in the future and all temporarily disabled members have
been restored.

Aki




I've seen that happen on several lists.

I disabled DMARC on my mail servers. I really like the concept but way 
way way too many false positives.


Mostly lists, but also what sometimes happens - primary MX for 
whatever.com is down so mail goes to their backup with then relays it to 
their primary when primary back up, but their backup MX obviously isn't 
in my SPF record and they have things mis-configured causing a DMARC 
trigger.


I like the concept but it seems to have too many problems in implementation.

DMARC policies

[Aki Tuomi]

Hi!

It seems we accidentically had a high amount of subscribers temporarily
disabled due to DMARC on some sender's host. We have now taken actions
to prevent this in the future and all temporarily disabled members have
been restored.

Aki

Re: [2.3.4] Segmentation faults

[Joan Moreau]

ANother (very very long) example : 


# gdb /usr/libexec/dovecot/indexer-worker
core.indexer-worker.0.3a33f56105e043de802a7dfcee265a07.21017.154353342400
GNU gdb (GDB) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
. 


For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/libexec/dovecot/indexer-worker...done.
[New LWP 21017]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `dovecot/indexer-worker'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x7f768b62b13e in file_lock_do (fd=18, path=0x564540376790
"/data/mail/grosjo.net/admin/mailboxes/QoS/dbox-Mails/dovecot.index.search",
lock_type=0, 
lock_method=FILE_LOCK_METHOD_FCNTL, timeout_secs=60,

error_r=0x7fff045010b0) at file-lock.c:173
173 {
(gdb) bt full
#0 0x7f768b62b13e in file_lock_do (fd=18, path=0x564540376790
"/data/mail/grosjo.net/admin/mailboxes/QoS/dbox-Mails/dovecot.index.search",
lock_type=0, 
lock_method=FILE_LOCK_METHOD_FCNTL, timeout_secs=60,

error_r=0x7fff045010b0) at file-lock.c:173
lock_type_str = 
started = 
ret = 
__func__ = "file_lock_do"
#1 0x7f768b62b5b6 in file_wait_lock_error (fd=18,
path=0x564540376790
"/data/mail/grosjo.net/admin/mailboxes/QoS/dbox-Mails/dovecot.index.search",
lock_type=0, 
lock_method=FILE_LOCK_METHOD_FCNTL, timeout_secs=,

lock_r=0x7fff04501118, error_r=0x7fff045010b0) at file-lock.c:318
lock = 
ret = 
#2 0x7f768b62b660 in file_wait_lock (fd=,
path=, lock_type=lock_type@entry=0,
lock_method=, timeout_secs=timeout_secs@entry=60, 
lock_r=lock_r@entry=0x7fff04501118) at file-lock.c:303

error = 0x564540376490 ""
ret = 
#3 0x7f768a976c87 in squat_trie_lock (trie=0x564540376490,
lock_type=0, file_lock_r=0x7fff04501118, dotlock_r=0x7fff04501120) at
squat-trie.c:294
ret = 
dotlock_r = 0x7fff04501120
file_lock_r = 0x7fff04501118
trie = 0x564540376490
ret = 
__func__ = "squat_trie_lock"
lock_type = 0
ret = 
__func__ = "squat_trie_lock"
#4 0x7f768a978627 in squat_trie_map (trie=0x564540376490,
building=) at squat-trie.c:1487
file_lock = 0x0
dotlock = 0x0
changed = 
ret = 
#5 0x7f768a97b19d in squat_uidlist_map_header
(uidlist=0x5645403767f0) at squat-uidlist.c:378
No locals.
#6 squat_uidlist_map (uidlist=0x5645403767f0) at squat-uidlist.c:477
mmap_hdr = 
ret = 
#7 0x7f768a97b432 in squat_uidlist_open (uidlist=0x5645403767f0) at
squat-uidlist.c:546
No locals.
#8 0x7f768a97b5aa in squat_uidlist_refresh (uidlist=)
at squat-uidlist.c:569
No locals.
#9 0x7f768a9787c2 in squat_trie_map (trie=0x564540376490,
building=) at squat-trie.c:1526
file_lock = 0x56454210c850
dotlock = 0x0
changed = 
ret = 
#10 0x7f768a97b19d in squat_uidlist_map_header
(uidlist=0x5645403767f0) at squat-uidlist.c:378
No locals.
#11 squat_uidlist_map (uidlist=0x5645403767f0) at squat-uidlist.c:477
mmap_hdr = 
ret = 
#12 0x7f768a97b432 in squat_uidlist_open (uidlist=0x5645403767f0) at
squat-uidlist.c:546
No locals.
#13 0x7f768a97b5aa in squat_uidlist_refresh (uidlist=) at squat-uidlist.c:569
No locals.
#14 0x7f768a9787c2 in squat_trie_map (trie=0x564540376490,
building=) at squat-trie.c:1526
file_lock = 0x56454210c7b0
dotlock = 0x0
changed = 
ret = 
#15 0x7f768a97b19d in squat_uidlist_map_header
(uidlist=0x5645403767f0) at squat-uidlist.c:378
No locals.
#16 squat_uidlist_map (uidlist=0x5645403767f0) at squat-uidlist.c:477
mmap_hdr = 
ret = 
#17 0x7f768a97b432 in squat_uidlist_open (uidlist=0x5645403767f0) at
squat-uidlist.c:546
No locals.
#18 0x7f768a97b5aa in squat_uidlist_refresh (uidlist=) at squat-uidlist.c:569
No locals.
#19 0x7f768a9787c2 in squat_trie_map (trie=0x564540376490,
building=) at squat-trie.c:1526
file_lock = 0x56454210c710
dotlock = 0x0
changed = 
ret = 
#20 0x7f768a97b19d in squat_uidlist_map_header
(uidlist=0x5645403767f0) at squat-uidlist.c:378
No locals.
#21 squat_uidlist_map (uidlist=0x5645403767f0) at squat-uidlist.c:477
mmap_hdr = 
ret = 
#22 0x7f768a97b432 in squat_uidlist_open (uidlist=0x5645403767f0) at
squat-uidlist.c:546
No locals.
#23 0x7f768a97b5aa in squat_uidlist_refresh (uidlist=) at squat-uidlist.c:569
No locals.
#24 0x7f768a9787c2 in squat_trie_map (trie=0x564540376490,
building=) at squat-trie.c:1526
file_lock = 0x56454210c670
dotlock = 0x0
changed = 
ret = 
#25 0x7f768a97b19d in squat_uidlist_map_header

Re: Error logs with virtual folder after upgrading to 2.3.4

[Florian Pritz]

On Wed, Nov 28, 2018 at 07:45:00PM +0200, Aki Tuomi 
 wrote:
> ProtectSystem=no
> 
> if you want writable /etc directory.

No need to disable everything just for that. This should be enough:

[Service]
ReadWritePaths=/etc/dovecot/virtual

Also you can use `systemctl edit dovecot.service` to let systemctl
handle the file name and daemon reloading. Assuming you have a recent
enough version, but I think that feature is kind of old.

Florian


signature.asc
Description: PGP signature

Re: Best way of merging mbox files

[Joseph Tam]

On Thu, 29 Nov 2018, Ruben Safir wrote:


aside from cat?


"cat" is fine if you're OK to have all your message in block sorted
(as opposed to globally) sorted order.  I think I made that plain in
my writeup.

Most mail readers will sort by the order in which it appears in
the mailbox.  If you want it time order (i.e. newest first/last),
and the mailbox is not sorted this way, you'll incur the overhead
of sorting it each time you have an index view of your message
(unless your mail reader caches these things).

Joseph Tam 

Re: Best way of merging mbox files

[Ruben Safir]

aside from cat?

On Thu, Nov 29, 2018 at 03:07:58PM -0800, Joseph Tam wrote:
> On Thu, 29 Nov 2018, Marc Roos wrote:
> 
> >When concatenating mbox files like described here
> >https://xaizek.github.io/2013-03-30/merge-mbox-mailboxes/. You will end
> >up with an 'unsorted' mbox file. Is this going to be a problem
> >esspecially when they are large >2GB's and new emails will be written to
> >it?
> 
> I don't think it will be a problem, but you might have to remove
> some headers (like the UUID header?).  However, I think dovecot
> ought to be able to cope with it anyways and regenerate the indices.
> 
> >The email client nicely sorts the message from folder A "foldera 5 last"
> >as last, but of course the mbox is not like this.
> >Is there a better solution for merging files?
> 
> As noted, the time order gets scrambled -- using your mail reader to
> get it back in time order requires sorting, an intensive operation.
> 
> It just so happen I've done this recently with a (GNU) awk script that
> merges multiple mailboxes into one mailbox, preserving time order.
> It assumes that each message starst with a From envelopes header with
> sorted timestamps e.g.
> 
>   From mic...@disney.com  Thu Nov 25 18:45:37 2018
>   From mic...@disney.com  Thu Nov 25 18:45:37 2018 -0400
> 
> Your're welcome to use it.  There's probably a more elegant way with
> doveadm/dsync.  Using a mail reader to sort the merged mailbox, then
> drag/drop/copy everything into a final mailbox could also work.
> 
> Joseph Tam 
> 
> #!/bin/sh
> #
> # Merge multiple mbox's into one assuming that each message
> # starts with /^From .* {year}$/ and they are sorted by time.
> #
> # -- Joseph Tam 
> #
> 
> [ x"$*" = x ] && {
>   echo "Usage:  $0 mbox-file ..."
>   exit 1
> }
> 
> gawk -v boxes="$*"function Tstamp(header) {
>   # Format:   Jan 22 21:00:48 2018 -0700
>   #   12345678901234567890123456
>   l = length(header)
>   spec = (substr(header,l-4,1)=="-")? substr(header,l-25,20) : 
> substr(header,l-19,20)
>   spec = substr(spec,17,4) " " ym[substr(spec,1,3)] 
> substr(spec,4,3) \
>" " substr(spec,8,2) " " substr(spec,11,2) " " 
> substr(spec,14,2)
>   return int(mktime(spec))
> 
>   }
> 
>   function DumpMessage(i) {
>   if (header[i]!="") {
>   printf("%s\n",header[i])
>   }
>   while ((getline x 0) {
>   if (x~/^From .*[0-9][0-9][0-9][0-9]$/) {
>   stamp[i] = Tstamp(x)
>   header[i] = x
>   printf("%s => [%d] %d\n",header[i],i,stamp[i]) 
> >"/dev/stderr"
>   return
>   }
>   print x
>   }
> 
>   printf("EOF[%d]\n",i) >"/dev/stderr"
>   stamp[i] = 2147483647
>   header[i] = ""
>   }
> 
>   BEGIN {
> ym["Jan"] = "01"; ym["Feb"] = "02"; ym["Mar"] = "03"; 
> ym["Apr"] = "04"
> ym["May"] = "05"; ym["Jun"] = "06"; ym["Jul"] = "07"; 
> ym["Aug"] = "08"
> ym["Sep"] = "09"; ym["Oct"] = "10"; ym["Nov"] = "11"; 
> ym["Dec"] = "12"
> 
>   n = split(boxes,mbox," ")
> 
>   # Read first header line from all boxes
>   for (i=1; i<=n; i++) {
>DumpMessage(i)
>   }
> 
>   # Loop until all maiboxes read
>   while (1) {
>   t = 2147483646
> 
>   # Find next message
>   for (i=1; i<=n; i++) {
>   if (stamp[i]<=t) {t=stamp[i]; j=i;}
>   }
> 
>   # If no more message, quit
>   if (t==2147483646) exit
> 
>   # Dump next message from mbox[j]
>   DumpMessage(j)
>   }
>   }'

-- 
So many immigrant groups have swept through our town
that Brooklyn, like Atlantis, reaches mythological
proportions in the mind of the world - RI Safir 1998
http://www.mrbrklyn.com 

DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002
http://www.nylxs.com - Leadership Development in Free Software
http://www2.mrbrklyn.com/resources - Unpublished Archive 
http://www.coinhangout.com - coins!
http://www.brooklyn-living.com 

Being so tracked is for FARM ANIMALS and and extermination camps, 
but incompatible with living as a free human being. -RI Safir 2013

Re: Best way of merging mbox files

[Joseph Tam]

On Thu, 29 Nov 2018, Marc Roos wrote:


When concatenating mbox files like described here
https://xaizek.github.io/2013-03-30/merge-mbox-mailboxes/. You will end
up with an 'unsorted' mbox file. Is this going to be a problem
esspecially when they are large >2GB's and new emails will be written to
it?


I don't think it will be a problem, but you might have to remove
some headers (like the UUID header?).  However, I think dovecot
ought to be able to cope with it anyways and regenerate the indices.


The email client nicely sorts the message from folder A "foldera 5 last"
as last, but of course the mbox is not like this.
Is there a better solution for merging files?


As noted, the time order gets scrambled -- using your mail reader to
get it back in time order requires sorting, an intensive operation.

It just so happen I've done this recently with a (GNU) awk script that
merges multiple mailboxes into one mailbox, preserving time order.
It assumes that each message starst with a From envelopes header with
sorted timestamps e.g.

From mic...@disney.com  Thu Nov 25 18:45:37 2018
From mic...@disney.com  Thu Nov 25 18:45:37 2018 -0400

Your're welcome to use it.  There's probably a more elegant way with
doveadm/dsync.  Using a mail reader to sort the merged mailbox, then
drag/drop/copy everything into a final mailbox could also work.

Joseph Tam 

#!/bin/sh
#
# Merge multiple mbox's into one assuming that each message
# starts with /^From .* {year}$/ and they are sorted by time.
#
#   -- Joseph Tam 
#

[ x"$*" = x ] && {
echo "Usage:  $0 mbox-file ..."
exit 1
}

gawk -v boxes="$*" 0) {
if (x~/^From .*[0-9][0-9][0-9][0-9]$/) {
stamp[i] = Tstamp(x)
header[i] = x
printf("%s => [%d] %d\n",header[i],i,stamp[i]) 
>"/dev/stderr"
return
}
print x
}

printf("EOF[%d]\n",i) >"/dev/stderr"
stamp[i] = 2147483647
header[i] = ""
}

BEGIN {
ym["Jan"] = "01"; ym["Feb"] = "02"; ym["Mar"] = "03"; ym["Apr"] = 
"04"
ym["May"] = "05"; ym["Jun"] = "06"; ym["Jul"] = "07"; ym["Aug"] = 
"08"
ym["Sep"] = "09"; ym["Oct"] = "10"; ym["Nov"] = "11"; ym["Dec"] = 
"12"

n = split(boxes,mbox," ")

# Read first header line from all boxes
for (i=1; i<=n; i++) {
 DumpMessage(i)
}

# Loop until all maiboxes read
while (1) {
t = 2147483646

# Find next message
for (i=1; i<=n; i++) {
if (stamp[i]<=t) {t=stamp[i]; j=i;}
}

# If no more message, quit
if (t==2147483646) exit

# Dump next message from mbox[j]
DumpMessage(j)
}
}'

Re: [2.3.4] Segmentation faults

[Joan Moreau]

finally managed to locate the dump 

here the output: 


# gdb /usr/libexec/dovecot/auth
/var/lib/systemd/coredump/core.auth.0.3a33f56105e043de802a7dfcee265a07.28130.154351611800
GNU gdb (GDB) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
. 


For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/libexec/dovecot/auth...done.
[New LWP 28130]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `dovecot/auth wor'.
Program terminated with signal SIGABRT, Aborted.
#0 0x7f739c35cd7f in raise () from /usr/lib/libc.so.6
(gdb) bt full
#0 0x7f739c35cd7f in raise () from /usr/lib/libc.so.6
No symbol table info available.
#1 0x7f739c347672 in abort () from /usr/lib/libc.so.6
No symbol table info available.
#2 0x7f739c39f878 in __libc_message () from /usr/lib/libc.so.6
No symbol table info available.
#3 0x7f739c3a618a in malloc_printerr () from /usr/lib/libc.so.6
No symbol table info available.
#4 0x7f739c3a7b27 in _int_free () from /usr/lib/libc.so.6
No symbol table info available.
#5 0x7f739cc33585 in mysql_close (mysql=0x5636af7abdc0) at
/usr/src/mariadb-10.3.11/libmariadb/libmariadb/mariadb_lib.c:1939
mysql = 0x5636af7abdc0
p = 
p = 
#6 0x5636aef08f7c in driver_sqlpool_disconnect (_db=0x5636af7aaf30)
at driver-sqlpool.c:590
conn__foreach_end = 0x5636af7ab570
db = 0x5636af7aaf30
conn = 0x5636af7ab560
#7 0x5636aeefed65 in db_sql_unref (_conn=0x5636af7b0020) at
db-sql.c:128
conn = 0x5636af7ae388
#8 0x5636aeef7e15 in userdb_deinit (userdb=0x5636af7afff8) at
userdb.c:191
idx = 0
__func__ = "userdb_deinit"
#9 0x5636aeede209 in auth_deinit (auth=0x5636af7afd58) at auth.c:335
passdb = 
userdb = 0x5636af7aff18
passdb = 
userdb = 
#10 auths_deinit () at auth.c:433
auth__foreach_end = 0x5636af7ae2f8
auth = 0x5636af7ae2f0
#11 0x5636aeedcf65 in main_deinit () at main.c:270
l = 
l = 
l_end = 
#12 main (argc=, argv=) at main.c:401
c = 

On 2018-11-29 18:07, Joan Moreau wrote:

Can't find any "core" files (updatedb ; locate "core"). Coredump are usually in /var/liv/systemd/coredump for other programs, but nothing for dovecot. 

Looks like issue is in 'auth' and 'indexer-worker'. Where can be the coredump files ? 


On 2018-11-28 18:13, Timo Sirainen wrote: See 
https://dovecot.org/bugreport.html#coredumps

Without a backtrace it's not really possible to figure out where it's crashing. 

On 28 Nov 2018, at 13.20, Joan Moreau  wrote: 


Where to get that ?

On 2018-11-27 08:50, Aki Tuomi wrote: 

It's still missing core dump (or bt full from it) 


Aki

On 27.11.2018 8.39, Joan Moreau wrote: 

Thank you Aki 

here the requested data (below) 

Please not as well that we have numerous subfolders (>50) and pretty big mailbox sizes (>20G) 

Bug appears mostly in auth process and index-worker 

dovecot -n : 


# 2.4.devel (de42b54aa): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.6.devel (65909cfa)
# OS: Linux 4.19.4-arch1-1-ARCH x86_64 ext4
# Hostname: gjserver
base_dir = /run/dovecot
default_login_user = dovecot
default_vsz_limit = 16 G
disable_plaintext_auth = no
listen = *
log_path = /var/log/mail/dovecot.log
mail_gid = mail
mail_location = mdbox:/data/mail/%d/%n:ALT=/data/mail/archives/%d/%n
mail_plugins = fts fts_squat
mail_uid = mailusers
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
mdbox_rotate_size = 24 M 

(...) 


passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
(the rest default values)

On 2018-11-25 08:08, Aki Tuomi wrote: 

On 25 November 2018 at 06:29 Joan Moreau < j...@grosjo.net> wrote: 

Hi 

THis is the lines I have in my dmesg (see below) 

In dovecot log , I see: 

Nov 25 04:26:47 auth-worker: Error: double free or corruption (fasttop) 

What do to about it ? 

Using lastest 2.3.4 version 

Thank you 

 

[132932.169265] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00 
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 aa 36 f5 7a 7f 00 
00 <40> 24 3d f5 7a 7f 00 00 21 80 00 00 00 00 00 00 00 94 16 d6 ee 55 
[134031.969596] auth[27031]: segfault at 55e509612c30 ip 
55e509612c30 sp 

"merging" folders Sent-Items Sent-Messages Sent

[Marc Roos]

I have a running server and would like to clean up a bit, and merge sent 
folders and trash folders etc.

Is this best done via the mailbox alias plugin like mentioned here? Or 
is there a better way? 

I prefer to have one folder listed at the client, and not "Unfortunately 
it also means that some clients will download the same mails to local 
cache multiple times."
https://wiki2.dovecot.org/Plugins/MailboxAlias


dovecot-2.2.10-8.el7.x86_64
CentOS Linux release 7.5.1804 (Core)

Relocate subfolders to different mount point

[Marc Roos]

I would like to relocate the subfolders of the Sent folder to a 
different location. Now send messages mail is stored in mbox format at 
the location /home/users/testuser/mail/.Sent. 
I would like to store the subfolders Sent/2017 and Sent/2018 at the 
location /home/archive/users/testuser/.Sent.2017 and 
/home/archive/users/testuser/.Sent.2018

Is this even possible?

I am using currently using this mail_location:
mail_location = 
mbox:~/mail:INBOX=/var/spool/mail/%u:CONTROL=~/mail/control:INDEX=/var/d
ovecot/%u/index:LAYOUT=maildir++

/var/dovecot is on a ssd
/var/spool/mail is on sda
~/mail is on sdb with these folders

[@mail]# ls -ac1
.folderc
.Drafts
.foldera
control
.Trash.folderc
.folderb
.Sent
.Sent.2018
.Trash

Re: [2.3.4] Segmentation faults

[Joan Moreau]

Can't find any "core" files (updatedb ; locate "core"). Coredump are
usually in /var/liv/systemd/coredump for other programs, but nothing for
dovecot. 


Looks like issue is in 'auth' and 'indexer-worker'. Where can be the
coredump files ? 


On 2018-11-28 18:13, Timo Sirainen wrote:


See https://dovecot.org/bugreport.html#coredumps

Without a backtrace it's not really possible to figure out where it's crashing. 

On 28 Nov 2018, at 13.20, Joan Moreau  wrote: 


Where to get that ?

On 2018-11-27 08:50, Aki Tuomi wrote: 

It's still missing core dump (or bt full from it) 


Aki

On 27.11.2018 8.39, Joan Moreau wrote: 

Thank you Aki 

here the requested data (below) 

Please not as well that we have numerous subfolders (>50) and pretty big mailbox sizes (>20G) 

Bug appears mostly in auth process and index-worker 

dovecot -n : 


# 2.4.devel (de42b54aa): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.6.devel (65909cfa)
# OS: Linux 4.19.4-arch1-1-ARCH x86_64 ext4
# Hostname: gjserver
base_dir = /run/dovecot
default_login_user = dovecot
default_vsz_limit = 16 G
disable_plaintext_auth = no
listen = *
log_path = /var/log/mail/dovecot.log
mail_gid = mail
mail_location = mdbox:/data/mail/%d/%n:ALT=/data/mail/archives/%d/%n
mail_plugins = fts fts_squat
mail_uid = mailusers
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character 
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy 
include variables body enotify environment mailbox date index ihave duplicate 
mime foreverypart extracttext
mdbox_rotate_size = 24 M 

(...) 


passdb {
args = /etc/dovecot/dovecot-sql.conf
driver = sql
}
(the rest default values)

On 2018-11-25 08:08, Aki Tuomi wrote: 

On 25 November 2018 at 06:29 Joan Moreau < j...@grosjo.net> wrote: 

Hi 

THis is the lines I have in my dmesg (see below) 

In dovecot log , I see: 

Nov 25 04:26:47 auth-worker: Error: double free or corruption (fasttop) 

What do to about it ? 

Using lastest 2.3.4 version 

Thank you 

 

[132932.169265] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00 
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 aa 36 f5 7a 7f 00 
00 <40> 24 3d f5 7a 7f 00 00 21 80 00 00 00 00 00 00 00 94 16 d6 ee 55 
[134031.969596] auth[27031]: segfault at 55e509612c30 ip 
55e509612c30 sp 7ffeb96dee48 error 15 
[134031.969603] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00 
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 3a d3 e4 ef 7f 00 
00 <40> b4 d9 e4 ef 7f 00 00 21 80 00 00 00 00 00 00 00 04 5f 09 e5 55 
[134081.497871] doveadm[28930]: segfault at 7ffe4a16efc8 ip 
7f393841013e sp 7ffe4a16efc0 error 6 in 
libdovecot.so.0.0.0[7f3938363000+e2000] 
[134081.497876] Code: 7d be e9 68 ff ff ff e8 10 4c f5 ff 41 57 41 89 cf 
41 56 49 89 f6 41 55 41 89 fd 31 ff 41 54 55 44 89 c5 53 89 d3 48 83 ec 
58 <4c> 89 4c 24 08 64 48 8b 04 25 28 00 00 00 48 89 44 24 48 31 c0 e8 
[134084.145731] doveadm[29186]: segfault at 7fff1cfdbff8 ip 
7f4376e32ffb sp 7fff1cfdc000 error 6 in 
libdovecot.so.0.0.0[7f4376d86000+e2000] 
[134084.145735] Code: ff 66 0f 1f 44 00 00 e9 9d dc f6 ff 0f 1f 00 48 83 
ec 08 48 83 3d 14 16 0b 00 00 0f 85 d2 76 f6 ff 31 f6 48 8d 3d 05 16 0b 
00  00 54 f5 ff 85 c0 0f 88 e4 76 f6 ff 48 83 c4 08 e9 69 dc f6 ff 
[135453.211242] indexer-worker[2539]: segfault at 7ffec3ba4ff8 ip 
7ffec43fdcff sp 7ffec3ba5000 error 6 
[135453.211245] Code: 95 4c 89 f7 48 89 75 d0 e8 5e fc ff ff 48 8b 75 d0 
e9 56 ff ff ff 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55 41 54 49 89 
f4 <53> 48 83 ec 08 48 85 ff 0f 84 b3 00 00 00 48 89 fb 4c 8d 35 69 c3 
[135453.730250] indexer-worker[9236]: segfault at 7fffed921ff8 ip 
7fb7c9c4f5b1 sp 7fffed922000 error 6 in 
libdovecot.so.0.0.0[7fb7c9ba2000+e2000] 
[135453.730256] Code: 2e 0f 1f 84 00 00 00 00 00 41 57 4d 89 cf 41 56 41 
89 fe 41 55 49 89 f5 41 54 41 89 d4 55 89 cd 53 48 83 ec 08 4c 8b 4c 24 
40  6a fb ff ff 85 c0 7e 4f 48 8b 05 7f f9 0a 00 be 38 00 00 00 48 
[135796.171575] auth[11121]: segfault at 555f8645cc30 ip 
555f8645cc30 sp 7ffcbb510868 error 15 
[135796.171586] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00 
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 ba ed 38 b7 7f 00 
00 <40> 34 f4 38 b7 7f 00 00 21 80 00 00 00 00 00 00 00 a4 43 86 5f 55 
[136710.562003] auth[17828]: segfault at 563443604c30 ip 
563443604c30 sp 7ffc1aa8b498 error 15 
[136710.562013] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00 
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 fa 48 da d5 7f 00 
00 <40> 74 4f da d5 7f 00 00 21 80 00 00 00 00 00 00 00 24 5e 43 34 56 
[138331.686718] auth[31046]: segfault at 55b27bc63c30 ip 
55b27bc63c30 sp 7ffd5d5b9298 error 15 
[138331.686721] Code: 00 00 00 00 00 00 00 00 00 00 80 00 00 00 00 00 00 
00 21 00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 20 9a 08 17 cd 7f 00 
00 <40> 14 0f 17 cd 7f 00 00 21 80 00 00 00 00 00 00 

Best way of merging mbox files

[Marc Roos]

When concatenating mbox files like described here 
https://xaizek.github.io/2013-03-30/merge-mbox-mailboxes/. You will end 
up with an 'unsorted' mbox file. Is this going to be a problem 
esspecially when they are large >2GB's and new emails will be written to 
it?
The email client nicely sorts the message from folder A "foldera 5 last" 
as last, but of course the mbox is not like this. 

Is there a better solution for merging files?

Having:

A folder A with messages:
FromSubject ReceivedSize
testfoldera 1   16:18   665 B   
testfoldera 2   16:18   665 B   
testfoldera 3   16:18   665 B   
testfoldera 4   16:18   665 B   
testfoldera 5 last  16:29   670 B   

A folder B with messages:
FromSubject ReceivedSize
testfolderb 1   16:23   665 B   
testfolderb 2   16:24   665 B   
testfolderb 3   16:24   665 B   
testfolderb 4   16:24   665 B   
testfolderb 5   16:24   665 B   

[@ mail] cat .foldera .folderb > .folderc

Getting a folder C with messages:
FromSubject ReceivedSize
testfoldera 1   16:18   665 B   
testfoldera 2   16:18   665 B   
testfoldera 3   16:18   665 B   
testfoldera 4   16:18   665 B   
Mail System Internal Data   DON'T DELETE THIS MESSAGE -- FOLDER 
INTERNAL DATA   16:19   454 B   
testfolderb 1   16:23   665 B   
testfolderb 2   16:24   665 B   
testfolderb 3   16:24   665 B   
testfolderb 4   16:24   665 B   
testfolderb 5   16:24   665 B   
testfoldera 5 last  16:29   670 B   

Re: Dovecot 2.3.4 crash

[azurit]

Citát azu...@pobox.sk:


Citát Timo Sirainen :


On 29 Nov 2018, at 15.46, azu...@pobox.sk wrote:


Hi,

is this a know problem? Newest Dovecot 2.3.4 package from  
repo.dovecot.org , Debian Stretch (fully upgraded).




Nov 29 14:25:11 server00 dovecot: lmtp(16854): Panic: file  
ostream-dot.c: line 208 (o_stream_dot_sendv): assertion failed:  
((size_t)ret == sent + added)


Is this proxying LMTP traffic?





Yes, it is. The main server is, currently, doing standard  
IMAP/POP3/LMTP for some clients and also proxying IMAP/POP3/LMTP for  
other clients (proxying to other servers). But this one error is  
related to one message which is stucked in postfix queue and cannot  
be delivered - the error appears everytime i do 'postfix flush'  
(error in postfix queue is 'lost connection with  
[private/dovecot-lmtp] while sending  
end of data -- message may be sent more than once').


There are no errors logged on proxy backend side.





Just to clarify this - that one message is to be proxied to other server.

Re: Dovecot 2.3.4 crash

[azurit]

Citát Timo Sirainen :


On 29 Nov 2018, at 15.46, azu...@pobox.sk wrote:


Hi,

is this a know problem? Newest Dovecot 2.3.4 package from  
repo.dovecot.org , Debian Stretch (fully upgraded).




Nov 29 14:25:11 server00 dovecot: lmtp(16854): Panic: file  
ostream-dot.c: line 208 (o_stream_dot_sendv): assertion failed:  
((size_t)ret == sent + added)


Is this proxying LMTP traffic?





Yes, it is. The main server is, currently, doing standard  
IMAP/POP3/LMTP for some clients and also proxying IMAP/POP3/LMTP for  
other clients (proxying to other servers). But this one error is  
related to one message which is stucked in postfix queue and cannot be  
delivered - the error appears everytime i do 'postfix flush' (error in  
postfix queue is 'lost connection with  
[private/dovecot-lmtp] while sending  
end of data -- message may be sent more than once').


There are no errors logged on proxy backend side.





dovecot/lmtp [local DATA](_start+0x2a) [0x565287c310ca]


Based on this I was thinking it wouldn't be. But this apparently  
happens only with sending SMTP/LMTP traffic with SSL. So do you  
think it's SSL-related with you also?  
https://www.dovecot.org/pipermail/dovecot/2018-November/113532.html  




We are, currently, not using SSL while proxying traffic.

Re: Dovecot 2.3.4 crash

[Timo Sirainen]

On 29 Nov 2018, at 15.46, azu...@pobox.sk wrote:
> 
> Hi,
> 
> is this a know problem? Newest Dovecot 2.3.4 package from repo.dovecot.org , 
> Debian Stretch (fully upgraded).
> 
> 
> 
> Nov 29 14:25:11 server00 dovecot: lmtp(16854): Panic: file ostream-dot.c: 
> line 208 (o_stream_dot_sendv): assertion failed: ((size_t)ret == sent + added)

Is this proxying LMTP traffic?

> dovecot/lmtp [local DATA](_start+0x2a) [0x565287c310ca]

Based on this I was thinking it wouldn't be. But this apparently happens only 
with sending SMTP/LMTP traffic with SSL. So do you think it's SSL-related with 
you also? https://www.dovecot.org/pipermail/dovecot/2018-November/113532.html 

Solr: URI Too Long

[azurit]

Hi,

i'm quite often getting this error:

Error: fts_solr: Lookup failed: 414 URI Too Long


Most web servers are limiting URI length to 8192 characters. Why not  
to use POST instead of GET when communicating with Solr? Solr is able  
to handle this ok, also see:

https://dzone.com/articles/solr-select-query-get-vs-post

azur

Dovecot 2.3.4 crash

[azurit]

Hi,

is this a know problem? Newest Dovecot 2.3.4 package from  
repo.dovecot.org , Debian Stretch (fully upgraded).




Nov 29 14:25:11 server00 dovecot: lmtp(16854): Panic: file  
ostream-dot.c: line 208 (o_stream_dot_sendv): assertion failed:  
((size_t)ret == sent + added)



Nov 29 14:25:11 server00 dovecot: lmtp(16854): Error: Raw backtrace:  
/usr/lib/dovecot/libdovecot.so.0(+0xd25e1) [0x7fa5f7b655e1] ->  
/usr/lib/dovecot/libdovecot.so.0(+0xd2681) [0x7
fa5f7b65681] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0)  
[0x7fa5f7acd093] -> /usr/lib/dovecot/libdovecot.so.0(+0xbb97b)  
[0x7fa5f7b4e97b] -> /usr/lib/dovecot/libdovecot.so.0(+0x
fa4ae) [0x7fa5f7b8d4ae] ->  
/usr/lib/dovecot/libdovecot.so.0(o_stream_sendv+0x2e) [0x7fa5f7b8da3e]  
-> /usr/lib/dovecot/libdovecot.so.0(io_stream_copy+0x52)  
[0x7fa5f7b8e512] -> /usr
/lib/dovecot/libdovecot.so.0(o_stream_send_istream+0x58)  
[0x7fa5f7b8e008] ->  
/usr/lib/dovecot/libdovecot.so.0(smtp_client_command_send_more+0x1b1)  
[0x7fa5f7ad7351] -> /usr/lib/dov
ecot/libdovecot.so.0(+0x4a096) [0x7fa5f7add096] ->  
/usr/lib/dovecot/libdovecot.so.0(+0xfd0c3) [0x7fa5f7b900c3] ->  
/usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x75) [0x7fa5f7b
7e5d5] ->  
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x109)  
[0x7fa5f7b7ffd9] ->  
/usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x56)  
[0x7fa5f7b7e6e6] -> /
usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fa5f7b7e8f8] ->  
/usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13)  
[0x7fa5f7af2d43] -> dovecot/lmtp [local DATA](main+
0x240) [0x565287c30f70] ->  
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)  
[0x7fa5f77142e1] -> dovecot/lmtp [local DATA](_start+0x2a)  
[0x565287c310ca]



Nov 29 14:25:11 server00 dovecot: lmtp(16854): Fatal: master:  
service(lmtp): child 16854 killed with signal 6 (core dumps disabled -  
https://dovecot.org/bugreport.html#coredumps)





azur

Re: Error logs with virtual folder after upgrading to 2.3.4

[Marcus Rueckert]

On Wed, 28 Nov 2018 19:45:00 +0200 (EET)
Aki Tuomi  wrote:

> If you are using systemd, you might want to create
> 
> /etc/systemd/system/service.dovecot.d/override.conf

/etc/systemd/systemd/dovecot.service.d/override.conf

easiest way to create it is:

systemctl edit dovecot.service
 
> with 
> 
> ProtectSystem=no

Dont you need?:)

[Service]
ProtectSystem=no


hth

darix

-- 
  openSUSE - SUSE Linux is my linux
  openSUSE is good for you
  www.opensuse.org

Re: determine why mail clients download mails 2nd time

[Poliman - Serwis]

pon., 26 lis 2018 o 09:25 Aki Tuomi  napisał(a):

>
> On 26.11.2018 10.01, Poliman - Serwis wrote:
>
> I have strange problem. In last Friday I had a problem with dovecot. It
> didn't see directories with mailboxes in /var/vmail. Somehow it crashed
> because probably two things (a lot of this type of lines in mail.err file):
> *Nov 23 07:05:02 s1 dovecot: lda(p...@poli.li ): Error: User
> initialization failed: Initializing mail storage from mail_location setting
> failed: mkdir(/var/vmail/poli.li/poli/Maildir
> ) failed: Permission denied (euid=5000(vmail)
> egid=5000(vmail) missing +w perm: /var/vmail, dir owned by 0:0 mode=0755)*
>
> /var/vmail is not owned by vmail:vmail, which I think it should be in your
> configuration.
>
Thank you for answer. Strange thing, because it is owned by vmail:vmail,
please take a look:
*drwxrwx--- 46 vmail vmail  4096 Aug 22 11:52 vmail*
I have /var/vmail directory binded to /var/vmail on secondary disk. Maybe
from some reason entry in /etc/fstab crashed? It happened after restart
server.

> and maybe
>
> *Nov 12 09:27:59 s1 dovecot: imap(ar...@firmastanaszek.pl
> ): Error: Corrupted index cache file
> /var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache
> : Broken MIME
> parts for mail UID 559 in mailbox INBOX: Cached MIME parts don't match
> message during parsing: Cached header size mismatch (parts=) Nov 12
> 09:27:59 s1 dovecot: imap(ar...@firmastanaszek.pl
> ): Error:
> unlink(/var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache
> ) failed: No
> such file or directory (in mail-cache.c:28)*
>
> and after this event probably all mailboxes download again all emails from
> 1.11 to 22.11. They were duplicated in mail clients.
>
> Unlikely that this causes all mailboxes to download all mails.
>
Honestly I am not sure that this was the reason of downloading emails
again. I am more sure that the first error "Permission denied" could do
this.
Do you know what could cause above error "Error: corrupted index cache
file" and how to fix it? It was happened suddenly without any changes in
dovecot configuration

> --
>
> *Pozdrawiam / Best Regards *
> *Piotr Bracha*
>
> Also which version of dovecot is this?
>
root@s1:/var# dovecot --version
2.2.22 (fe789d2)

> Aki
>


-- 

*Pozdrawiam / Best Regards*
*Piotr Bracha*