Re: [Dovecot-news] v2.2.29.rc1 released

[Aki Tuomi]

> On April 7, 2017 at 6:54 PM Aki Tuomi  wrote:
> 
> 
> 
> > On April 7, 2017 at 6:48 PM "Daniel J. Luke"  wrote:
> > 
> > 
> > On Apr 7, 2017, at 11:17 AM, Aki Tuomi  wrote:
> > >> On April 7, 2017 at 6:00 PM "Daniel J. Luke"  wrote:
> > >> On Apr 7, 2017, at 3:01 AM, Aki Tuomi  wrote:
> >  On April 7, 2017 at 9:38 AM Timo Sirainen  wrote:
> >  On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
> > > 
> > > On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
> > >> Planning to release v2.2.29 on Monday. Please find and report any 
> > >> bugs before that.
> > > 
> > > I'm seeing still seeing the assert that started showing up for me 
> > > with 2.2.28 
> > > (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
> > > 
> > > Below I generate it using doveadm with dovecot 2.2.29rc1 (output 
> > > slightly cleaned up so the backtrace is easier to read)
> > > 
> > > % sudo doveadm index -A \*
> > > doveadm(dluke): Panic: file mailbox-list.c: line 1159 
> > > (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, 
> > > path, strlen(root_dir)) == 0)
> >  
> >  This is with mbox? I thought this had been happening already a long 
> >  time.. Or if not mbox, what's your doveconf -n?
> > >> 
> > >> this is mbox & lucene
> > >> 
> > >>> for mbox and lucene, there is a workaround, which is to create 
> > >>> directory under the INDEXES directory called lucene-indexes.
> > >>> 
> > >>> This way the directory creation is not attempted, this is a known bug, 
> > >>> but just hasn't been fixed yet.
> > >> 
> > >> That directory already exists.
> > >> 
> > >> mail_location = mbox:~/Mail/:INBOX=~/.mbox
> > >> 
> > >> % ls -l ~/Mail/.imap/lucene-indexes/
> > >> total 429344
> > >> -rw---  1 dluke  dluke   209M Apr  6 11:12 _30ms.cfs
> > >> -rw---  1 dluke  dluke   1.0M Apr  7 08:10 _30rt.cfs
> > >> -rw---  1 dluke  dluke   7.2K Apr  7 08:10 _30ru.cfs
> > >> -rw---  1 dluke  dluke   6.3K Apr  7 08:11 _30rv.cfs
> > >> -rw---  1 dluke  dluke   7.6K Apr  7 08:11 _30rw.cfs
> > >> -rw---  1 dluke  dluke   3.8K Apr  7 10:57 _30rx.cfs
> > >> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30ry.cfs
> > >> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30rz.cfs
> > >> -rw---  1 dluke  dluke   2.3K Apr  7 10:58 dovecot-expunges.log
> > >> -rw---  1 dluke  dluke20B Apr  7 10:58 segments.gen
> > >> -rw---  1 dluke  dluke   244B Apr  7 10:58 segments_6214
> > >> 
> > >> When running 2.2.27 this doesn't happen.
> > > 
> > > Can you try moving it one folder up, so that it's not under .imap?
> > 
> > Sure, that fixes that, but I end up with lots of log messages like:
> > 
> > 2017-04-07 11:43:53.958590-0400  localhost log[5770]: (libdovecot.0.dylib) 
> > indexer-worker(dluke): Error: Syncing mailbox lucene-indexes/_0.cfs failed: 
> > Mailbox isn't a valid mbox file
> > 
> > (and the lucene-indexes "folder" appears in Mail.app, which is going to 
> > confuse some of my users)
> > 
> > -- 
> > Daniel J. Luke
> 
> Just wanted to factor in the bug.
> 
> So.. as workaround, you *could* move indexes to separate directory, using 
> INDEXES= parameter. And move the lucene-indexes to this folder. But thank 
> you, this helps a bit.
> 
> Aki

Hi Daniel,

https://github.com/dovecot/core/compare/c42ec3f%5E...2bc82f0.patch

we believe this fixes your issue.

Aki

Re: [Dovecot-news] v2.2.29.rc1 released

[Aki Tuomi]

> On April 10, 2017 at 8:31 PM "Daniel J. Luke"  wrote:
> 
> 
> On Apr 7, 2017, at 11:54 AM, Aki Tuomi  wrote:
> > So.. as workaround, you *could* move indexes to separate directory, using 
> > INDEXES= parameter. And move the lucene-indexes to this folder. But thank 
> > you, this helps a bit.
> 
> I'm happy to do anything else to test if you'd like.
> 
> I did go ahead and set INDEXES and can confirm that this does work around the 
> bug.
> 
> -- 
> Daniel J. Luke

This will not make into 2.2.29 as it's being released very soon, but we'll try 
to fix this in 2.2.30 then. We have opened issue in our tracker.

Aki

Re: [Dovecot-news] v2.2.29.rc1 released

[Daniel J. Luke]

On Apr 7, 2017, at 11:54 AM, Aki Tuomi  wrote:
> So.. as workaround, you *could* move indexes to separate directory, using 
> INDEXES= parameter. And move the lucene-indexes to this folder. But thank 
> you, this helps a bit.

I'm happy to do anything else to test if you'd like.

I did go ahead and set INDEXES and can confirm that this does work around the 
bug.

-- 
Daniel J. Luke

Re: [Dovecot-news] v2.2.29.rc1 released

[Aki Tuomi]

> On April 7, 2017 at 6:48 PM "Daniel J. Luke"  wrote:
> 
> 
> On Apr 7, 2017, at 11:17 AM, Aki Tuomi  wrote:
> >> On April 7, 2017 at 6:00 PM "Daniel J. Luke"  wrote:
> >> On Apr 7, 2017, at 3:01 AM, Aki Tuomi  wrote:
>  On April 7, 2017 at 9:38 AM Timo Sirainen  wrote:
>  On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
> > 
> > On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
> >> Planning to release v2.2.29 on Monday. Please find and report any bugs 
> >> before that.
> > 
> > I'm seeing still seeing the assert that started showing up for me with 
> > 2.2.28 
> > (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
> > 
> > Below I generate it using doveadm with dovecot 2.2.29rc1 (output 
> > slightly cleaned up so the backtrace is easier to read)
> > 
> > % sudo doveadm index -A \*
> > doveadm(dluke): Panic: file mailbox-list.c: line 1159 
> > (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, 
> > path, strlen(root_dir)) == 0)
>  
>  This is with mbox? I thought this had been happening already a long 
>  time.. Or if not mbox, what's your doveconf -n?
> >> 
> >> this is mbox & lucene
> >> 
> >>> for mbox and lucene, there is a workaround, which is to create directory 
> >>> under the INDEXES directory called lucene-indexes.
> >>> 
> >>> This way the directory creation is not attempted, this is a known bug, 
> >>> but just hasn't been fixed yet.
> >> 
> >> That directory already exists.
> >> 
> >> mail_location = mbox:~/Mail/:INBOX=~/.mbox
> >> 
> >> % ls -l ~/Mail/.imap/lucene-indexes/
> >> total 429344
> >> -rw---  1 dluke  dluke   209M Apr  6 11:12 _30ms.cfs
> >> -rw---  1 dluke  dluke   1.0M Apr  7 08:10 _30rt.cfs
> >> -rw---  1 dluke  dluke   7.2K Apr  7 08:10 _30ru.cfs
> >> -rw---  1 dluke  dluke   6.3K Apr  7 08:11 _30rv.cfs
> >> -rw---  1 dluke  dluke   7.6K Apr  7 08:11 _30rw.cfs
> >> -rw---  1 dluke  dluke   3.8K Apr  7 10:57 _30rx.cfs
> >> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30ry.cfs
> >> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30rz.cfs
> >> -rw---  1 dluke  dluke   2.3K Apr  7 10:58 dovecot-expunges.log
> >> -rw---  1 dluke  dluke20B Apr  7 10:58 segments.gen
> >> -rw---  1 dluke  dluke   244B Apr  7 10:58 segments_6214
> >> 
> >> When running 2.2.27 this doesn't happen.
> > 
> > Can you try moving it one folder up, so that it's not under .imap?
> 
> Sure, that fixes that, but I end up with lots of log messages like:
> 
> 2017-04-07 11:43:53.958590-0400  localhost log[5770]: (libdovecot.0.dylib) 
> indexer-worker(dluke): Error: Syncing mailbox lucene-indexes/_0.cfs failed: 
> Mailbox isn't a valid mbox file
> 
> (and the lucene-indexes "folder" appears in Mail.app, which is going to 
> confuse some of my users)
> 
> -- 
> Daniel J. Luke

Just wanted to factor in the bug.

So.. as workaround, you *could* move indexes to separate directory, using 
INDEXES= parameter. And move the lucene-indexes to this folder. But thank you, 
this helps a bit.

Aki

Re: [Dovecot-news] v2.2.29.rc1 released

[Daniel J. Luke]

On Apr 7, 2017, at 11:17 AM, Aki Tuomi  wrote:
>> On April 7, 2017 at 6:00 PM "Daniel J. Luke"  wrote:
>> On Apr 7, 2017, at 3:01 AM, Aki Tuomi  wrote:
 On April 7, 2017 at 9:38 AM Timo Sirainen  wrote:
 On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
> 
> On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
>> Planning to release v2.2.29 on Monday. Please find and report any bugs 
>> before that.
> 
> I'm seeing still seeing the assert that started showing up for me with 
> 2.2.28 
> (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
> 
> Below I generate it using doveadm with dovecot 2.2.29rc1 (output slightly 
> cleaned up so the backtrace is easier to read)
> 
> % sudo doveadm index -A \*
> doveadm(dluke): Panic: file mailbox-list.c: line 1159 
> (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, 
> strlen(root_dir)) == 0)
 
 This is with mbox? I thought this had been happening already a long time.. 
 Or if not mbox, what's your doveconf -n?
>> 
>> this is mbox & lucene
>> 
>>> for mbox and lucene, there is a workaround, which is to create directory 
>>> under the INDEXES directory called lucene-indexes.
>>> 
>>> This way the directory creation is not attempted, this is a known bug, but 
>>> just hasn't been fixed yet.
>> 
>> That directory already exists.
>> 
>> mail_location = mbox:~/Mail/:INBOX=~/.mbox
>> 
>> % ls -l ~/Mail/.imap/lucene-indexes/
>> total 429344
>> -rw---  1 dluke  dluke   209M Apr  6 11:12 _30ms.cfs
>> -rw---  1 dluke  dluke   1.0M Apr  7 08:10 _30rt.cfs
>> -rw---  1 dluke  dluke   7.2K Apr  7 08:10 _30ru.cfs
>> -rw---  1 dluke  dluke   6.3K Apr  7 08:11 _30rv.cfs
>> -rw---  1 dluke  dluke   7.6K Apr  7 08:11 _30rw.cfs
>> -rw---  1 dluke  dluke   3.8K Apr  7 10:57 _30rx.cfs
>> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30ry.cfs
>> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30rz.cfs
>> -rw---  1 dluke  dluke   2.3K Apr  7 10:58 dovecot-expunges.log
>> -rw---  1 dluke  dluke20B Apr  7 10:58 segments.gen
>> -rw---  1 dluke  dluke   244B Apr  7 10:58 segments_6214
>> 
>> When running 2.2.27 this doesn't happen.
> 
> Can you try moving it one folder up, so that it's not under .imap?

Sure, that fixes that, but I end up with lots of log messages like:

2017-04-07 11:43:53.958590-0400  localhost log[5770]: (libdovecot.0.dylib) 
indexer-worker(dluke): Error: Syncing mailbox lucene-indexes/_0.cfs failed: 
Mailbox isn't a valid mbox file

(and the lucene-indexes "folder" appears in Mail.app, which is going to confuse 
some of my users)

-- 
Daniel J. Luke

Re: [Dovecot-news] v2.2.29.rc1 released

[Aki Tuomi]

> On April 7, 2017 at 6:00 PM "Daniel J. Luke"  wrote:
> 
> 
> On Apr 7, 2017, at 3:01 AM, Aki Tuomi  wrote:
> >> On April 7, 2017 at 9:38 AM Timo Sirainen  wrote:
> >> On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
> >>> 
> >>> On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
>  Planning to release v2.2.29 on Monday. Please find and report any bugs 
>  before that.
> >>> 
> >>> I'm seeing still seeing the assert that started showing up for me with 
> >>> 2.2.28 
> >>> (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
> >>> 
> >>> Below I generate it using doveadm with dovecot 2.2.29rc1 (output slightly 
> >>> cleaned up so the backtrace is easier to read)
> >>> 
> >>> % sudo doveadm index -A \*
> >>> doveadm(dluke): Panic: file mailbox-list.c: line 1159 
> >>> (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, 
> >>> strlen(root_dir)) == 0)
> >> 
> >> This is with mbox? I thought this had been happening already a long time.. 
> >> Or if not mbox, what's your doveconf -n?
> 
> this is mbox & lucene
> 
> > for mbox and lucene, there is a workaround, which is to create directory 
> > under the INDEXES directory called lucene-indexes.
> > 
> > This way the directory creation is not attempted, this is a known bug, but 
> > just hasn't been fixed yet.
> 
> That directory already exists.
> 
> mail_location = mbox:~/Mail/:INBOX=~/.mbox
> 
> % ls -l ~/Mail/.imap/lucene-indexes/
> total 429344
> -rw---  1 dluke  dluke   209M Apr  6 11:12 _30ms.cfs
> -rw---  1 dluke  dluke   1.0M Apr  7 08:10 _30rt.cfs
> -rw---  1 dluke  dluke   7.2K Apr  7 08:10 _30ru.cfs
> -rw---  1 dluke  dluke   6.3K Apr  7 08:11 _30rv.cfs
> -rw---  1 dluke  dluke   7.6K Apr  7 08:11 _30rw.cfs
> -rw---  1 dluke  dluke   3.8K Apr  7 10:57 _30rx.cfs
> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30ry.cfs
> -rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30rz.cfs
> -rw---  1 dluke  dluke   2.3K Apr  7 10:58 dovecot-expunges.log
> -rw---  1 dluke  dluke20B Apr  7 10:58 segments.gen
> -rw---  1 dluke  dluke   244B Apr  7 10:58 segments_6214
> 
> When running 2.2.27 this doesn't happen.
> -- 
> Daniel J. Luke

Can you try moving it one folder up, so that it's not under .imap?

Aki

Re: [Dovecot-news] v2.2.29.rc1 released

[Daniel J. Luke]

On Apr 7, 2017, at 3:01 AM, Aki Tuomi  wrote:
>> On April 7, 2017 at 9:38 AM Timo Sirainen  wrote:
>> On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
>>> 
>>> On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
 Planning to release v2.2.29 on Monday. Please find and report any bugs 
 before that.
>>> 
>>> I'm seeing still seeing the assert that started showing up for me with 
>>> 2.2.28 (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
>>> 
>>> Below I generate it using doveadm with dovecot 2.2.29rc1 (output slightly 
>>> cleaned up so the backtrace is easier to read)
>>> 
>>> % sudo doveadm index -A \*
>>> doveadm(dluke): Panic: file mailbox-list.c: line 1159 
>>> (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, 
>>> strlen(root_dir)) == 0)
>> 
>> This is with mbox? I thought this had been happening already a long time.. 
>> Or if not mbox, what's your doveconf -n?

this is mbox & lucene

> for mbox and lucene, there is a workaround, which is to create directory 
> under the INDEXES directory called lucene-indexes.
> 
> This way the directory creation is not attempted, this is a known bug, but 
> just hasn't been fixed yet.

That directory already exists.

mail_location = mbox:~/Mail/:INBOX=~/.mbox

% ls -l ~/Mail/.imap/lucene-indexes/
total 429344
-rw---  1 dluke  dluke   209M Apr  6 11:12 _30ms.cfs
-rw---  1 dluke  dluke   1.0M Apr  7 08:10 _30rt.cfs
-rw---  1 dluke  dluke   7.2K Apr  7 08:10 _30ru.cfs
-rw---  1 dluke  dluke   6.3K Apr  7 08:11 _30rv.cfs
-rw---  1 dluke  dluke   7.6K Apr  7 08:11 _30rw.cfs
-rw---  1 dluke  dluke   3.8K Apr  7 10:57 _30rx.cfs
-rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30ry.cfs
-rw---  1 dluke  dluke   3.8K Apr  7 10:58 _30rz.cfs
-rw---  1 dluke  dluke   2.3K Apr  7 10:58 dovecot-expunges.log
-rw---  1 dluke  dluke20B Apr  7 10:58 segments.gen
-rw---  1 dluke  dluke   244B Apr  7 10:58 segments_6214

When running 2.2.27 this doesn't happen.
-- 
Daniel J. Luke

Re: [Dovecot-news] v2.2.29.rc1 released

[Aki Tuomi]

> On April 7, 2017 at 9:38 AM Timo Sirainen  wrote:
> 
> 
> On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
> > 
> > On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
> >> Planning to release v2.2.29 on Monday. Please find and report any bugs 
> >> before that.
> > 
> > I'm seeing still seeing the assert that started showing up for me with 
> > 2.2.28 (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
> > 
> > Below I generate it using doveadm with dovecot 2.2.29rc1 (output slightly 
> > cleaned up so the backtrace is easier to read)
> > 
> > % sudo doveadm index -A \*
> > doveadm(dluke): Panic: file mailbox-list.c: line 1159 
> > (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, 
> > strlen(root_dir)) == 0)
> 
> This is with mbox? I thought this had been happening already a long time.. Or 
> if not mbox, what's your doveconf -n?

for mbox and lucene, there is a workaround, which is to create directory under 
the INDEXES directory called lucene-indexes.

This way the directory creation is not attempted, this is a known bug, but just 
hasn't been fixed yet.

Aki

Re: [Dovecot-news] v2.2.29.rc1 released

[Timo Sirainen]

On 7 Apr 2017, at 2.25, Daniel J. Luke  wrote:
> 
> On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
>> Planning to release v2.2.29 on Monday. Please find and report any bugs 
>> before that.
> 
> I'm seeing still seeing the assert that started showing up for me with 2.2.28 
> (https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)
> 
> Below I generate it using doveadm with dovecot 2.2.29rc1 (output slightly 
> cleaned up so the backtrace is easier to read)
> 
> % sudo doveadm index -A \*
> doveadm(dluke): Panic: file mailbox-list.c: line 1159 
> (mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, 
> strlen(root_dir)) == 0)

This is with mbox? I thought this had been happening already a long time.. Or 
if not mbox, what's your doveconf -n?

Re: [Dovecot-news] v2.2.29.rc1 released

[Daniel J. Luke]

On Apr 6, 2017, at 1:33 PM, Timo Sirainen  wrote:
> Planning to release v2.2.29 on Monday. Please find and report any bugs before 
> that.

I'm seeing still seeing the assert that started showing up for me with 2.2.28 
(https://www.dovecot.org/pipermail/dovecot/2017-February/107250.html)

Below I generate it using doveadm with dovecot 2.2.29rc1 (output slightly 
cleaned up so the backtrace is easier to read)

% sudo doveadm index -A \*
doveadm(dluke): Panic: file mailbox-list.c: line 1159 
(mailbox_list_try_mkdir_root): assertion failed: (strncmp(root_dir, path, 
strlen(root_dir)) == 0)
doveadm(dluke): Error: Raw backtrace: 
2   libdovecot.0.dylib  0x00010cfed284 default_fatal_finish 
+ 36 
-> 3   libdovecot.0.dylib  0x00010cfed05d 
default_fatal_handler + 61 
-> 4   libdovecot.0.dylib  0x00010cfed5a9 i_panic + 169 
-> 5   libdovecot-storage.0.dylib  0x00010ce49677 
mailbox_list_try_mkdir_root + 1207 
-> 6   libdovecot-storage.0.dylib  0x00010ce49759 
mailbox_list_mkdir_root + 25 
-> 7   lib21_fts_lucene_plugin.so  0x00010ee6ab39 
fts_backend_lucene_update_set_build_key + 73 
-> 8   lib20_fts_plugin.so 0x00010d2017cf 
fts_backend_update_set_build_key + 79 
-> 9   lib20_fts_plugin.so 0x00010d2027a7 fts_build_mail + 
1479 
-> 10  lib20_fts_plugin.so 0x00010d20795a fts_mail_precache 
+ 794 
-> 11  libdovecot-storage.0.dylib  0x00010ce2ff59 mail_precache + 
25 
-> 12  doveadm 0x00010cd319df cmd_index_run + 
1007 
-> 13  doveadm 0x00010cd2c0e5 
doveadm_mail_next_user + 421 
-> 14  doveadm 0x00010cd2d6f5 
doveadm_mail_cmd_exec + 645 
-> 15  doveadm 0x00010cd2d32d 
doveadm_cmd_ver2_to_mail_cmd_wrapper + 1629 
-> 16  doveadm 0x00010cd3c696 
doveadm_cmd_run_ver2 + 1046 
-> 17  doveadm 0x00010cd3c257 
doveadm_cmd_try_run_ver2 + 55 
-> 18  doveadm 0x00010cd3ef79 main + 1001 
-> 19  libdyld.dylib   0x7fff93655235 start + 1
Abort

-- 
Daniel J. Luke