Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

On Mon, Jan 16, 2017 at 06:54:33PM +0100, chrysn wrote:
> Oswald, do you think you could apply that patch to the 1.2 branch?
> 
that's entirely out of the question.
i need to finally make an 1.3 release. i just have a bunch of things
cooking which i want to get done first ...

Bug#782054: mbsync: New version cannot open Maildir boxes

[chrysn]

I just checked whether the patch of
0f24ca31b5fafe5228d0e99f460f1c823121b296 could be trivially applied, but
it does not even apply with fuzz, and has "both modified" sections in
several places.

Oswald, do you think you could apply that patch to the 1.2 branch?

Thanks
chrysn

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom


signature.asc
Description: PGP signature

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

On Tue, Jul 28, 2015 at 09:21:57AM +0100, Ian Campbell wrote:
> It would be super useful if mbsync -l could produce the actual literal path
> of the Maildir to which a given folder was being sync'd. With sufficient
> debug/verbosity I can infer from the "reading sync state" message, but
> having a concise way to see how things will end up would greatly simplify
> tweaking the config.
> 
-l isn't a debugging option, so that's out of scope.
i could make the output of -Dm more direct in that regard.
anyway, this belongs to the isync list or feature tracker.

> So, please could you recommend a set of options which produce a
> Maildir tree which is compatible with Evolution's Maildir++?
> 
git master already has support for real maildir++ (as i understand it,
anyway) and entirely dotless subfolders, but evolution's interpretation
of the format definitely seems a bit funny, so that remains
incompatible.

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Hi!

On Sat, 2015-09-05 at 14:18:21 +0100, Ian Campbell wrote:
> On Sat, 2015-09-05 at 13:53 +0200, Guillem Jover wrote:
> > On Fri, 2015-09-04 at 10:38:09 +0100, Ian Campbell wrote:
> > > The change to Master being the primary thing. This results in the:
> > > 
> > > [Account]
> > > |-INBOX
> > > |- PATCHES
> > > |  |- WIP
> > > |  `- FOO
> > > `- Cronspam
> > > 
> > > maildir hierarchy which I noted in Message #75 I'd be happy to switch to 
> > > if
> > > necessary.
> > 
> > Right, and thanks for the update, unfortunately that's not a conformant
> > Maildir++ layout,
> 
> FWIW the disk layout which Evolution displays as above is:
> 
> Maildir/{cur,tmp,new} # AKA "INBOX"
> Maildir/.PATCHES/{cur,tmp,new}
> Maildir/.PATCHES.WIP/{cur,tmp,new}
> Maildir/.PATCHES.FOO/{cur,tmp,new}
> Maildir/.Cronspam/{cur,tmp,new}

Ah, right, sorry yes that to me seems to be proper Maildir++,

> Just mentioning for clarity in case you thought it was:
> 
> Maildir/{cur,tmp,new} # AKA "INBOX"
> Maildir/PATCHES/{cur,tmp,new}

Exactly. :)

> because I thought the first (with the dots) _was_ compliant Maildir++.

Yes, I think so.

> > > NB: I'm still tinkering with things, so I'm not 100% sure this is correct,
> > > but on first glance it looks so and I'm also not sure if it is applicable
> > > to the issue in your context but I hope it turns out to be helpful!
> > 
> > I've not had time to look into this again, but once I do, I guess I'll
> > try to prepare a patch, post it here, and if it gets rejected run with
> > a local fork. Either that or try to discover how to properly configure
> > the program to do what I want, in case it is really possible. :)

Ok, so I tried your recipe, and didn't fully work for me, and ended
up having a look, because had already spent enough time again messing
with the config file that it was getting a bit frustrating.

Here's a couple of patches I prepared that seem to do what I'd expect,
and I've been running with those from git master [C], but only with
«Sync Pull» for now. The first adds quite some verbosity to be able to
see what's actually going on, the second "fixes" the Maildir++ support
for me. Once I've tested it more, I'll probably be preparing local
packages with the patches.

  [C] commit 167964933f42d32f1cc2a119453e71c5048e86e2

Thanks,
Guillem
From c4b51b5c8df236d272987820e801db20bd78d398 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Sun, 6 Sep 2015 21:47:57 +0200
Subject: [PATCH 1/2] XXX: Add more verbosity

---
 src/drv_maildir.c |  3 +++
 src/main.c| 26 +-
 src/sync.c|  2 +-
 3 files changed, 29 insertions(+), 2 deletions(-)

diff --git a/src/drv_maildir.c b/src/drv_maildir.c
index a91b7cc..74fdd81 100644
--- a/src/drv_maildir.c
+++ b/src/drv_maildir.c
@@ -125,6 +125,8 @@ maildir_join_path( maildir_store_conf_t *conf, const char *prefix, const char *b
 	int pl, bl, n, sub = 0;
 	char c;
 
+	info(" maildir join source = '%s'\n", box);
+
 	pl = strlen( prefix );
 	for (bl = 0, n = 0; (c = box[bl]); bl++)
 		if (c == '/') {
@@ -165,6 +167,7 @@ maildir_join_path( maildir_store_conf_t *conf, const char *prefix, const char *b
 		}
 	}
 	*p = 0;
+	info(" maildir join dest = '%s'\n", out);
 	return out;
 }
 
diff --git a/src/main.c b/src/main.c
index c8c3bf2..03c2bad 100644
--- a/src/main.c
+++ b/src/main.c
@@ -215,6 +215,13 @@ cmp_box_names( const void *a, const void *b )
 	return strcmp( as, bs );
 }
 
+static void
+print_boxes_list(string_list_t *boxes, int box)
+{
+	for (; boxes; boxes = boxes->next)
+		info("  box[%d] name = %s\n", box, boxes->string);
+}
+
 static char **
 filter_boxes( string_list_t *boxes, const char *prefix, string_list_t *patterns )
 {
@@ -225,8 +232,10 @@ filter_boxes( string_list_t *boxes, const char *prefix, string_list_t *patterns
 
 	pfxl = prefix ? strlen( prefix ) : 0;
 	for (; boxes; boxes = boxes->next) {
-		if (!starts_with( boxes->string, -1, prefix, pfxl ))
+		if (!starts_with( boxes->string, -1, prefix, pfxl )) {
+			info("  skipping prefix: %s\n", boxes->string);
 			continue;
+		}
 		fnot = 1;
 		for (cpat = patterns; cpat; cpat = cpat->next) {
 			ps = cpat->string;
@@ -240,6 +249,8 @@ filter_boxes( string_list_t *boxes, const char *prefix, string_list_t *patterns
 break;
 			}
 		}
+		if (fnot)
+			info("  ignoring pattern: %s\n", boxes->string);
 		if (!fnot) {
 			if (num + 1 >= rnum)
 boxarr = nfrealloc( boxarr, (rnum = (rnum + 10) * 2) * sizeof(*boxarr) );
@@ -817,6 +828,15 @@ sync_chans( main_vars_t *mvars, int ent )
 			mvars->chanptr->boxlist = 2;
 			boxes[M] = filter_boxes( mvars->ctx[M]->boxes, mvars->chan->boxes[M], mvars->chan->patterns );
 			boxes[S] = filter_boxes( mvars->ctx[S]->boxes, mvars->chan->boxes[S], mvars->chan->patterns );
+
+			print_boxes_list(mvars->ctx[M]->boxes, M);
+			print_boxes_list(mvars->ctx[S]->boxes, S);
+
+			for (mb = 0; boxes[M][mb]; mb++)
+info(" box: M=%s\n", boxes[M][mb]);
+			for (sb = 0; boxes[S][sb]; sb++)
+info(

Bug#782054: mbsync: New version cannot open Maildir boxes

[Ian Campbell]

On Sat, 2015-09-05 at 13:53 +0200, Guillem Jover wrote:
> Hi Ian!
> 
> On Fri, 2015-09-04 at 10:38:09 +0100, Ian Campbell wrote:
> > On Tue, 2015-07-28 at 09:21 +0100, Ian Campbell wrote:
> > > I'm suffering from a similar issue to Guillem,
> 
> > I had a realisation which might be useful to you as well: The 
> trailing "/"
> > on a store:mailbox is (or can be) significant, e.g. the following 
> diff
> > against my previous redacted mbsyncrc appears (on first glance) to 
> work and
> > to do what I desire:
> 
> > The change to Master being the primary thing. This results in the:
> > 
> > [Account]
> > |-INBOX
> > |- PATCHES
> > |  |- WIP
> > |  `- FOO
> > `- Cronspam
> > 
> > maildir hierarchy which I noted in Message #75 I'd be happy to 
> switch to if
> > necessary.
> 
> Right, and thanks for the update, unfortunately that's not a conformant
> Maildir++ layout,

FWIW the disk layout which Evolution displays as above is:

Maildir/{cur,tmp,new} # AKA "INBOX"
Maildir/.PATCHES/{cur,tmp,new}
Maildir/.PATCHES.WIP/{cur,tmp,new}
Maildir/.PATCHES.FOO/{cur,tmp,new}
Maildir/.Cronspam/{cur,tmp,new}

Just mentioning for clarity in case you thought it was:

Maildir/{cur,tmp,new} # AKA "INBOX"
Maildir/PATCHES/{cur,tmp,new}

because I thought the first (with the dots) _was_ compliant Maildir++.

>  which is what I'd like to have on my client system to
> match the filesystem layout of my server system.

I can see why you might want that (orthogonally to whether the single .
prefix is valid Maildir++ or not).

> > NB: I'm still tinkering with things, so I'm not 100% sure this is correct,
> > but on first glance it looks so and I'm also not sure if it is applicable
> > to the issue in your context but I hope it turns out to be helpful!
> 
> I've not had time to look into this again, but once I do, I guess I'll
> try to prepare a patch, post it here, and if it gets rejected run with
> a local fork. Either that or try to discover how to properly configure
> the program to do what I want, in case it is really possible. :)

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Hi Ian!

On Fri, 2015-09-04 at 10:38:09 +0100, Ian Campbell wrote:
> On Tue, 2015-07-28 at 09:21 +0100, Ian Campbell wrote:
> > I'm suffering from a similar issue to Guillem,

> I had a realisation which might be useful to you as well: The trailing "/"
> on a store:mailbox is (or can be) significant, e.g. the following diff
> against my previous redacted mbsyncrc appears (on first glance) to work and
> to do what I desire:

> The change to Master being the primary thing. This results in the:
> 
> [Account]
> |-INBOX
> |- PATCHES
> |  |- WIP
> |  `- FOO
> `- Cronspam
> 
> maildir hierarchy which I noted in Message #75 I'd be happy to switch to if
> necessary.

Right, and thanks for the update, unfortunately that's not a conformant
Maildir++ layout, which is what I'd like to have on my client system to
match the filesystem layout of my server system.

> NB: I'm still tinkering with things, so I'm not 100% sure this is correct,
> but on first glance it looks so and I'm also not sure if it is applicable
> to the issue in your context but I hope it turns out to be helpful!

I've not had time to look into this again, but once I do, I guess I'll
try to prepare a patch, post it here, and if it gets rejected run with
a local fork. Either that or try to discover how to properly configure
the program to do what I want, in case it is really possible. :)

Thanks,
Guillem

Bug#782054: mbsync: New version cannot open Maildir boxes

[Ian Campbell]

On Tue, 2015-07-28 at 09:21 +0100, Ian Campbell wrote:
> I'm suffering from a similar issue to Guillem,

Guillem,

I had a realisation which might be useful to you as well: The trailing "/"
on a store:mailbox is (or can be) significant, e.g. the following diff
against my previous redacted mbsyncrc appears (on first glance) to work and
to do what I desire:

@@ -8,7 +8,7 @@
 Account XXX
 
 MaildirStore XXX-local
-Path ~/Maildir/..
+Path ~/Maildir/
 Inbox ~/Maildir/
 #AltMap yes
 Flatten .
@@ -22,9 +22,9 @@
 SyncState *
 
 Channel XXX-folders
-Master :XXX-remote:INBOX/
+Master :XXX-remote:INBOX
 Slave :XXX-local:
-Patterns * !INBOX
+Patterns * !
 Create Slave
 Expunge Both
 SyncState *

The change to Master being the primary thing. This results in the:

[Account]
|-INBOX
|- PATCHES
|  |- WIP
|  `- FOO
`- Cronspam

maildir hierarchy which I noted in Message #75 I'd be happy to switch to if
necessary.

NB: I'm still tinkering with things, so I'm not 100% sure this is correct,
but on first glance it looks so and I'm also not sure if it is applicable
to the issue in your context but I hope it turns out to be helpful!

Cheers,
Ian.

Bug#782054: mbsync: New version cannot open Maildir boxes

[Ian Campbell]

Hi Oswald,

I'm suffering from a similar issue to Guillem, which I'll mention below,
but first a wishlist feature request (I hope you don't mind):

It would be super useful if mbsync -l could produce the actual literal path
of the Maildir to which a given folder was being sync'd. With sufficient
debug/verbosity I can infer from the "reading sync state" message, but
having a concise way to see how things will end up would greatly simplify
tweaking the config.

Anyway, on to the issue I'm having:

On Wed, 8 Apr 2015 09:53:40 +0200 Oswald Buddenhagen 
 wrote:
> i suspect this is due to the trailing dot in the Path specification,
> i.e., your attempt to create a namespace which uniformly uses leading
> dots, not only for subfolders.

I currently have a Maildir with:

$ cat ~/Maildir/..maildir++
maildir++ 1
$

and a layout like:

~/Maildir/{new,tmp,cur} # AKA INBOX
~/Maildir/..PATCHES.WIP/{new,tmp,cur} # From now on {...}
~/Maildir/..PATCHES.FOO/{...}
~/Maildir/..Cronspam/{...}

Which results (in the Evolution MUA GUI) in a tree like:

[Account]
`-INBOX
  |- PATCHES
  |  |- WIP
  |  `- FOO
  `- Cronspam

As an alternative I could switch to:

~/Maildir/{new,tmp,cur} # AKA INBOX
~/Maildir/.PATCHES.WIP/{new,tmp,cur} # From now on {...}
~/Maildir/.PATCHES.FOO/{...}
~/Maildir/.Cronspam/{...}

Which would result in:

[Account]
|-INBOX
|- PATCHES
|  |- WIP
|  `- FOO
`- Cronspam

I'd be fine with switching to this if that is better supported somehow.

However this:

~/Maildir/{new,tmp,cur} # AKA INBOX
~/Maildir/PATCHES.WIP/{new,tmp,cur} # From now on {...}
~/Maildir/PATCHES.FOO/{...}
~/Maildir/Cronspam/{...}

Results in:
[Account]
`-INBOX

i.e. subfolders without the leading . are ignored. It was my understanding
(confirmed by the links which Guillem has posted) that the leading . was a
requirement of Maildir++.

I'm currently using 1.1.2-1 but due to this happening:

Maildir error: UID 580 is beyond highest assigned UID 153.

I thought I should upgrade to 1.2.0-1 (I know how to fix this manually, and
have done, but I'd rather upgrade). On upgrade I trip over the "Error:
channel server: slave [...] cannot be opened." issue which Guillem
describes.

So, please could you recommend a set of options which produce a Maildir
tree which is compatible with Evolution's Maildir++?

My (redacted) .mbsyncrc is attached.

FWIW the other end in this case is a MS Exchange IMAP, I don't know which
version.

Many thanks,
Ian.IMAPAccount XXX
Host x
User x
Pass x
CertificateFile /etc/ssl/certs/ca-certificates.crt

IMAPStore XXX-remote
Account XXX

MaildirStore XXX-local
Path ~/Maildir/..
Inbox ~/Maildir/
#AltMap yes
Flatten .

Channel XXX-inbox
Master :XXX-remote:
Slave :XXX-local:
Patterns INBOX
Create Slave # Without this no sync
Expunge Both
SyncState *

Channel XXX-folders
Master :XXX-remote:INBOX/
Slave :XXX-local:
Patterns * !INBOX
Create Slave
Expunge Both
SyncState *

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

On Sun, May 10, 2015 at 04:04:59AM +0200, Guillem Jover wrote:
> And while I'm not trying to be obtuse here, the way I read the specs I
> don't see anything wrong with my reasoning.
> 
well, maybe you should re-read the mbsync manual then, to get a factual
basis for your reasoning. you are *still* not getting what Path is.
you're looking for Inbox. you don't even need a Path.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Hi!

On Tue, 2015-05-05 at 10:16:11 +0200, Oswald Buddenhagen wrote:
> On Mon, May 04, 2015 at 10:40:19PM +0200, Guillem Jover wrote:
> > Either that or I'm very confused…

> i'd bet on the latter. ;)

I'm not sure if we are talking/thinking past each other, though. And
while I'm not trying to be obtuse here, the way I read the specs I
don't see anything wrong with my reasoning. So if it really is that
I'm confused, I'd appreaciate to understand why. :)

> Path is not the top-level folder - it is a "namespace" that contains
> top-level folders.

Ok, but a "namespace" is an IMAP (or an mbsync) concept not a Maildir++
concept. And from my reading of the spec a ~/Maildir with (sub)folders
not starting with . is not a valid Maildir++.

> this is also what you'd find on an actual imap server.

Perhaps through the IMAP protocol point of view, but certainly not from
its on-disk layout. As I pointed out in my previous reply, the dovecot [D]
and the courier [M] docs are pretty explicit on how Maildir++ is layed out
on-disk. The dovecot docs even include examples for valid Maildir++ boxes,
which match exactly the on-disk layout on my imap server. And that was the
reason I switched my client on-disk layouts to match.

[D] 
[M] 

Here's the relevant excerpt from the dovecot doc:

,---
Directory Structure

Dovecot uses Maildir++ directory layout for organizing mailbox directories.
This means that all the folders are directly inside ~/Maildir directory:

~/Maildir/new, ~/Maildir/cur and ~/Maildir/tmp directories contain the
  messages for INBOX. The tmp directory is used during delivery, new
  messages arrive in new and read shall be moved to cur by the clients.
~/Maildir/.folder/ is a mailbox folder
~/Maildir/.folder.subfolder/ is a subfolder of a folder (ie.
  "folder/subfolder")

Most importantly this means that if your maildir folders exist in
eg. ~/Maildir/folder and ~/Maildir/folder/subfolder, Dovecot won't
see them unless you rename them to Maildir++ layout. v1.1 supports
them by adding :LAYOUT=fs to mail_location.
`---

As it stands the on-disk layout mbsync is currently using at least with
«SubFolders Maildir++» seems pretty weird to me, and as stated before
it seems to force ~/Maildir to not be a Maildir++, while its
subdirectories might be so.

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

On Mon, May 04, 2015 at 10:40:19PM +0200, Guillem Jover wrote:
> Either that or I'm very confused…
> 
i'd bet on the latter. ;)
Path is not the top-level folder - it is a "namespace" that contains
top-level folders.
this is also what you'd find on an actual imap server.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

On Mon, 2015-05-04 at 21:58:32 +0200, Oswald Buddenhagen wrote:
> On Mon, May 04, 2015 at 08:12:08PM +0200, Guillem Jover wrote:
> > It seems to me that either «SubFolders Maildir++» does not work as
> > documented or the documentation is not clear enough, (or I don't
> > understand it).

> your config attempts to have dots prefixed to the top-level folders'
> names. there is no standard that would cover that, and i'm not going to
> restore the fuzziness that allowed your configuration to work pretty
> much by accident.
> it might be possible to trick mbsync by setting the Path one level up
> and using "Slave :store:Maildir/", so it would think your top-level
> folders are subfolders.

AFAICS my layout is the standard Maildir++. A top Maildir with
subfolders inside that have a ‘.’ prefix. That is the exact same thing
as documented in
,
which is pretty explicit. Or
.

And in isync's master, AFAIUI the documented top/ is an actual Maildir
subfolder(?) (w/o a . prefix), inside the Maildir ~/Maildir/ dir? And
that one has subsubfolders. All of which seem to go against the
standard? Either that or I'm very confused…

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

On Mon, May 04, 2015 at 08:12:08PM +0200, Guillem Jover wrote:
> It seems to me that either «SubFolders Maildir++» does not work as
> documented or the documentation is not clear enough, (or I don't
> understand it).
> 
your config attempts to have dots prefixed to the top-level folders'
names. there is no standard that would cover that, and i'm not going to
restore the fuzziness that allowed your configuration to work pretty
much by accident.
it might be possible to trick mbsync by setting the Path one level up
and using "Slave :store:Maildir/", so it would think your top-level
folders are subfolders.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Hi!

On Mon, 2015-05-04 at 19:12:25 +0200, Oswald Buddenhagen wrote:
> you need to use SubFolders Verbatim, Path without the trailing dot, and
> rename the folders to actually be a real hierarchy with no leading dots.

Hmm, but I do want to use the Maildir++ layout (as documented in the
man page) which I'm currently using. Having to switch the on-disk
layout would be quite annoying, and I'd have to go back to use a
symlink farm to make the server and the clients all uniform. I'd
rather stick with the current version in unstable. :/

It seems to me that either «SubFolders Maildir++» does not work as
documented or the documentation is not clear enough, (or I don't
understand it).

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

you need to use SubFolders Verbatim, Path without the trailing dot, and
rename the folders to actually be a real hierarchy with no leading dots.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Hi!

On Mon, 2015-05-04 at 09:46:27 +0200, Oswald Buddenhagen wrote:
> i made a bunch of fixes (for 1.2.x) and added the SubFolders option to
> select the folder naming style (for 1.3), the latter of which should
> make the dot hack unnecessary (by consistently not using leading dots).
> let me know if any issues remain.

Right, I noticed the commits in git, tested and didn't work so I
thought there where probably more changes pending.

I might be doing something wrong though. Commenting Flatten, removing
the trailing . from Path, and adding «SubFolders Maildir++», results
in a hierarchy like follows:

  ~/mail/folder/.subfolder

Tried now with -DmM and it only finds INBOX for the local Maildir store.
Triying also the different SubFolders values and Verbatim is the only one
that sees all local Maildirs (in Maildir++ format), but then uses the
wrong hiearchy when syncing:

  ~/mail/folder/subfolder

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

i made a bunch of fixes (for 1.2.x) and added the SubFolders option to
select the folder naming style (for 1.3), the latter of which should
make the dot hack unnecessary (by consistently not using leading dots).
let me know if any issues remain.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

On Wed, 2015-04-08 at 09:53:40 +0200, Oswald Buddenhagen wrote:
> On Tue, Apr 07, 2015 at 06:42:02PM +0200, Guillem Jover wrote:
> > pattern '*' (effective '*'): Path, no INBOX
> > got mailbox list from slave:
> > [nothing]

> well, that explains "a bit".

Ah! Had not noticed that.

> i suspect this is due to the trailing dot in the Path specification,
> i.e., your attempt to create a namespace which uniformly uses leading
> dots, not only for subfolders.

Yes, given that dovecot uses Maildir++, I wanted to use it too in the
client w/o having to use a symlink farm.

> what changed is the level of trust mbsync puts into the box listings.
> i'm not quite sure why it doesn't try to create the slave mailboxes,
> though. will have to investigate.

Ok, thanks.

Regards,
Guillem


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

On Tue, Apr 07, 2015 at 06:42:02PM +0200, Guillem Jover wrote:
> pattern '*' (effective '*'): Path, no INBOX
> got mailbox list from slave:
> [nothing]
>
well, that explains "a bit".

i suspect this is due to the trailing dot in the Path specification,
i.e., your attempt to create a namespace which uniformly uses leading
dots, not only for subfolders.

what changed is the level of trust mbsync puts into the box listings.
i'm not quite sure why it doesn't try to create the slave mailboxes,
though. will have to investigate.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Hi

On Tue, 2015-04-07 at 09:04:13 +0200, Oswald Buddenhagen wrote:
> the output of -DMn would be a good start in either case.

Ok, attached a redacted log (please let me know if I omitted something
that would otherwise help). And a (redacted) gdb dump of the state just
prior to one of the instance errors:

,---
Breakpoint 1, box_confirmed2 (svars=0x64db20, t=1) at sync.c:1056
1056error( "Error: channel %s: %s 
%s cannot be opened.\n",
(gdb) p *svars
$6 = {t = {0, 1}, cb = 0x405e08 , aux = 0x7fffe120, 
  dname = 0x64cdb0 "/.state/mbsync/debian.lists.announce", 
  jname = 0x6a6440 "/.state/mbsync/debian.lists.announce.journal",
  nname = 0x6a64b0 "/.state/mbsync/debian.lists.announce.new", 
  lname = 0x6a6520 "/.state/mbsync/debian.lists.announce.lock", 
  box_name = {0x64dc40 "debian/lists/announce", 
0x64dc60 "debian.lists.announce"}, jfp = 0x0, nfp = 0x0, srecs = 0x64cdf0, 
  srecadd = 0x6b34b0, chan = 0x628d00, ctx = {0x77f99010, 0x62a680}, 
  drv = {0x627860 , 0x627940 }, orig_name = {
0x64c880 "debian/lists/announce", 0x64c880 "debian/lists/announce"}, 
  new_msgs = {0x0, 0x0}, state = {25600, 8192}, ref_count = 2, nsrecs = 102, 
  ret = 0, lfd = 5, existing = 1, replayed = 0, new_pending = {0, 0}, 
  flags_pending = {0, 0}, trash_pending = {0, 0}, maxuid = {103, 0}, 
  newmaxuid = {103, 0}, uidval = {1207596529, 1404794291}, newuid = {0, 0}, 
  mmaxxuid = 2147483647, smaxxuid = 0}
(gdb) p *svars->chan
$7 = {next = 0x0, name = 0x628d60 "server", stores = {0x628b70, 0x6280f0}, 
  boxes = {0x0, 0x0}, sync_state = 0x628e30 "/.state/mbsync/", 
  patterns = 0x628e10, ops = {7183, 47}, max_messages = 0, 
  expire_unread = -1 '\377', use_internal_date = 0 '\000'}
(gdb) c
Error: channel server: slave debian/lists/announce cannot be opened.
C: 0/1  B: 3/74  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
`---

If you need additional debugging dumps, I'll gladly oblige.

Thanks,
Guillem
Opening slave store local...
pattern '*' (effective '*'): Path, no INBOX
got mailbox list from slave:
Connection is now encrypted
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE 
AUTH=PLAIN] Dovecot ready.
Logging in...
Authenticating with SASL mechanism PLAIN...
>>> 1 AUTHENTICATE PLAIN 
1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT 
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND 
URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED 
I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH 
LIST-STATUS SPECIAL-USE BINARY MOVE] Logged in
>>> 2 NAMESPACE
* NAMESPACE (("" "/")) NIL NIL
2 OK Namespace completed.
pattern '*' (effective '*'): Path, no INBOX
>>> 3 LIST "" "*"
* LIST (\HasNoChildren) "/" debian/lists/announce
* LIST (\HasNoChildren) "/" debian/lists/devel-announce
* LIST (\HasNoChildren) "/" debian/lists/security-announce
[…]
* LIST (\HasNoChildren) "/" INBOX
3 OK List completed.
got mailbox list from master:
  INBOX
  debian/lists/security-announce
  debian/lists/devel-announce
  debian/lists/announce
[…]
Opening master box debian/lists/announce...
>>> 6 SELECT "debian/lists/announce"
Opening slave box debian/lists/announce...
* OK [CLOSED] Previous mailbox closed.
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags 
permitted.
* 102 EXISTS
* 0 RECENT
* OK [UNSEEN 70] First unseen.
* OK [UIDVALIDITY 1207596529] UIDs valid
* OK [UIDNEXT 104] Predicted next UID
6 OK [READ-WRITE] Select completed (0.000 secs).
Error: channel server: slave debian/lists/announce cannot be opened.
[…]
Opening master box debian/lists/devel-announce...
>>> 11 SELECT "debian/lists/devel-announce"
Opening slave box debian/lists/devel-announce...
* OK [CLOSED] Previous mailbox closed.
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags 
permitted.
* 2055 EXISTS
* 0 RECENT
* OK [UNSEEN 1243] First unseen.
* OK [UIDVALIDITY 1207596536] UIDs valid
* OK [UIDNEXT 2057] Predicted next UID
11 OK [READ-WRITE] Select completed (0.001 secs).
Error: channel server: slave debian/lists/devel-announce cannot be opened.
[…]
Opening master box debian/lists/security-announce...
>>> 24 SELECT "debian/lists/security-announce"
Opening slave box debian/lists/security-announce...
* OK [CLOSED] Previous mailbox closed.
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags 
permitted.
* 1924 EXISTS
* 0 RECENT
* OK [UNSEEN 1924] First unseen.
* OK [UIDVALIDITY 1207612933] UIDs valid
* OK [UIDNEXT 1925] Predicted next UID
24 OK [READ-WRITE] Select completed (0.000 secs).
Error: channel server: slave debian/lists/security-announce cannot be opened.
[…]
>>> 78 LOGOUT
* BYE Logging out
78 OK Logout completed.

Bug#782054: mbsync: New version cannot open Maildir boxes

[Oswald Buddenhagen]

strange, i'd have suspected d8225390fcd3d31577d3d74ab3d18b8762c5008b
first (dovecot users are expected to be affected in particular, as many
appear to have a NAMESPACE of "INBOX." configured).

the output of -DMn would be a good start in either case.


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org

Bug#782054: mbsync: New version cannot open Maildir boxes

[Guillem Jover]

Package: isync
Version: 1.2.0-1
Severity: important

Hi!

I tried the new upstream release (thanks!), and it fails to open local
Maildir boxes, using exactly the same configuration file. For each
mailbox the error message is:

,---
$ mbsync -a
…
C: 0/1  B: 2/74  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Error: channel server: slave debian/lists/announce cannot be opened.
…
`---

And the mails are marked in the server (Dovecot) as having been
retrieved (switched from new to old), but they are not in the local
store. I tried some of the new verbose and debug options, but they
are not very helpful when it comes to local boxes. But if I do:

,---
$ mbsync server:debian/lists/announce
C: 1/1  B: 1/1  M: +0/0 *0/0 #0/0  S: +2/2 *0/0 #0/0
`---

It gets properly synced to the local box, w/o any error. Bisecting it
gives me commit d9a983add6c627f50f3894f938e4e4892476ad22 as the culprit.


And this is a redacted version of my config:

,--- .mbsyncrc ---
MaildirStore local
Path ~/mail/.
Inbox ~/mail/.my.inbox
Flatten .

IMAPStore server
Host server-address
User server-user
UseIMAPS yes
RequireSSL yes
UseTLSv1 no
UseTLSv1.1 no
UseTLSv1.2 yes
CertificateFile cert-path

Channel server
Master :server:
Slave :local:
Patterns * !INBOX !Archive/ !Spam/
Create Slave
Expunge None
Sync Pull Push New ReNew Flags Delete
SyncState ~/.state/mbsync/
`---

Thanks,
Guillem


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org