Re: Folder subscription issue

[ellie timoney]

I think they said 3.0.8, which is not the latest 3.0, but I also don't think 
anything about replication/subscriptions/etc has changed since then, so it 
might as well be

It would also be very interesting to know whether the master and replica 
definitely both have the same value for "unixhierarchysep". I don't think it 
should matter? But maybe there's a bug hiding there that's making it matter.

On Sun, Jul 14, 2019, at 11:03 PM, Bron Gondwana wrote:
> And this is even more exciting and I'd love to know the version!
> 
> Bron.
> 
> On Fri, Jul 12, 2019, at 23:33, Egoitz Aurrekoetxea wrote:
>> By the way I think I found some more…..
>> 
>> When a sync_client in user mode… creates a non existing user in the replica… 
>> if the user has a dot… the quota gets properly created, but seen, sub files 
>> are wrongly created… for instance….
>> 
>> -rw--- 1 cyrus cyrus 2576528 Jul 12 13:03 f.veiga.conversations
>> -rw--- 1 cyrus cyrus 88 Jul 12 13:03 f.veiga.counters
>> -rw--- 1 cyrus cyrus 451 Jul 12 10:43 f.veiga.sub
>> -rw--- 1 cyrus cyrus 16 Jul 12 01:32 f.veiga.xapianactive
>> -rw--- 1 cyrus cyrus 2584 Apr 3 01:48 f^veiga.seen
>> -rw--- 1 cyrus cyrus 316 Apr 3 01:48 f^veiga.sub
>> 
>> The Apr 3 files are from the initial sync when the box was empty… the first 
>> ones (with the dot and not ^) are the actual files being used by Cyrus…. And 
>> updated with the replication and so…
>> 
>> It seems this to be the reason because I didn’t see in the initial sync any 
>> subscribed folders… but later when set them from a mua where properly 
>> replicated…
>> 
>> With Sieve seemed to happen something similar… but in this case with the ‘^’ 
>> and ‘.’ In the directory name….
>> 
>> It’s like the name used for creating the subscribing and seen databases in a 
>> user mode replication was not properly handled by Cyrus… because it does the 
>> same as with quota database with indeed with ‘^’ is properly created….
>> 
>> Thanks! Bye!
>> 
>> 
>> sarenet
>> *Egoitz Aurrekoetxea*
>> Dpto. de sistemas
>> 944 209 470
>> Parque Tecnológico. Edificio 103
>> 48170 Zamudio (Bizkaia)
>> ego...@sarenet.es
>> www.sarenet.es
>> 
>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>> 
>>> El 10 jul 2019, a las 11:39, Egoitz Aurrekoetxea  
>>> escribió:
>>> 
>>> Hi!
>>> 
>>> It happens with any folder… in fact Trash is not the folder we would 
>>> announce in special-use as Trash… it’s just a normal folder really here…. 
>>> It’s a generally happening thing with rename of folders inside the own 
>>> hierarchy inside the own user… (don’t really know if a rename mailbox for 
>>> changing partition would have the same issue). Not something related to the 
>>> Trash folder...
>>> 
>>> Cheers!
>>> 
>>> 
>>> sarenet
>>> *Egoitz Aurrekoetxea*
>>> Dpto. de sistemas
>>> 944 209 470
>>> Parque Tecnológico. Edificio 103
>>> 48170 Zamudio (Bizkaia)
>>> ego...@sarenet.es
>>> www.sarenet.es
>>> 
>>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>>> 
 El 10 jul 2019, a las 11:34, Sebastian Hagedorn  
 escribió:
 
 Hi,
 
 I'm curious if this only happens for rename to trash, or for all renames
 of subscribed folders. IMHO it makes no sense to automatically subscribe
 to a folder in the trash. So perhaps the bug isn't in the replication
 code but rather in the handling of rename to trash?
 
 
 Am 10.07.19 um 11:11 Uhr schrieb Egoitz Aurrekoetxea:
> About the folder subscription issue, I think I got something, at least a
> close approximation... When a user causes in mua a rename mailbox (a
> rename for a folder caused by a folder move in hierarchy), after the own
> rename, if folders were subscribed “should” (for the "plain user" at
> least) become subscribed in the new path. It seems that after a user
> rename in Cyrus the new folder is automatically subscribed (even if no
> subscribe command is sent by the mua). But this, does not cause in the
> replica (in the slave, if SUB is not sent by the client)
> a sync_apply_changesub() or something like entering in the
> “ move_subscription” condition in mboxlist_renamemailbox(), and then,
> the folder is properly renamed but not subscribed in the slave. I think
> this is what I’m suffering. Obviously, if after a rename the mua sends a
> subscribe too, no issue is seen. I think the problem happens when a
> mailbox rename happens and a SUB is not send later.
> 
> An example : 
> 
> The folder domain.com
> !user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
> is moved (renamed) to trash.
> 
> May 16 16:16:50 mx6c imap[83976]: conversations_rename_folder: renamed
> domain.com
> !user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
> to domain.com !user.parta^partb.Trash.XINGANG
> May 16 16:16:50 mx6c imap[83976]: Rename: domain.

Re: Folder subscription issue

[Bron Gondwana]

And this is even more exciting and I'd love to know the version!

Bron.

On Fri, Jul 12, 2019, at 23:33, Egoitz Aurrekoetxea wrote:
> By the way I think I found some more…..
> 
> When a sync_client in user mode… creates a non existing user in the replica… 
> if the user has a dot… the quota gets properly created, but seen, sub files 
> are wrongly created… for instance….
> 
> -rw--- 1 cyrus cyrus 2576528 Jul 12 13:03 f.veiga.conversations
> -rw--- 1 cyrus cyrus 88 Jul 12 13:03 f.veiga.counters
> -rw--- 1 cyrus cyrus 451 Jul 12 10:43 f.veiga.sub
> -rw--- 1 cyrus cyrus 16 Jul 12 01:32 f.veiga.xapianactive
> -rw--- 1 cyrus cyrus 2584 Apr 3 01:48 f^veiga.seen
> -rw--- 1 cyrus cyrus 316 Apr 3 01:48 f^veiga.sub
> 
> The Apr 3 files are from the initial sync when the box was empty… the first 
> ones (with the dot and not ^) are the actual files being used by Cyrus…. And 
> updated with the replication and so…
> 
> It seems this to be the reason because I didn’t see in the initial sync any 
> subscribed folders… but later when set them from a mua where properly 
> replicated…
> 
> With Sieve seemed to happen something similar… but in this case with the ‘^’ 
> and ‘.’ In the directory name….
> 
> It’s like the name used for creating the subscribing and seen databases in a 
> user mode replication was not properly handled by Cyrus… because it does the 
> same as with quota database with indeed with ‘^’ is properly created….
> 
> Thanks! Bye!
> 
> 
> sarenet
> *Egoitz Aurrekoetxea*
> Dpto. de sistemas
> 944 209 470
> Parque Tecnológico. Edificio 103
> 48170 Zamudio (Bizkaia)
> ego...@sarenet.es
> www.sarenet.es
> 
> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
> 
>> El 10 jul 2019, a las 11:39, Egoitz Aurrekoetxea  
>> escribió:
>> 
>> Hi!
>> 
>> It happens with any folder… in fact Trash is not the folder we would 
>> announce in special-use as Trash… it’s just a normal folder really here…. 
>> It’s a generally happening thing with rename of folders inside the own 
>> hierarchy inside the own user… (don’t really know if a rename mailbox for 
>> changing partition would have the same issue). Not something related to the 
>> Trash folder...
>> 
>> Cheers!
>> 
>> 
>> sarenet
>> *Egoitz Aurrekoetxea*
>> Dpto. de sistemas
>> 944 209 470
>> Parque Tecnológico. Edificio 103
>> 48170 Zamudio (Bizkaia)
>> ego...@sarenet.es
>> www.sarenet.es
>> 
>> Antes de imprimir este correo electrónico piense si es necesario hacerlo.
>> 
>>> El 10 jul 2019, a las 11:34, Sebastian Hagedorn  
>>> escribió:
>>> 
>>> Hi,
>>> 
>>> I'm curious if this only happens for rename to trash, or for all renames
>>> of subscribed folders. IMHO it makes no sense to automatically subscribe
>>> to a folder in the trash. So perhaps the bug isn't in the replication
>>> code but rather in the handling of rename to trash?
>>> 
>>> 
>>> Am 10.07.19 um 11:11 Uhr schrieb Egoitz Aurrekoetxea:
 About the folder subscription issue, I think I got something, at least a
 close approximation... When a user causes in mua a rename mailbox (a
 rename for a folder caused by a folder move in hierarchy), after the own
 rename, if folders were subscribed “should” (for the "plain user" at
 least) become subscribed in the new path. It seems that after a user
 rename in Cyrus the new folder is automatically subscribed (even if no
 subscribe command is sent by the mua). But this, does not cause in the
 replica (in the slave, if SUB is not sent by the client)
 a sync_apply_changesub() or something like entering in the
 “ move_subscription” condition in mboxlist_renamemailbox(), and then,
 the folder is properly renamed but not subscribed in the slave. I think
 this is what I’m suffering. Obviously, if after a rename the mua sends a
 subscribe too, no issue is seen. I think the problem happens when a
 mailbox rename happens and a SUB is not send later.
 
 An example : 
 
 The folder domain.com
 !user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
 is moved (renamed) to trash.
 
 May 16 16:16:50 mx6c imap[83976]: conversations_rename_folder: renamed
 domain.com
 !user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
 to domain.com !user.parta^partb.Trash.XINGANG
 May 16 16:16:50 mx6c imap[83976]: Rename: domain.com
 !user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
 -> domain.com !user.parta^partb.Trash.XINGANG
 May 16 16:16:50 mx6c imap[83976]: Deleted mailbox domain.com
 !user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
 
 In the master (sync), the folder in Trash is subscribed but in the slave
 it is not, and remains subscribed the folder in the original location in
 the “.sub” file.
 
 A diff between the master (replication client) .sub file and the slave’s
 o

Re: Folder subscription issue

[Bron Gondwana]

On Wed, Jul 10, 2019, at 19:11, Egoitz Aurrekoetxea wrote:
> Good morning,
> 
> In previous thread (with the title slightly incorrect) I talk about an issue 
> suffered some day with Sieve script and folder subscriptions. The Sieve part, 
> was related to the migration, so for the moment let’s forget about it (for me 
> at least) as it has never reproduced again since then.

Sorry, I missed the previous thread. Did you mention which version of Cyrus you 
are running? There's clearly a bug and it would be good to know which 
version(s) it affects.

> About the folder subscription issue, I think I got something, at least a 
> close approximation... When a user causes in mua a rename mailbox (a rename 
> for a folder caused by a folder move in hierarchy), after the own rename, if 
> folders were subscribed “should” (for the "plain user" at least) become 
> subscribed in the new path. It seems that after a user rename in Cyrus the 
> new folder is automatically subscribed (even if no subscribe command is sent 
> by the mua). But this, does not cause in the replica (in the slave, if SUB is 
> not sent by the client) a sync_apply_changesub() or something like entering 
> in the “ move_subscription” condition in mboxlist_renamemailbox(), and then, 
> the folder is properly renamed but not subscribed in the slave. I think this 
> is what I’m suffering. Obviously, if after a rename the mua sends a subscribe 
> too, no issue is seen. I think the problem happens when a mailbox rename 
> happens and a SUB is not send later.

Subscriptions aren't automatically renamed when a folder is renamed, but they 
should be automatically renamed for a user rename. Intermediate replicated 
states can be bit messy due to race conditions with replication, but I believe 
it should always wind up in the final correct state. If not, that's a bug for 
sure!

> An example : 
> 
> The folder domain.com!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG is 
> moved (renamed) to trash.
> 
> May 16 16:16:50 mx6c imap[83976]: conversations_rename_folder: renamed 
> domain.com!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG to 
> domain.com!user.parta^partb.Trash.XINGANG
> May 16 16:16:50 mx6c imap[83976]: Rename: 
> domain.com!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG -> 
> domain.com!user.parta^partb.Trash.XINGANG
> May 16 16:16:50 mx6c imap[83976]: Deleted mailbox 
> domain.com!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
> 
> In the master (sync), the folder in Trash is subscribed but in the slave it 
> is not, and remains subscribed the folder in the original location in the 
> “.sub” file.

This might just be a matter of failing to sync_log the subscription in that 
codepath. Hmm...

But there is a question of whether we should even be renaming the subscription 
- RFC3501 is a little silent on this issue, but it does say in a couple of 
places that the server MUST NOT remove a subscription even if the mailbox with 
that name doesn't exist, which makes renaming subscriptions a bit unclear. I'll 
check in with the authors of draft-ietf-extra-imap4rev2 and ask for this to be 
clarified next time around!

> A diff between the master (replication client) .sub file and the slave’s one 
> is (mx5 is the master, the client and 6 the slave): 
> 
> --- subscripciones-mx5c/parta.partb.sub 2019-07-10 08:47:29.0 +0200
> +++ subscripciones-mx6c/parta.partb.sub 2019-07-10 08:48:08.0 +0200
> 
> +domain.com!user.parta^partb.Archives.2019.TRAFICO.CHINA.XINGANG
> -domain.com!user.parta^partb.Trash.XINGANG
> 
> Perhaps a move_subscription to one or sync_apply_changesub should be forced 
> in order to fix it?. I have seen issues with Outlook 2016 and Thunderbird… 
> both mua… perhaps by RFC they should send the SUB command but… it’s the only 
> theory I could arrive to… I have a ton more cases like this one.. Data gets 
> properly handled but subscriptions have this issue (perhaps we could say is a 
> mua issue but….)..

Bron.
--
 Bron Gondwana, CEO, Fastmail Pty Ltd
 br...@fastmailteam.com


Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus