[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Stewart Smith]

On Mon, Jan 25, 2010 at 01:46:59PM +1300, martin f krafft wrote:
> Stewart, you've worked most on this so far. Would you like to share
> your thoughts?

Just posted a new thread with my latest experiments. Things look
rather good from a storage size point of view. Still a few things to
work out though.

-- 
Stewart Smith

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Stewart Smith]

On Mon, Jan 25, 2010 at 01:46:59PM +1300, martin f krafft wrote:
 Stewart, you've worked most on this so far. Would you like to share
 your thoughts?

Just posted a new thread with my latest experiments. Things look
rather good from a storage size point of view. Still a few things to
work out though.

-- 
Stewart Smith
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Sebastian Spaeth]

On Mon, 25 Jan 2010 14:49:00 +0100, "Sebastian Spaeth"  wrote:
> While notmuchsync fullfils my needs, it is a kludge. It needs to call
> "notmuch" for each mail where a MailDir flag has changed (which can be
> quite often on an initial run, where most mails are likely to be read),
> this can take a long, long time. It would makes sense IMHO to at least
> pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1].

Once python bindings exist for the notmuch shared library, I am sure we
can speed notmuchsync up a lot by keeping the connection open and
tagging all mails in one go rather than executing a separate binary for
each mail. So, this approach might still be feasible.
Sebastian

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Sebastian Spaeth]

On Mon, 25 Jan 2010 14:49:00 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 While notmuchsync fullfils my needs, it is a kludge. It needs to call
 notmuch for each mail where a MailDir flag has changed (which can be
 quite often on an initial run, where most mails are likely to be read),
 this can take a long, long time. It would makes sense IMHO to at least
 pick pioto's don't set unread if 'S' flag is set on notmuch new[1].

Once python bindings exist for the notmuch shared library, I am sure we
can speed notmuchsync up a lot by keeping the connection open and
tagging all mails in one go rather than executing a separate binary for
each mail. So, this approach might still be feasible.
Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Sebastian Spaeth  [2010.01.26.0249 +1300]:
> While notmuchsync fullfils my needs, it is a kludge. It needs to
> call "notmuch" for each mail where a MailDir flag has changed
> (which can be quite often on an initial run, where most mails are
> likely to be read), this can take a long, long time. It would
> makes sense IMHO to at least pick pioto's "don't set unread if 'S'
> flag is set" on notmuch new[1].

I am sure this could be implemented with libnotmuch if it proves to
be useful.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"it isn't pollution that's harming the environment.
 it's the impurities in our air and water that are doing it."
  - dan quayle

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Asheesh Laroia  [2010.01.25.1819 +1300]:
> You say "Ouch" but you should know Dovecot *already* does this. I
> don't mind interoperating with that.
>
> See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues
> with the specification", subsection "Locking". I term this theQ
> famous readdir() race.

Yikes. IMAP (including dovecot) just SUCKS.

> Without this lock, Maildir is fundamentally incompatible with IMAP
> -- one Maildir-using process modifying message flags could make
> a different Maildir-using process think said message is actually
> deleted. In the case of temporary disappearing mails in Mutt
> locally, that's not the end of the world. For IMAP, it will make
> the IMAP daemon (one of the Maildir-using processes) send a note
> to IMAP clients saying that the message has been deleted and
> expunged.
[?]
> Just don't fall into the trap of thinking Maildir is compatible
> with IMAP. It's not, because as I understand things, the
> filesystem doesn't guarantee that you can actually iterate across
> a directory's files if another process is modifying the list of
> files.

This is all perfect reason to concentrate even more on designing
a store that could potentially make IMAP obsolete once and for all!

The current idea is to sync Git downstream only, and find a way to
keep multiple copies of a tagstore in sync, by way of the "server
instance" (where mail is received/delivered). Deleting messages
would then be something like setting the notmuch::deleted tag, which
clients would honour; on the server, a cleanup process would run
regularly to actually delete the blobs associated with deleted
messages. This would then propogate the next time one pulls from
Git.

Whether to store history (commit objects) or just collections (tree
objects) needs to be investigated.

> >But there are still good reasons why you'd want to have IMAP
> >capability too, e.g. Webmail. Given the atomicity problems that
> >come from Git, maybe an IMAP server reading from the Git store
> >would make sense.
> 
> It wouldn't be too hard to write a FUSE filesystem that presented
> an interface to a Git repository that didn't allow the contents of
> files to be modified. Then Dovecot could think it's interacting
> with the filesystem.

Yes, a FUSE layer (which adds a daemon), or a lightweight access
API via libnotmuch. Probably the former using the latter. ;)

> Aww, I like Maildir flags, but if there's a sync tool, I'm fine
> with that.
[?]
> I'm not sure, but maybe it's safe if you refuse to ever modify
> a message's flags in the filename.

The main point is that there is nothing really in Maildir filenames
that you couldn't equally (and possibly better) represent in the
notmuch::* tag namespace, and then there is benefit in only having
one used primarily (which means notmuchsync can do whatever it
wants without affecting or messing with notmuch).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"if I can't dance, i don't want to be part of your revolution."
- emma goldman

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Sebastian Spaeth]

On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft  
wrote:
> I think we all kinda agreed that the Maildir flags should not be
> used by notmuch and that things like Sebastian's notmuchsync should
> be used if people wanted flags represented in Maildir filenames.

While notmuchsync fullfils my needs, it is a kludge. It needs to call
"notmuch" for each mail where a MailDir flag has changed (which can be
quite often on an initial run, where most mails are likely to be read),
this can take a long, long time. It would makes sense IMHO to at least
pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1].

Or - at the very least - not to set the "unread" flag by default.

Sebastian

[1] pioto's noarg-count branch (http://git.pioto.org/gitweb/notmuch.git
Announced in mail id:20100121204201.1C82764A0E at aether.pioto.org)

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Asheesh Laroia  [2010.01.21.1928 +1300]:
> >I suppose that I never actually considered merges on the IMAP
> >server side, but obviously the IMAP server has to work off a clone,
> >and that means it needs to merge.
> 
> It's not "merge" that's unsafe; that just builds a tree in the git
> index (assuming no conflicts). It's the ensuing process of git
> writing a tree to the filesystem that is problematic.

There is no way to make that atomic, I am afraid. As you say.

> I could probably actually write a wrapper that locks the Maildir
> while git is operating. It would probably be specific to each IMAP
> server.

Ouch! I'd really rather not go there.

> Note that this mean git is fundamentally incompatible with
> Maildir, not just IMAP servers.

We had an idea about using Git to replace IMAP altogether, along
with making notmuch use a bare Git repository as object store. The
idea is that notmuch uses low-level Git commands to access the .git
repository (from which you can still checkout a tree tying the blobs
into a Maildir). The benefit would be compression, lower inode count
(due to packs), and backups using clones/merges.

You could either have the MDA write to a Git repo on the server side
and use git packs to download mail to a local clone, or one could
have e.g. offlineimap grow a Git storage backend. The interface to
notmuch would be the same.

If we used this, all the rename and delete code would be refactored
into Git and could be removed from notmuch. In addition, notmuch
could actually use Git tree objects to represent the results of
searches, and you could checkout these trees. However, deleting
messages from search results would not have any effect on the
message or its existence in other search results, much like what
happens with mairix nowadays.

I think we all kinda agreed that the Maildir flags should not be
used by notmuch and that things like Sebastian's notmuchsync should
be used if people wanted flags represented in Maildir filenames.

Instead of a Maildir checkout, notmuch could provide an interface to
browse the store contents in a way that could make it accessible to
mutt. The argument is that with 'notmuch {ls,cat,rm,?}', a mutt
backend could be trivially written. I am not sure about that, but
it's worth a try.

But there are still good reasons why you'd want to have IMAP
capability too, e.g. Webmail. Given the atomicity problems that come
from Git, maybe an IMAP server reading from the Git store would make
sense.

However, this all sounds like a lot of NIH and reinvention. It's
a bit like the marriage between the hypothetical Maildir2 and Git,
which is definitely worth pursuing. Before we embark on any of this,
however, we'd need to define the way in which Git stores mail.

Stewart, you've worked most on this so far. Would you like to share
your thoughts?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"reife des mannes, das ist es,
 den ernst wiedergefunden zu haben, den
 man hatte als kind beim spiel."
-- friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Mike Kelly]

On Mon, 25 Jan 2010 14:49:00 +0100, "Sebastian Spaeth"  wrote:
> 
> On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft  
> wrote:
> > I think we all kinda agreed that the Maildir flags should not be
> > used by notmuch and that things like Sebastian's notmuchsync should
> > be used if people wanted flags represented in Maildir filenames.
> 
> While notmuchsync fullfils my needs, it is a kludge. It needs to call
> "notmuch" for each mail where a MailDir flag has changed (which can be
> quite often on an initial run, where most mails are likely to be read),
> this can take a long, long time. It would makes sense IMHO to at least
> pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1].

notmuchsync, as currently implemented, suffers from major performance
issues, in my opinion. It's a useful short term workaround, but not a
good long term solution.

But, I personally will always be using both notmuch and some other IMAP
client (my phone). I want the two to remain in sync easily enough.
notmuch is already much more robust with respect to that than sup, I
think (in terms of handling renames without barfing, etc).

At the very least, I want `notmuch new` to be able to:

  If it sees a rename that involves changing maildir flags, alter the
  related tags as necessary.

  Similarly, provide a mechanism for correlating the folder name with
  some set of tags, and change those tags as messages are moved around.  

For example, I might have:

~/.notmuch-config:

[database]
path=/home/pioto/mail
...
[tags]
pioto at pioto.org/INBOX.ListMail.notmuch = notmuch

So, a 'tags' section, where each key is the folder name, relative to the
db path, and the value is one or more tag names

This means that I could relabel a message in gmail, for example, and
have the changes apply to notmuch at my next offlineimap run. And, it
means that my existing procmail rules will still be useful both to
notmuch, and to my phone, for the purpose of categorizing things.

I agree that all this should be optional. But, since it is likely the
behavior most people would expect, I think it should be the default.

PS. You mean the 'new-unread' branch, not the 'noarg-count' branch, from
my repo.

-- 
Mike Kelly

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Asheesh Laroia]

On Mon, 25 Jan 2010, martin f krafft wrote:

> also sprach Asheesh Laroia  [2010.01.21.1928 
> +1300]:
>>> I suppose that I never actually considered merges on the IMAP server 
>>> side, but obviously the IMAP server has to work off a clone, and that 
>>> means it needs to merge.
>>
>> It's not "merge" that's unsafe; that just builds a tree in the git 
>> index (assuming no conflicts). It's the ensuing process of git writing 
>> a tree to the filesystem that is problematic.
>
> There is no way to make that atomic, I am afraid. As you say.
>
>> I could probably actually write a wrapper that locks the Maildir while 
>> git is operating. It would probably be specific to each IMAP server.
>
> Ouch! I'd really rather not go there.

You say "Ouch" but you should know Dovecot *already* does this. I don't 
mind interoperating with that.

See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues with 
the specification", subsection "Locking". I term this the famous readdir() 
race. Without this lock, Maildir is fundamentally incompatible with IMAP 
-- one Maildir-using process modifying message flags could make a 
different Maildir-using process think said message is actually deleted. In 
the case of temporary disappearing mails in Mutt locally, that's not the 
end of the world. For IMAP, it will make the IMAP daemon (one of the 
Maildir-using processes) send a note to IMAP clients saying that the 
message has been deleted and expunged.

>> Note that this mean git is fundamentally incompatible with Maildir, not 
>> just IMAP servers.
>
> We had an idea about using Git to replace IMAP altogether, along with 
> making notmuch use a bare Git repository as object store. The idea is 
> that notmuch uses low-level Git commands to access the .git repository 
> (from which you can still checkout a tree tying the blobs into a 
> Maildir). The benefit would be compression, lower inode count (due to 
> packs), and backups using clones/merges.

Sure, that makes sense to me.

> You could either have the MDA write to a Git repo on the server side and 
> use git packs to download mail to a local clone, or one could have e.g. 
> offlineimap grow a Git storage backend. The interface to notmuch would 
> be the same.

Yeah, I generally like this.

> If we used this, all the rename and delete code would be refactored into 
> Git and could be removed from notmuch. In addition, notmuch could 
> actually use Git tree objects to represent the results of searches, and 
> you could checkout these trees. However, deleting messages from search 
> results would not have any effect on the message or its existence in 
> other search results, much like what happens with mairix nowadays.

That's okay with me.

> I think we all kinda agreed that the Maildir flags should not be used by 
> notmuch and that things like Sebastian's notmuchsync should be used if 
> people wanted flags represented in Maildir filenames.

Aww, I like Maildir flags, but if there's a sync tool, I'm fine with that.

> Instead of a Maildir checkout, notmuch could provide an interface to 
> browse the store contents in a way that could make it accessible to 
> mutt. The argument is that with 'notmuch {ls,cat,rm,?}', a mutt backend 
> could be trivially written. I am not sure about that, but it's worth a 
> try.

Sure.

> But there are still good reasons why you'd want to have IMAP capability 
> too, e.g. Webmail. Given the atomicity problems that come from Git, 
> maybe an IMAP server reading from the Git store would make sense.

It wouldn't be too hard to write a FUSE filesystem that presented an 
interface to a Git repository that didn't allow the contents of files to 
be modified. Then Dovecot could think it's interacting with the 
filesystem.

> However, this all sounds like a lot of NIH and reinvention. It's
> a bit like the marriage between the hypothetical Maildir2 and Git,
> which is definitely worth pursuing. Before we embark on any of this,
> however, we'd need to define the way in which Git stores mail.

Sure. If it were me, I'd just say, "For phase 1 of notmuch, just have git 
store Maildir spools." When you need a filesystem interface for e.g. 
Dovecot, have a FUSE wrapper.

See how far that can take you, and then see if version 2 is necessary. 
(-:

> Stewart, you've worked most on this so far. Would you like to share your 
> thoughts?

I'll listen, too.

Just don't fall into the trap of thinking Maildir is compatible with IMAP. 
It's not, because as I understand things, the filesystem doesn't guarantee 
that you can actually iterate across a directory's files if another 
process is modifying the list of files.

I'm not sure, but maybe it's safe if you refuse to ever modify a 
message's flags in the filename.

Anyway, as I see it, further hacks that aren't much worse than Dovecot's 
should be considered okay, unless you have a more elegant design up your 
sleeve.

If I'm slightly wrong about something, try to give me the benefit of 
doubt. It's past 

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Sebastian Spaeth]

On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft madd...@madduck.net wrote:
 I think we all kinda agreed that the Maildir flags should not be
 used by notmuch and that things like Sebastian's notmuchsync should
 be used if people wanted flags represented in Maildir filenames.

While notmuchsync fullfils my needs, it is a kludge. It needs to call
notmuch for each mail where a MailDir flag has changed (which can be
quite often on an initial run, where most mails are likely to be read),
this can take a long, long time. It would makes sense IMHO to at least
pick pioto's don't set unread if 'S' flag is set on notmuch new[1].

Or - at the very least - not to set the unread flag by default.

Sebastian

[1] pioto's noarg-count branch (http://git.pioto.org/gitweb/notmuch.git
Announced in mail id:20100121204201.1c82764...@aether.pioto.org)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Mike Kelly]

On Mon, 25 Jan 2010 14:49:00 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 
 On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft madd...@madduck.net 
 wrote:
  I think we all kinda agreed that the Maildir flags should not be
  used by notmuch and that things like Sebastian's notmuchsync should
  be used if people wanted flags represented in Maildir filenames.
 
 While notmuchsync fullfils my needs, it is a kludge. It needs to call
 notmuch for each mail where a MailDir flag has changed (which can be
 quite often on an initial run, where most mails are likely to be read),
 this can take a long, long time. It would makes sense IMHO to at least
 pick pioto's don't set unread if 'S' flag is set on notmuch new[1].

notmuchsync, as currently implemented, suffers from major performance
issues, in my opinion. It's a useful short term workaround, but not a
good long term solution.

But, I personally will always be using both notmuch and some other IMAP
client (my phone). I want the two to remain in sync easily enough.
notmuch is already much more robust with respect to that than sup, I
think (in terms of handling renames without barfing, etc).

At the very least, I want `notmuch new` to be able to:

  If it sees a rename that involves changing maildir flags, alter the
  related tags as necessary.

  Similarly, provide a mechanism for correlating the folder name with
  some set of tags, and change those tags as messages are moved around.  

For example, I might have:

~/.notmuch-config:

[database]
path=/home/pioto/mail
...
[tags]
pi...@pioto.org/INBOX.ListMail.notmuch = notmuch

So, a 'tags' section, where each key is the folder name, relative to the
db path, and the value is one or more tag names

This means that I could relabel a message in gmail, for example, and
have the changes apply to notmuch at my next offlineimap run. And, it
means that my existing procmail rules will still be useful both to
notmuch, and to my phone, for the purpose of categorizing things.

I agree that all this should be optional. But, since it is likely the
behavior most people would expect, I think it should be the default.

PS. You mean the 'new-unread' branch, not the 'noarg-count' branch, from
my repo.

-- 
Mike Kelly
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Sebastian Spaeth sebast...@sspaeth.de [2010.01.26.0249 +1300]:
 While notmuchsync fullfils my needs, it is a kludge. It needs to
 call notmuch for each mail where a MailDir flag has changed
 (which can be quite often on an initial run, where most mails are
 likely to be read), this can take a long, long time. It would
 makes sense IMHO to at least pick pioto's don't set unread if 'S'
 flag is set on notmuch new[1].

I am sure this could be implemented with libnotmuch if it proves to
be useful.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
it isn't pollution that's harming the environment.
 it's the impurities in our air and water that are doing it.
  - dan quayle
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Asheesh Laroia ashe...@asheesh.org [2010.01.21.1928 +1300]:
 I suppose that I never actually considered merges on the IMAP
 server side, but obviously the IMAP server has to work off a clone,
 and that means it needs to merge.
 
 It's not merge that's unsafe; that just builds a tree in the git
 index (assuming no conflicts). It's the ensuing process of git
 writing a tree to the filesystem that is problematic.

There is no way to make that atomic, I am afraid. As you say.

 I could probably actually write a wrapper that locks the Maildir
 while git is operating. It would probably be specific to each IMAP
 server.

Ouch! I'd really rather not go there.

 Note that this mean git is fundamentally incompatible with
 Maildir, not just IMAP servers.

We had an idea about using Git to replace IMAP altogether, along
with making notmuch use a bare Git repository as object store. The
idea is that notmuch uses low-level Git commands to access the .git
repository (from which you can still checkout a tree tying the blobs
into a Maildir). The benefit would be compression, lower inode count
(due to packs), and backups using clones/merges.

You could either have the MDA write to a Git repo on the server side
and use git packs to download mail to a local clone, or one could
have e.g. offlineimap grow a Git storage backend. The interface to
notmuch would be the same.

If we used this, all the rename and delete code would be refactored
into Git and could be removed from notmuch. In addition, notmuch
could actually use Git tree objects to represent the results of
searches, and you could checkout these trees. However, deleting
messages from search results would not have any effect on the
message or its existence in other search results, much like what
happens with mairix nowadays.

I think we all kinda agreed that the Maildir flags should not be
used by notmuch and that things like Sebastian's notmuchsync should
be used if people wanted flags represented in Maildir filenames.

Instead of a Maildir checkout, notmuch could provide an interface to
browse the store contents in a way that could make it accessible to
mutt. The argument is that with 'notmuch {ls,cat,rm,…}', a mutt
backend could be trivially written. I am not sure about that, but
it's worth a try.

But there are still good reasons why you'd want to have IMAP
capability too, e.g. Webmail. Given the atomicity problems that come
from Git, maybe an IMAP server reading from the Git store would make
sense.

However, this all sounds like a lot of NIH and reinvention. It's
a bit like the marriage between the hypothetical Maildir2 and Git,
which is definitely worth pursuing. Before we embark on any of this,
however, we'd need to define the way in which Git stores mail.

Stewart, you've worked most on this so far. Would you like to share
your thoughts?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
reife des mannes, das ist es,
 den ernst wiedergefunden zu haben, den
 man hatte als kind beim spiel.
-- friedrich nietzsche
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[martin f krafft]

also sprach Asheesh Laroia ashe...@asheesh.org [2010.01.25.1819 +1300]:
 You say Ouch but you should know Dovecot *already* does this. I
 don't mind interoperating with that.

 See http://wiki.dovecot.org/MailboxFormat/Maildir, section Issues
 with the specification, subsection Locking. I term this theQ
 famous readdir() race.

Yikes. IMAP (including dovecot) just SUCKS.

 Without this lock, Maildir is fundamentally incompatible with IMAP
 -- one Maildir-using process modifying message flags could make
 a different Maildir-using process think said message is actually
 deleted. In the case of temporary disappearing mails in Mutt
 locally, that's not the end of the world. For IMAP, it will make
 the IMAP daemon (one of the Maildir-using processes) send a note
 to IMAP clients saying that the message has been deleted and
 expunged.
[…]
 Just don't fall into the trap of thinking Maildir is compatible
 with IMAP. It's not, because as I understand things, the
 filesystem doesn't guarantee that you can actually iterate across
 a directory's files if another process is modifying the list of
 files.

This is all perfect reason to concentrate even more on designing
a store that could potentially make IMAP obsolete once and for all!

The current idea is to sync Git downstream only, and find a way to
keep multiple copies of a tagstore in sync, by way of the server
instance (where mail is received/delivered). Deleting messages
would then be something like setting the notmuch::deleted tag, which
clients would honour; on the server, a cleanup process would run
regularly to actually delete the blobs associated with deleted
messages. This would then propogate the next time one pulls from
Git.

Whether to store history (commit objects) or just collections (tree
objects) needs to be investigated.

 But there are still good reasons why you'd want to have IMAP
 capability too, e.g. Webmail. Given the atomicity problems that
 come from Git, maybe an IMAP server reading from the Git store
 would make sense.
 
 It wouldn't be too hard to write a FUSE filesystem that presented
 an interface to a Git repository that didn't allow the contents of
 files to be modified. Then Dovecot could think it's interacting
 with the filesystem.

Yes, a FUSE layer (which adds a daemon), or a lightweight access
API via libnotmuch. Probably the former using the latter. ;)

 Aww, I like Maildir flags, but if there's a sync tool, I'm fine
 with that.
[…]
 I'm not sure, but maybe it's safe if you refuse to ever modify
 a message's flags in the filename.

The main point is that there is nothing really in Maildir filenames
that you couldn't equally (and possibly better) represent in the
notmuch::* tag namespace, and then there is benefit in only having
one used primarily (which means notmuchsync can do whatever it
wants without affecting or messing with notmuch).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if I can't dance, i don't want to be part of your revolution.
- emma goldman
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch