Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-02-14 Thread Stewart Smith
On Mon, Jan 25, 2010 at 01:46:59PM +1300, martin f krafft wrote:
 Stewart, you've worked most on this so far. Would you like to share
 your thoughts?

Just posted a new thread with my latest experiments. Things look
rather good from a storage size point of view. Still a few things to
work out though.

-- 
Stewart Smith
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-27 Thread Sebastian Spaeth
On Mon, 25 Jan 2010 14:49:00 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 While notmuchsync fullfils my needs, it is a kludge. It needs to call
 notmuch for each mail where a MailDir flag has changed (which can be
 quite often on an initial run, where most mails are likely to be read),
 this can take a long, long time. It would makes sense IMHO to at least
 pick pioto's don't set unread if 'S' flag is set on notmuch new[1].

Once python bindings exist for the notmuch shared library, I am sure we
can speed notmuchsync up a lot by keeping the connection open and
tagging all mails in one go rather than executing a separate binary for
each mail. So, this approach might still be feasible.
Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-25 Thread Sebastian Spaeth

On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft madd...@madduck.net wrote:
 I think we all kinda agreed that the Maildir flags should not be
 used by notmuch and that things like Sebastian's notmuchsync should
 be used if people wanted flags represented in Maildir filenames.

While notmuchsync fullfils my needs, it is a kludge. It needs to call
notmuch for each mail where a MailDir flag has changed (which can be
quite often on an initial run, where most mails are likely to be read),
this can take a long, long time. It would makes sense IMHO to at least
pick pioto's don't set unread if 'S' flag is set on notmuch new[1].

Or - at the very least - not to set the unread flag by default.

Sebastian

[1] pioto's noarg-count branch (http://git.pioto.org/gitweb/notmuch.git
Announced in mail id:20100121204201.1c82764...@aether.pioto.org)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-25 Thread Mike Kelly
On Mon, 25 Jan 2010 14:49:00 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 
 On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft madd...@madduck.net 
 wrote:
  I think we all kinda agreed that the Maildir flags should not be
  used by notmuch and that things like Sebastian's notmuchsync should
  be used if people wanted flags represented in Maildir filenames.
 
 While notmuchsync fullfils my needs, it is a kludge. It needs to call
 notmuch for each mail where a MailDir flag has changed (which can be
 quite often on an initial run, where most mails are likely to be read),
 this can take a long, long time. It would makes sense IMHO to at least
 pick pioto's don't set unread if 'S' flag is set on notmuch new[1].

notmuchsync, as currently implemented, suffers from major performance
issues, in my opinion. It's a useful short term workaround, but not a
good long term solution.

But, I personally will always be using both notmuch and some other IMAP
client (my phone). I want the two to remain in sync easily enough.
notmuch is already much more robust with respect to that than sup, I
think (in terms of handling renames without barfing, etc).

At the very least, I want `notmuch new` to be able to:

  If it sees a rename that involves changing maildir flags, alter the
  related tags as necessary.

  Similarly, provide a mechanism for correlating the folder name with
  some set of tags, and change those tags as messages are moved around.  

For example, I might have:

~/.notmuch-config:

[database]
path=/home/pioto/mail
...
[tags]
pi...@pioto.org/INBOX.ListMail.notmuch = notmuch

So, a 'tags' section, where each key is the folder name, relative to the
db path, and the value is one or more tag names

This means that I could relabel a message in gmail, for example, and
have the changes apply to notmuch at my next offlineimap run. And, it
means that my existing procmail rules will still be useful both to
notmuch, and to my phone, for the purpose of categorizing things.

I agree that all this should be optional. But, since it is likely the
behavior most people would expect, I think it should be the default.

PS. You mean the 'new-unread' branch, not the 'noarg-count' branch, from
my repo.

-- 
Mike Kelly
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-25 Thread martin f krafft
also sprach Sebastian Spaeth sebast...@sspaeth.de [2010.01.26.0249 +1300]:
 While notmuchsync fullfils my needs, it is a kludge. It needs to
 call notmuch for each mail where a MailDir flag has changed
 (which can be quite often on an initial run, where most mails are
 likely to be read), this can take a long, long time. It would
 makes sense IMHO to at least pick pioto's don't set unread if 'S'
 flag is set on notmuch new[1].

I am sure this could be implemented with libnotmuch if it proves to
be useful.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
it isn't pollution that's harming the environment.
 it's the impurities in our air and water that are doing it.
  - dan quayle
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-24 Thread martin f krafft
also sprach Asheesh Laroia ashe...@asheesh.org [2010.01.25.1819 +1300]:
 You say Ouch but you should know Dovecot *already* does this. I
 don't mind interoperating with that.

 See http://wiki.dovecot.org/MailboxFormat/Maildir, section Issues
 with the specification, subsection Locking. I term this theQ
 famous readdir() race.

Yikes. IMAP (including dovecot) just SUCKS.

 Without this lock, Maildir is fundamentally incompatible with IMAP
 -- one Maildir-using process modifying message flags could make
 a different Maildir-using process think said message is actually
 deleted. In the case of temporary disappearing mails in Mutt
 locally, that's not the end of the world. For IMAP, it will make
 the IMAP daemon (one of the Maildir-using processes) send a note
 to IMAP clients saying that the message has been deleted and
 expunged.
[…]
 Just don't fall into the trap of thinking Maildir is compatible
 with IMAP. It's not, because as I understand things, the
 filesystem doesn't guarantee that you can actually iterate across
 a directory's files if another process is modifying the list of
 files.

This is all perfect reason to concentrate even more on designing
a store that could potentially make IMAP obsolete once and for all!

The current idea is to sync Git downstream only, and find a way to
keep multiple copies of a tagstore in sync, by way of the server
instance (where mail is received/delivered). Deleting messages
would then be something like setting the notmuch::deleted tag, which
clients would honour; on the server, a cleanup process would run
regularly to actually delete the blobs associated with deleted
messages. This would then propogate the next time one pulls from
Git.

Whether to store history (commit objects) or just collections (tree
objects) needs to be investigated.

 But there are still good reasons why you'd want to have IMAP
 capability too, e.g. Webmail. Given the atomicity problems that
 come from Git, maybe an IMAP server reading from the Git store
 would make sense.
 
 It wouldn't be too hard to write a FUSE filesystem that presented
 an interface to a Git repository that didn't allow the contents of
 files to be modified. Then Dovecot could think it's interacting
 with the filesystem.

Yes, a FUSE layer (which adds a daemon), or a lightweight access
API via libnotmuch. Probably the former using the latter. ;)

 Aww, I like Maildir flags, but if there's a sync tool, I'm fine
 with that.
[…]
 I'm not sure, but maybe it's safe if you refuse to ever modify
 a message's flags in the filename.

The main point is that there is nothing really in Maildir filenames
that you couldn't equally (and possibly better) represent in the
notmuch::* tag namespace, and then there is benefit in only having
one used primarily (which means notmuchsync can do whatever it
wants without affecting or messing with notmuch).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if I can't dance, i don't want to be part of your revolution.
- emma goldman
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch