Re: [notmuch] Notmuch performance (literally, in my case)

2011-07-28 Thread martin f krafft
also sprach martin f krafft madd...@madduck.net [2010.03.16.1900 +0100]:
 I use ext4 with data=ordered, and while notmuch is writing the
 Xapian database, most I/O stalls on the machine:
 
   - Firefox does not get any mouse events
   - Vim blocks writing the viminfo file
   - All disk operations queue for multiple seconds.
 
 So no, ext4 is not a solution. Is it just me, or should no
 filesystem of this world be able to hog a system this badly? I think
 the culprit is the IO-scheduler.

I just wanted to send a little update on this. Even though the Linux I/O
scheduler performs abysmally during the Xapian database updates,
I can report two improvements, at least to my situation:

  1. The 3.0 kernel seems to be better, but I did not quantify this
 in any way, and I might just as well be wrong.

  2. http://bugs.debian.org/635768 explains the (also I/O-related)
 lockups we've seen. Micah offered the tip that the actual fault
 lies with the awesome WM.

Cheers,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
writing a book about debian
 is like hitting a moving target
 with a champagne bottle cork.
 -- arky
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


patchwork

2011-06-30 Thread martin f krafft
Are people using http://patchwork.notmuchmail.org at all? The reason
of my asking is that it receives quite a lot of spam and I do not
need to invest this time for a service noone uses.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"'oh, that was easy,' says Man, and for an encore goes on to prove
 that black is white and gets himself killed on the next zebra
 crossing."
-- douglas adams, "the hitchhiker's guide to the galaxy"

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: digital_signature_gpg.asc
Type: application/pgp-signature
Size: 1124 bytes
Desc: Digital signature (see 
http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
URL: 



patchwork

2011-06-30 Thread martin f krafft
Are people using http://patchwork.notmuchmail.org at all? The reason
of my asking is that it receives quite a lot of spam and I do not
need to invest this time for a service noone uses.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
'oh, that was easy,' says Man, and for an encore goes on to prove
 that black is white and gets himself killed on the next zebra
 crossing.
-- douglas adams, the hitchhiker's guide to the galaxy
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Mail in git

2011-05-21 Thread martin f krafft
also sprach Stewart Smith  [2010.02.17.1107 +0100]:
> On Wed, 17 Feb 2010 11:21:51 +1100, Stewart Smith  flamingspork.com> wrote:
> > Using fast-import is interesting. Does it update the working tree? The
> > big thing I wanted to avoid was creating a working tree (another million
> > inodes being created is not ever what I need)
> > 
> > Also interesting is the mention of creating packs on the fly... this
> > could save the time in first writing the object and then packing it (as
> > my script does).
> > 
> > I'm going to play with this
> 
> and I did.

Has anyone worked on this since?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"one should never allow one's mind
 and one's foot to wander at the same time."
-- edward perkins (yes, the librarian)

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: digital_signature_gpg.asc
Type: application/pgp-signature
Size: 1124 bytes
Desc: Digital signature (see 
http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
URL: 



Re: [notmuch] Mail in git

2011-05-21 Thread martin f krafft
also sprach Stewart Smith stew...@flamingspork.com [2010.02.17.1107 +0100]:
 On Wed, 17 Feb 2010 11:21:51 +1100, Stewart Smith stew...@flamingspork.com 
 wrote:
  Using fast-import is interesting. Does it update the working tree? The
  big thing I wanted to avoid was creating a working tree (another million
  inodes being created is not ever what I need)
  
  Also interesting is the mention of creating packs on the fly... this
  could save the time in first writing the object and then packing it (as
  my script does).
  
  I'm going to play with this
 
 and I did.

Has anyone worked on this since?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
one should never allow one's mind
 and one's foot to wander at the same time.
-- edward perkins (yes, the librarian)
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] Mailstore abstraction v4 - part 2 (maildir synchronization)

2010-04-12 Thread martin f krafft
also sprach Michal Sojka  [2010.04.12.1347 +0200]:
> > Wouldn't it be better to postpone synchronisation of the tags
> > until after emacs is done with the message?
> 
> Theoretically, it would be possible, but if, for some reason, the
> synchronization step would not happen, then the tags in the
> database and in the mailstore will be inconsistent and next run of
> notmuch new would reset the tags according to the state in
> mailstore.

Well, sure. But then again, notmuch (nor IMAP or Maildir) isn't
transactional anyway. There are many other ways in which the
database and store can get out of sync. And you are about to add
another redundancy.

> The current implementation takes tags in mailstore as
> authoritative and ensures that tags in mailstore are always
> updated before tags in the database.

So why store them in the database at all?

> > I understand this might be hard to make work with mailstore
> > abstraction. Wouldn't it make more sense to have emacs call
> > 'notmuch cat', which returns the entire message, removes the
> > unread tag, changes the filename, and updates the database?
> 
> I do not like the fact that cat would do two things - cat and tag.
> And then, 'unread' tag is not the only one which can be changed.

I wouldn't want an unread tag in the first place, especially not
with Maildir semantics. In this case, what should really happen is:

  1. cat feeds a message to client
  2. client instructs notmuch to update tags
 - some tags require changes in the database
 - others require filename changes, which must be completed in
   unison with a database update so the new filename is stored.
  3. user asks to see attachment, which the client can fulfill using
 either a cached copy from (1.) in a tempfile, or by simply
 asking for the message again, via notmuch search.

> > The message returned by cat would be stored in a temporary file
> > for use by the client (emacs). And if the message was needed
> > again, you could just search for it again.
> > 
> > I dislike the idea of heuristically probing a Maildir for files.
> 
> Well, I do not plan to use wired heuristics. At the end there will
> be readdir() to traverse the cur/ directory to find the file with
> the same part before flags. Since the S flag will probably be the
> most frequent change, I may add one single test for added S flag
> before trying more expensive readdir().

What is the point of storing these tags in the Maildir anyway? If
you want to make this information (e.g. new, seen, unread) available
to MUAs accessing Maildir directly, keep in mind that the database
and mailstore will very quickly grow inconsistent until the next
notmuch-new run, e.g. as messages are moved, or flags ('F') are
added in a way that the notmuch database is not updated.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"friendships last when each friend thinks he has
 a slight superiority over the other."
   -- honor? de balzac

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[PATCH] Mailstore abstraction v4 - part 2 (maildir synchronization)

2010-04-12 Thread martin f krafft
also sprach Michal Sojka  [2010.04.08.1713 +0200]:
>I'm working on the solution - if the mailstore cannot open the
>message with the name passed, it tries different names with
>different maildir flags.

Wouldn't it be better to postpone synchronisation of the tags until
after emacs is done with the message?

I understand this might be hard to make work with mailstore
abstraction. Wouldn't it make more sense to have emacs call 'notmuch
cat', which returns the entire message, removes the unread tag,
changes the filename, and updates the database?

The message returned by cat would be stored in a temporary file for
use by the client (emacs). And if the message was needed again, you
could just search for it again.

I dislike the idea of heuristically probing a Maildir for files.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"i don't think so," said rene descartes. just then, he vanished.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [PATCH] Mailstore abstraction v4 - part 2 (maildir synchronization)

2010-04-12 Thread martin f krafft
also sprach Michal Sojka sojk...@fel.cvut.cz [2010.04.08.1713 +0200]:
I'm working on the solution - if the mailstore cannot open the
message with the name passed, it tries different names with
different maildir flags.

Wouldn't it be better to postpone synchronisation of the tags until
after emacs is done with the message?

I understand this might be hard to make work with mailstore
abstraction. Wouldn't it make more sense to have emacs call 'notmuch
cat', which returns the entire message, removes the unread tag,
changes the filename, and updates the database?

The message returned by cat would be stored in a temporary file for
use by the client (emacs). And if the message was needed again, you
could just search for it again.

I dislike the idea of heuristically probing a Maildir for files.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
i don't think so, said rene descartes. just then, he vanished.
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] Mailstore abstraction v4 - part 2 (maildir synchronization)

2010-04-12 Thread martin f krafft
also sprach Michal Sojka sojk...@fel.cvut.cz [2010.04.12.1347 +0200]:
  Wouldn't it be better to postpone synchronisation of the tags
  until after emacs is done with the message?
 
 Theoretically, it would be possible, but if, for some reason, the
 synchronization step would not happen, then the tags in the
 database and in the mailstore will be inconsistent and next run of
 notmuch new would reset the tags according to the state in
 mailstore.

Well, sure. But then again, notmuch (nor IMAP or Maildir) isn't
transactional anyway. There are many other ways in which the
database and store can get out of sync. And you are about to add
another redundancy.

 The current implementation takes tags in mailstore as
 authoritative and ensures that tags in mailstore are always
 updated before tags in the database.

So why store them in the database at all?

  I understand this might be hard to make work with mailstore
  abstraction. Wouldn't it make more sense to have emacs call
  'notmuch cat', which returns the entire message, removes the
  unread tag, changes the filename, and updates the database?
 
 I do not like the fact that cat would do two things - cat and tag.
 And then, 'unread' tag is not the only one which can be changed.

I wouldn't want an unread tag in the first place, especially not
with Maildir semantics. In this case, what should really happen is:

  1. cat feeds a message to client
  2. client instructs notmuch to update tags
 - some tags require changes in the database
 - others require filename changes, which must be completed in
   unison with a database update so the new filename is stored.
  3. user asks to see attachment, which the client can fulfill using
 either a cached copy from (1.) in a tempfile, or by simply
 asking for the message again, via notmuch search.

  The message returned by cat would be stored in a temporary file
  for use by the client (emacs). And if the message was needed
  again, you could just search for it again.
  
  I dislike the idea of heuristically probing a Maildir for files.
 
 Well, I do not plan to use wired heuristics. At the end there will
 be readdir() to traverse the cur/ directory to find the file with
 the same part before flags. Since the S flag will probably be the
 most frequent change, I may add one single test for added S flag
 before trying more expensive readdir().

What is the point of storing these tags in the Maildir anyway? If
you want to make this information (e.g. new, seen, unread) available
to MUAs accessing Maildir directly, keep in mind that the database
and mailstore will very quickly grow inconsistent until the next
notmuch-new run, e.g. as messages are moved, or flags ('F') are
added in a way that the notmuch database is not updated.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
friendships last when each friend thinks he has
 a slight superiority over the other.
   -- honoré de balzac
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Notmuch shared library

2010-04-01 Thread martin f krafft
also sprach martin f krafft  [2010.04.01.1324 +0200]:
> Please avoid rpath. The better solution is probably to create
> a wrapper for notmuch, which prepends to LD_LIBRARY_PATH.

E.g. http://wiki.debian.org/RpathIssue

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

a common mistake that people make
when trying to design something completely foolproof
was to underestimate the ingenuity of complete fools.
 -- douglas adams, "mostly harmless"

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100401/bc9ae064/attachment-0001.pgp>


[notmuch] Notmuch shared library

2010-04-01 Thread martin f krafft
also sprach Michal Sojka  [2010.04.01.1310 +0200]:
> this can be solved by the following patch, but I don't know how portable
> it is. You can see the efect of this by

Please avoid rpath. The better solution is probably to create
a wrapper for notmuch, which prepends to LD_LIBRARY_PATH.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

when everything is coming your way, you're in the wrong lane.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Notmuch shared library

2010-04-01 Thread martin f krafft
also sprach Michal Sojka sojk...@fel.cvut.cz [2010.04.01.1310 +0200]:
 this can be solved by the following patch, but I don't know how portable
 it is. You can see the efect of this by

Please avoid rpath. The better solution is probably to create
a wrapper for notmuch, which prepends to LD_LIBRARY_PATH.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
when everything is coming your way, you're in the wrong lane.
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Notmuch shared library

2010-04-01 Thread martin f krafft
also sprach martin f krafft madd...@madduck.net [2010.04.01.1324 +0200]:
 Please avoid rpath. The better solution is probably to create
 a wrapper for notmuch, which prepends to LD_LIBRARY_PATH.

E.g. http://wiki.debian.org/RpathIssue

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
a common mistake that people make
when trying to design something completely foolproof
was to underestimate the ingenuity of complete fools.
 -- douglas adams, mostly harmless
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Notmuch performance (literally, in my case)

2010-03-16 Thread martin f krafft
also sprach Aneesh Kumar K. V  
[2010.03.16.1810 +0100]:
> Ext3 fsync related issue is a know problem due to the way journalling is
> handled in ext3. The solution for that would be data=writeback ( with
> its loss of data integrity ) or not yet upstreamed data=guarded. Another
> option would be to try ext4 which should not be impacted that badly by
> the data=ordered journalled mode

I use ext4 with data=ordered, and while notmuch is writing the
Xapian database, most I/O stalls on the machine:

  - Firefox does not get any mouse events
  - Vim blocks writing the viminfo file
  - All disk operations queue for multiple seconds.

So no, ext4 is not a solution. Is it just me, or should no
filesystem of this world be able to hog a system this badly? I think
the culprit is the IO-scheduler.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"vulgarity is simply the conduct of other people."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Notmuch performance (literally, in my case)

2010-03-16 Thread martin f krafft
also sprach Aneesh Kumar K. V aneesh.ku...@linux.vnet.ibm.com 
[2010.03.16.1810 +0100]:
 Ext3 fsync related issue is a know problem due to the way journalling is
 handled in ext3. The solution for that would be data=writeback ( with
 its loss of data integrity ) or not yet upstreamed data=guarded. Another
 option would be to try ext4 which should not be impacted that badly by
 the data=ordered journalled mode

I use ext4 with data=ordered, and while notmuch is writing the
Xapian database, most I/O stalls on the machine:

  - Firefox does not get any mouse events
  - Vim blocks writing the viminfo file
  - All disk operations queue for multiple seconds.

So no, ext4 is not a solution. Is it just me, or should no
filesystem of this world be able to hog a system this badly? I think
the culprit is the IO-scheduler.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
vulgarity is simply the conduct of other people.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Debian package

2010-03-05 Thread martin f krafft
also sprach Xavier Maillard  [2010.03.05.0611 +0100]:
> I am using Debian GNU/linux SID and notmuch and I'd like to be
> bleading edge with notmuch. So I am trying to figure out whether
> someone is maintaining a Debian package against notmuch git
> repository or not. If the former, that's nice.

http://packages.debian.org/search?keywords=notmuch

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"ah, but a man's reach should exceed his grasp,
 or what's a heaven for?"
-- robert browning

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Debian package

2010-03-04 Thread martin f krafft
also sprach Xavier Maillard x...@gnu.org [2010.03.05.0611 +0100]:
 I am using Debian GNU/linux SID and notmuch and I'd like to be
 bleading edge with notmuch. So I am trying to figure out whether
 someone is maintaining a Debian package against notmuch git
 repository or not. If the former, that's nice.

http://packages.debian.org/search?keywords=notmuch

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
ah, but a man's reach should exceed his grasp,
 or what's a heaven for?
-- robert browning
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] an other ready-to-use store option for notmuch : CouchDB

2010-03-02 Thread martin f krafft
also sprach Paul R  [2010.03.02.1626 +0100]:
> CouchDB databases can be replicated and synced in both directions.
> Conflicts are lazily handled.

What does this mean?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

fitter, healthier, more productive
like a pig, in a cage, on antibiotics
  -- radiohead

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] [PATCH] Do not segfault on empty mime parts

2010-03-02 Thread martin f. krafft
notmuch previously unconditionally checked mime parts for various
properties, but not for NULL, which is the case if libgmime encounters
an empty mime part.

Upon encounter of an empty mime part, the following is printed to
stderr (the second line due to my patch):

  (process:17197): gmime-CRITICAL **: g_mime_message_get_mime_part: assertion 
`GMIME_IS_MESSAGE (message)' failed
  Warning: Not indexing empty mime part.

This is probably a bug that should get addressed in libgmime, but for
not, my patch is an acceptable workaround.

Signed-off-by: martin f. krafft 
---
 lib/index.cc |5 +
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index cf93025..0d6640b 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -336,6 +336,11 @@ _index_mime_part (notmuch_message_t *message,
 GMimeContentDisposition *disposition;
 char *body;

+if (! part) {
+   fprintf (stderr, "Warning: Not indexing empty mime part.\n");
+   return;
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
-- 
1.7.0



[notmuch] [PATCH] notmuch-reply: Use a shorter 'On, X Y wrote:' line

2010-03-02 Thread martin f krafft
also sprach Sebastian Spaeth  [2010.03.02.1337 +0100]:
> Previously, we would output: 'On Thu, 25 Feb 2010 14:32:54 +0100,
> Sebastian Spaeth  wrote:' now it is: 'On
> 2010-02-25, Sebastian Spaeth wrote:'
> 
> In case we don't find a '<' (as indicator for 'Realname '),
> we still use the whole from address.

This makes me cringe. I don't think replying should be
notmuch-functionality in the first place.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"if builders built buildings the way
 programmers wrote programs,
 then the first woodpecker that came along
 would destroy civilization."
  -- gerald weinberg

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] [PATCH] notmuch-reply: Use a shorter 'On, X Y wrote:' line

2010-03-02 Thread martin f krafft
also sprach Sebastian Spaeth sebast...@sspaeth.de [2010.03.02.1337 +0100]:
 Previously, we would output: 'On Thu, 25 Feb 2010 14:32:54 +0100,
 Sebastian Spaeth sebast...@sspaeth.de wrote:' now it is: 'On
 2010-02-25, Sebastian Spaeth wrote:'
 
 In case we don't find a '' (as indicator for 'Realname email'),
 we still use the whole from address.

This makes me cringe. I don't think replying should be
notmuch-functionality in the first place.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if builders built buildings the way
 programmers wrote programs,
 then the first woodpecker that came along
 would destroy civilization.
  -- gerald weinberg
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] an other ready-to-use store option for notmuch : CouchDB

2010-03-02 Thread martin f krafft
also sprach Paul R paul.r...@gmail.com [2010.03.02.1626 +0100]:
 CouchDB databases can be replicated and synced in both directions.
 Conflicts are lazily handled.

What does this mean?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
fitter, healthier, more productive
like a pig, in a cage, on antibiotics
  -- radiohead
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] [PATCH] Do not segfault on empty mime parts

2010-03-02 Thread martin f. krafft
notmuch previously unconditionally checked mime parts for various
properties, but not for NULL, which is the case if libgmime encounters
an empty mime part.

Upon encounter of an empty mime part, the following is printed to
stderr (the second line due to my patch):

  (process:17197): gmime-CRITICAL **: g_mime_message_get_mime_part: assertion 
`GMIME_IS_MESSAGE (message)' failed
  Warning: Not indexing empty mime part.

This is probably a bug that should get addressed in libgmime, but for
not, my patch is an acceptable workaround.

Signed-off-by: martin f. krafft madd...@madduck.net
---
 lib/index.cc |5 +
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index cf93025..0d6640b 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -336,6 +336,11 @@ _index_mime_part (notmuch_message_t *message,
 GMimeContentDisposition *disposition;
 char *body;
 
+if (! part) {
+   fprintf (stderr, Warning: Not indexing empty mime part.\n);
+   return;
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
-- 
1.7.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] What's so great about notmuch?

2010-02-27 Thread martin f krafft
also sprach Carl Worth  [2010.02.26.2108 +0100]:
>What's your favorite thing about notmuch?

a. that it's an important step forward towards a completely
   tag-based e-mail setup

b. that it is implemented with a library, a UI, and clients on top,
   rather than directly as a GUI. ;)

>What about notmuch makes it distinctive compared to other email
>programs?

the prospect of tags and fast search, accessible in a way to make
integration with other programs possible.

>If someone were to implement a new email system from scratch, but
>capturing the "ideas" of notmuch, what would it have to have?

simplicity and search. Ideally, however, it would be implemented
such that synchronising between different machines were a core
feature.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

if you see an onion ring -- answer it!

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Initial tagging

2010-02-26 Thread martin f krafft
also sprach Jameson Rollins  [2010.02.26.1600 
+0100]:
> > Me too! :-)
> 
> Isn't that what "folders" are?  At least that's what they seem to me.  I
> define a specific search, give it a name, and save it as a folder.

There's a 1:? relationship between folders and messages. I might
want a message associated with multiple contexts.

-- 
 .''`.   martin f. krafft   Related projects:
: :'  :  proud Debian developer   http://debiansystem.info
`. `'`   http://people.debian.org/~madduckhttp://vcs-pkg.org
  `-  Debian - when you have better things to do than fixing systems

"you know you're a hopeless geek when you misspell 'date' as 'data'"
   -- branden robinson
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100226/84366df4/attachment.pgp>


Re: [notmuch] Initial tagging

2010-02-26 Thread martin f krafft
also sprach Jameson Rollins jroll...@finestructure.net [2010.02.26.1600 
+0100]:
  Me too! :-)
 
 Isn't that what folders are?  At least that's what they seem to me.  I
 define a specific search, give it a name, and save it as a folder.

There's a 1:∞ relationship between folders and messages. I might
want a message associated with multiple contexts.

-- 
 .''`.   martin f. krafft madd...@d.o  Related projects:
: :'  :  proud Debian developer   http://debiansystem.info
`. `'`   http://people.debian.org/~madduckhttp://vcs-pkg.org
  `-  Debian - when you have better things to do than fixing systems
 
you know you're a hopeless geek when you misspell 'date' as 'data'
   -- branden robinson


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] patchwork test instance

2010-02-25 Thread martin f krafft
also sprach Carl Worth  [2010.02.24.2008 +0100]:
> I agree that in its current form it's not useful, (it seems to be just a
> long list of the patches that have ever been submitted).
> 
> I seem to recall Martin saying that patches that get accepted will be
> automatically detected and removed from the list. Is that successfully
> happening now?

Yes, I've automatically tagged approx. 140 patches "accepted" so
far.

> In the meantime, if someone does want to push some buttons in
> patchwork, nothing before 2009-12-01 at least is interesting.

Okay, all of those can be tagged superseeded.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

kermit: why are there so many songs about rainbows?
fozzy: that's part of what rainbows do.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] patchwork test instance

2010-02-24 Thread martin f krafft
also sprach Carl Worth  [2010.02.24.2010 +0100]:
> > Carl, would you consider bouncing messages to addresses like
> > patchwork+rejected at patchwork.notmuchmail.org? That would make it
> > trivial for me to write glue to update patchwork automatically.
> 
> Now you're talking my language, Martin!
> 
> I'm disinclined to go to a web browser, find the right buttons and
> push them, but it's easy for me to add an extra address when
> declining a patch, (since I have to write that email message
> already and I can even just add a keybinding to add the extra
> address).

This is trivial to implement, but I don't see myself able to do that
anytime soon. The basic idea is the same as the Git hook[0], and if
someone wanted to take a shot, I'd be happy to create an account on
the machine hosting the patchwork instance.

0. http://lists.ozlabs.org/pipermail/patchwork/2010-February/000224.html

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"most people become bankrupt through having invested too heavily in
 the prose of life. to have ruined one's self over poetry is an
 honour."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] patchwork test instance

2010-02-24 Thread martin f krafft
also sprach Carl Worth cwo...@cworth.org [2010.02.24.2010 +0100]:
  Carl, would you consider bouncing messages to addresses like
  patchwork+rejec...@patchwork.notmuchmail.org? That would make it
  trivial for me to write glue to update patchwork automatically.
 
 Now you're talking my language, Martin!
 
 I'm disinclined to go to a web browser, find the right buttons and
 push them, but it's easy for me to add an extra address when
 declining a patch, (since I have to write that email message
 already and I can even just add a keybinding to add the extra
 address).

This is trivial to implement, but I don't see myself able to do that
anytime soon. The basic idea is the same as the Git hook[0], and if
someone wanted to take a shot, I'd be happy to create an account on
the machine hosting the patchwork instance.

0. http://lists.ozlabs.org/pipermail/patchwork/2010-February/000224.html

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
most people become bankrupt through having invested too heavily in
 the prose of life. to have ruined one's self over poetry is an
 honour.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] notmuch for mutt (was: vim client)

2010-02-21 Thread martin f krafft
also sprach Ruben Pollan  [2010.02.19.2112 +0100]:
> > 1. will there be a usable  ncurses or mutt version that supports
> > notmuch anytime soon?
> 
> I started to work on that I while ago. I didn't hack on it for
> a while, but I hope I'll return to it soon. Anyway to create
> a proper good client is a lot of work, I don't think I'll be able
> to make something usable in months (if I ever make it).

How would mutt support notmuch? Could you elaborate on the way you
plan to integrate them?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"it is easier to be a lover than a husband for the simple reason
 that it is more difficult to be witty every day
 than to say pretty things from time to time."
   -- honor? de balzac

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] notmuch for mutt (was: vim client)

2010-02-20 Thread martin f krafft
also sprach Ruben Pollan mes...@sindominio.net [2010.02.19.2112 +0100]:
  1. will there be a usable  ncurses or mutt version that supports
  notmuch anytime soon?
 
 I started to work on that I while ago. I didn't hack on it for
 a while, but I hope I'll return to it soon. Anyway to create
 a proper good client is a lot of work, I don't think I'll be able
 to make something usable in months (if I ever make it).

How would mutt support notmuch? Could you elaborate on the way you
plan to integrate them?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
it is easier to be a lover than a husband for the simple reason
 that it is more difficult to be witty every day
 than to say pretty things from time to time.
   -- honoré de balzac
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] nested tag trees (was: Mail in git)

2010-02-19 Thread martin f krafft
also sprach Ben Gamari  [2010.02.18.1810 +1300]:
> Yeah, this is a bit of a bummer. This is really a stretch, but I wonder
> if the git folks would accept patches/minor database semantics changes
> in the name of making git more flexible as a general purpose object
> database. I really doubt it, but you never know.

I am pretty sure they won't. Git is a content tracker, not a general
purpose filesystem. It's a bit of a shame.

> > Instead of nested subtrees, think of 16 subtrees forming
> > a level-1 hash table, or 256 for level-2, which really *ought*
> > to be enough.
> > 
> > Anyway, rewriting a tree object is pretty much exactly the same
> > as removing a line (e.g. a message ID) from a file (e.g. a tag),
> > as that file would have to be fully rewritten.
> > 
> This is very true, but exactly do you mean by this statement?

That any form of tag-to-message mapping will be expensive when you
have a million messages referenced. If you used symlinks like mairix
does, any manipulation would require changes to the directory index,
which ? curiously ? functions much like the subtree approach you
proposed.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"the faster i go, the behinder i get."
-- lewis carroll

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git ancestry and sync problems (was: Mail in git)

2010-02-19 Thread martin f krafft
also sprach racin at free.fr  [2010.02.18.2134 +1300]:
> I don't understand the problem. Why not just letting all "inbox"
> mails in a regular Maildir, and use git only when they have been
> explicit archived?

I don't archive my mail. I would like to be able to bring mails from
the past back into circulation at any time, without duplicating
them, or potentially having to discard the metadata when moving them
into a Maildir.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"aus der kriegsschule des lebens -
 was mich nicht umbringt, macht mich h?rter."
 - friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] nested tag trees (was: Mail in git)

2010-02-18 Thread martin f krafft
also sprach Ben Gamari  [2010.02.18.1744 +1300]:
> I believe you would. The problem isn't the messages (well, that's
> a problem too), it's the fact that the tree (e.g. tab) objects
> which reference the messages are immutable (I believe). This
> presents us with the difficult circumstance of being unable to
> modify a tag after it has been created. Therefore, as far as I can
> tell, we need to rewrite the tag's tree object whenever we add or
> remove a message. This was the reason I suggested nesting tag
> trees, although this only partially solves the issue.

You are absolutely right, and I think nesting tag trees is an
interesting idea to pursue. It *would* make it impossible to ever
check out the metatree into the filesystem, or rather result in
subdirectories that the user shouldn't need to worry about.

Instead of nested subtrees, think of 16 subtrees forming a level-1
hash table, or 256 for level-2, which really *ought* to be enough.

Anyway, rewriting a tree object is pretty much exactly the same as
removing a line (e.g. a message ID) from a file (e.g. a tag), as
that file would have to be fully rewritten.

> > This can probably be further optimised, but still: it's not
> > quite as nice as enumerating all parents of a message in O(1)
> > time (which would still result in O(m?n)).
> > 
> Yeah, I'm not sure how well this would scale on truly massive mail
> stores.

The more I think about this, the more I want to implement this
between evenless and Git, i.e. as a porcelain layer, since then
I could also use it for vcs-home[0]. In fact, maybe one day we can
store ~ and mail all in one Git repo, with different porcelains for
different use-cases, and notmuch indexing it all anyway. ;)

0. http://vcs-home.madduck.net

Let's continue the technical discussion on the Git list, okay?

http://marc.info/?l=git=126646636824600=2
id:20100218041240.GA4127 at lapse.rw.madduck.net

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"i hate vulgar realism in literature. the man who could call a spade
 a spade should be compelled to use one. it is the only thing he is
 fit for."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] nested tag trees (was: Mail in git)

2010-02-18 Thread martin f krafft
[Taking a private message back to the list with permission]

also sprach Ben Gamari  [2010.02.18.1620 +1300]:
> This is a very good point. From what I've read about the database
> format, I can't think of any way that reverse dependencies could be
> easily found, unfortunately. If there really is no way to do this, then
> we could have a problem. I'm not sure rewriting tens of megabytes
> everytime you receive a mail message is acceptable.

You would not need to do that, since the messages don't change, and
thus their blobs remain the same.

However, for every manipulation of a message, you would need to
iterate *all* tag trees (O(n)) and update the ones referencing the
message (also O(n)).

The entire process will still be O(n) per message, and O(m?n) for
all:

  messages=[list of messages]
  add_tags=[list of tags to add]
  remove_tags=[list of tags to remove]
  tagtrees=[all tag trees]
  trees_to_update=[]

  for t in remove_tags:
if intersection(t.tree.children, messages):
  T = new_tree(t.name)
  write_tree(T, t.tree.children - messages)
  write_tree(t.tree, [])
  t.tree = T

  for t in add_tags:
t.tree = new_tree(t.name)
rewrite_tree(t.tree, messages)

This can probably be further optimised, but still: it's not quite as
nice as enumerating all parents of a message in O(1) time (which
would still result in O(m?n)).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"... (ethik und ?sthetik sind eins.)"
   -- wittgenstein

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] nested tag trees (was: Mail in git)

2010-02-18 Thread martin f krafft
also sprach martin f krafft  [2010.02.18.1548 +1300]:
> True ? iff we find a way to enumerate trees referencing a given
> blob or tree so that we can walk up the hierarchy. I could look
> right now, but I am about to cross half of the globe tomorrow, so
> I have other things I should rather be doing. Sorry.

http://marc.info/?l=git=126646636824600=2
id:20100218041240.GA4127 at lapse.rw.madduck.net

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"although occasionally there is something to be said for solitude."
  -- special agent dale cooper

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100218/f74dc6f1/attachment.pgp>


[notmuch] nested tag trees (was: Mail in git)

2010-02-18 Thread martin f krafft
also sprach Ben Gamari  [2010.02.18.1519 +1300]:
> > So retagging is really just writing a new tree with a modified list
> > of references.
> > 
> Certainly, however if you have a large tag (>100,000 messages), this
> list of reference could easily be tens of megabytes. For this reason, it
> seems like the added overhead of nesting trees would be well worth it.

True ? iff we find a way to enumerate trees referencing a given
blob or tree so that we can walk up the hierarchy. I could look
right now, but I am about to cross half of the globe tomorrow, so
I have other things I should rather be doing. Sorry.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"men always want to be a woman's first love.
 women have a more subtle instinct:
 what they like is to be a man's last romance."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git ancestry and sync problems (was: Mail in git)

2010-02-18 Thread martin f krafft
also sprach martin f krafft  [2010.02.18.1500 +1300]:
> > > If we keep ancestry though, we are reusing existing working code for
> > > backup (git-pull :)
> > 
> > This is one of the reasons I feel it's important we keep it. And as is
> > stated below, the storage overhead is minimal.
> 
> Absolutely; Stewart mentioned at LCA to forego the porcelain and
> harness the power of the plumbing, and I knew back then that this
> would be among the first things of which to convince him once he had
> the basic idea out. ;)

Except I fear that as soon as we allow manipulation of the local
store, we'll potentially run into this problem:

  http://notmuchmail.org/pipermail/notmuch/2010/001114.html
  id:20100112045152.GA15275 at lapse.rw.madduck.net

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

it's as bad as you think, and they are out to get you.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100218/2414ba0d/attachment-0001.pgp>


[notmuch] Mail in git

2010-02-18 Thread martin f krafft
also sprach Ben Gamari  [2010.02.18.1401 +1300]:
> > If we keep ancestry though, we are reusing existing working code for
> > backup (git-pull :)
> 
> This is one of the reasons I feel it's important we keep it. And as is
> stated below, the storage overhead is minimal.

Absolutely; Stewart mentioned at LCA to forego the porcelain and
harness the power of the plumbing, and I knew back then that this
would be among the first things of which to convince him once he had
the basic idea out. ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

DISCLAIMER: this entire message is privileged communication, intended
for the sole use of its recipients only. If you read it even though
you know you aren't supposed to, you're a poopy-head.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Mail in git

2010-02-18 Thread martin f krafft
also sprach Ben Gamari  [2010.02.18.1339 +1300]:
> Yes, it would be linear in number of tags. I suppose if messages
> weren't stored in the top-level tree nodes, then it would still be
> linear, although with a slope equal to the reciprocal of the fan-out.
> This has the potential to be very reasonable performance-wise.

Messages are never stored in tree nodes; all these do are store
references to objects (blobs) holding messages. I bet you know this,
but I just wanted to make it explicit.

So retagging is really just writing a new tree with a modified list
of references.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"no survivors? then where do the stories come from I wonder?"
   -- captain jack sparrow

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Mail in git

2010-02-18 Thread martin f krafft
also sprach Ben Gamari  [2010.02.18.0834 +1300]:
> Excerpts from Mark Anderson's message of Wed Feb 17 14:23:48 -0500
> 2010:
> > But if we have notmuch as a cache of the tags, then don't we
> > already know the tree objects that need updating?  Yes, we would
> > probably need some consistency checks for when things don't work
> > as planned, but in the common case we ought to always know.
> > 
> Cached or not, rewriting would still be an incredibly (e.g.
> prohibitively or close to it) expensive operation for a large
> mailstore.

Why? Well, would involve creating n objects and unlinking n objects
for n tags, but it would be constant in the number of messages, no?

> > Perhaps I'm misunderstanding these tree objects, and you're
> > suggesting that we don't even tell notmuch about them.
> > 
> I think it would be unwise to teach notmuch anything about the
> underlying store. That would be leaking way too many
> implementation details into

I agree. Also, it would introduce redundancy.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"twenty-four hour room-service must be one of the
 premiere achievements of modern civilization."
  -- special agent dale cooper

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Git ancestry and sync problems (was: Mail in git)

2010-02-18 Thread martin f krafft
also sprach ra...@free.fr ra...@free.fr [2010.02.18.2134 +1300]:
 I don't understand the problem. Why not just letting all inbox
 mails in a regular Maildir, and use git only when they have been
 explicit archived?

I don't archive my mail. I would like to be able to bring mails from
the past back into circulation at any time, without duplicating
them, or potentially having to discard the metadata when moving them
into a Maildir.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
aus der kriegsschule des lebens -
 was mich nicht umbringt, macht mich härter.
 - friedrich nietzsche
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Mail in git

2010-02-17 Thread martin f krafft
also sprach Stewart Smith  [2010.02.15.1329 +1300]:
> What about adding more mail to the archive?
> 
> So the way I think is that you use a Maildir for day to day mail
> (e.g. delivery) and every so often you run some magic command that
> takes old mail out of the Maildir and stores it in the git repo.

Either that, or the other idea we had (which I prefer), which would
basically be:

  evenless-submit ? add a new mail (and return a hash ID)
and invoke a hook, e.g. to let notmuch know
  evenless-cat? print the full mail given ID with headers to stdout
  evenless-delete ? unlink a mail identified by hash ID
and invoke a hook, e.g. to let notmuch know

If we expose the submit and delete functionality at the notmuch
level, then we don't need the hooks for then evenless would be
plumbing.

Anything to avoid a cronjob would be good, I think.

Then we need a notmuch backend for mutt etc.. For those who still
want to use a regular Maildir, let them use the worktree.

What I am wondering is if (explicit) tags couldn't be represented as
tree-objects with this.

  evenless-link   ? link a message object with a tree object
  evenless?unlink ? unlink a message object from tree object
[replaces evenless-unlink]

messages would then be deleted whenever using git-gc.

No idea how this would sync if we don't keep ancestry. Otoh, it
would probably not be very expensive to do just that.

notmuch would then only search and provide the hash ID(s); tags
would be a function of storage.

Is it possible to find out all trees that reference a given object
with Git in constant or sub-linear time?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"the question of whether computers can think
 is like the question of whether submarines can swim."
 -- edsgar w. dijkstra

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Mail in git

2010-02-17 Thread martin f krafft
also sprach Ben Gamari bgam...@gmail.com [2010.02.18.0834 +1300]:
 Excerpts from Mark Anderson's message of Wed Feb 17 14:23:48 -0500
 2010:
  But if we have notmuch as a cache of the tags, then don't we
  already know the tree objects that need updating?  Yes, we would
  probably need some consistency checks for when things don't work
  as planned, but in the common case we ought to always know.
  
 Cached or not, rewriting would still be an incredibly (e.g.
 prohibitively or close to it) expensive operation for a large
 mailstore.

Why? Well, would involve creating n objects and unlinking n objects
for n tags, but it would be constant in the number of messages, no?

  Perhaps I'm misunderstanding these tree objects, and you're
  suggesting that we don't even tell notmuch about them.
  
 I think it would be unwise to teach notmuch anything about the
 underlying store. That would be leaking way too many
 implementation details into

I agree. Also, it would introduce redundancy.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
twenty-four hour room-service must be one of the
 premiere achievements of modern civilization.
  -- special agent dale cooper
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Mail in git

2010-02-17 Thread martin f krafft
also sprach Ben Gamari bgam...@gmail.com [2010.02.18.1401 +1300]:
  If we keep ancestry though, we are reusing existing working code for
  backup (git-pull :)
 
 This is one of the reasons I feel it's important we keep it. And as is
 stated below, the storage overhead is minimal.

Absolutely; Stewart mentioned at LCA to forego the porcelain and
harness the power of the plumbing, and I knew back then that this
would be among the first things of which to convince him once he had
the basic idea out. ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
DISCLAIMER: this entire message is privileged communication, intended
for the sole use of its recipients only. If you read it even though
you know you aren't supposed to, you're a poopy-head.
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] nested tag trees (was: Mail in git)

2010-02-17 Thread martin f krafft
[Taking a private message back to the list with permission]

also sprach Ben Gamari bgam...@gmail.com [2010.02.18.1620 +1300]:
 This is a very good point. From what I've read about the database
 format, I can't think of any way that reverse dependencies could be
 easily found, unfortunately. If there really is no way to do this, then
 we could have a problem. I'm not sure rewriting tens of megabytes
 everytime you receive a mail message is acceptable.

You would not need to do that, since the messages don't change, and
thus their blobs remain the same.

However, for every manipulation of a message, you would need to
iterate *all* tag trees (O(n)) and update the ones referencing the
message (also O(n)).

The entire process will still be O(n) per message, and O(m×n) for
all:

  messages=[list of messages]
  add_tags=[list of tags to add]
  remove_tags=[list of tags to remove]
  tagtrees=[all tag trees]
  trees_to_update=[]

  for t in remove_tags:
if intersection(t.tree.children, messages):
  T = new_tree(t.name)
  write_tree(T, t.tree.children - messages)
  write_tree(t.tree, [])
  t.tree = T

  for t in add_tags:
t.tree = new_tree(t.name)
rewrite_tree(t.tree, messages)

This can probably be further optimised, but still: it's not quite as
nice as enumerating all parents of a message in O(1) time (which
would still result in O(m×n)).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
... (ethik und ästhetik sind eins.)
   -- wittgenstein
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] nested tag trees (was: Mail in git)

2010-02-17 Thread martin f krafft
also sprach Ben Gamari bgam...@gmail.com [2010.02.18.1744 +1300]:
 I believe you would. The problem isn't the messages (well, that's
 a problem too), it's the fact that the tree (e.g. tab) objects
 which reference the messages are immutable (I believe). This
 presents us with the difficult circumstance of being unable to
 modify a tag after it has been created. Therefore, as far as I can
 tell, we need to rewrite the tag's tree object whenever we add or
 remove a message. This was the reason I suggested nesting tag
 trees, although this only partially solves the issue.

You are absolutely right, and I think nesting tag trees is an
interesting idea to pursue. It *would* make it impossible to ever
check out the metatree into the filesystem, or rather result in
subdirectories that the user shouldn't need to worry about.

Instead of nested subtrees, think of 16 subtrees forming a level-1
hash table, or 256 for level-2, which really *ought* to be enough.

Anyway, rewriting a tree object is pretty much exactly the same as
removing a line (e.g. a message ID) from a file (e.g. a tag), as
that file would have to be fully rewritten.

  This can probably be further optimised, but still: it's not
  quite as nice as enumerating all parents of a message in O(1)
  time (which would still result in O(m×n)).
  
 Yeah, I'm not sure how well this would scale on truly massive mail
 stores.

The more I think about this, the more I want to implement this
between evenless and Git, i.e. as a porcelain layer, since then
I could also use it for vcs-home[0]. In fact, maybe one day we can
store ~ and mail all in one Git repo, with different porcelains for
different use-cases, and notmuch indexing it all anyway. ;)

0. http://vcs-home.madduck.net

Let's continue the technical discussion on the Git list, okay?

http://marc.info/?l=gitm=126646636824600w=2
id:20100218041240.ga4...@lapse.rw.madduck.net

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
i hate vulgar realism in literature. the man who could call a spade
 a spade should be compelled to use one. it is the only thing he is
 fit for.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] [PATCH] notmuch: Respect maildir message flags

2010-02-16 Thread martin f krafft
also sprach Stewart Smith  [2010.02.16.1521 +1300]:
> What about putting them all in there except for the seen tag, with
> the seen tag dictating if it gets marked 'unread' or not? I cannot
> imagine where somebody would want this not to be the case... it
> was bad enough discovering 100,000 unread messages :)

Well, I do keep messages unread even after I saw them, until I can
actually read them...

> What about this patch (just with those few things fixed)?

Thanks!

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"i dislike arguments of any kind. they are always vulgar, and often
 convincing."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] [PATCH] notmuch: Respect maildir message flags

2010-02-16 Thread martin f krafft
also sprach Stewart Smith  [2010.02.16.1458 +1300]:
> + case 'R': /* replied */
> + notmuch_message_add_tag (message, "answered");
> + break;

'r' means replied, not 'answered'.

> + case 'T': /* trashed */
> + notmuch_message_add_tag (message, "deleted");
> + break;

Same. trashed and deleted are not the same thing.

I don't want to get into an argument over this, because I think this
already exposes a problem: you are putting into global namespace
something not everyone might want, or agree with.

Why not use 'maildirflags::replied' instead? People can always map
that to something in the global namespace.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"geld ist das brecheisen der macht."
 - friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Mail in git

2010-02-16 Thread martin f krafft
also sprach Stewart Smith stew...@flamingspork.com [2010.02.15.1329 +1300]:
 What about adding more mail to the archive?
 
 So the way I think is that you use a Maildir for day to day mail
 (e.g. delivery) and every so often you run some magic command that
 takes old mail out of the Maildir and stores it in the git repo.

Either that, or the other idea we had (which I prefer), which would
basically be:

  evenless-submit — add a new mail (and return a hash ID)
and invoke a hook, e.g. to let notmuch know
  evenless-cat— print the full mail given ID with headers to stdout
  evenless-delete — unlink a mail identified by hash ID
and invoke a hook, e.g. to let notmuch know

If we expose the submit and delete functionality at the notmuch
level, then we don't need the hooks for then evenless would be
plumbing.

Anything to avoid a cronjob would be good, I think.

Then we need a notmuch backend for mutt etc.. For those who still
want to use a regular Maildir, let them use the worktree.

What I am wondering is if (explicit) tags couldn't be represented as
tree-objects with this.

  evenless-link   — link a message object with a tree object
  evenless–unlink – unlink a message object from tree object
[replaces evenless-unlink]

messages would then be deleted whenever using git-gc.

No idea how this would sync if we don't keep ancestry. Otoh, it
would probably not be very expensive to do just that.

notmuch would then only search and provide the hash ID(s); tags
would be a function of storage.

Is it possible to find out all trees that reference a given object
with Git in constant or sub-linear time?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
the question of whether computers can think
 is like the question of whether submarines can swim.
 -- edsgar w. dijkstra
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] notmuch: Respect maildir message flags

2010-02-15 Thread martin f krafft
also sprach Stewart Smith stew...@flamingspork.com [2010.02.16.1458 +1300]:
 + case 'R': /* replied */
 + notmuch_message_add_tag (message, answered);
 + break;

'r' means replied, not 'answered'.

 + case 'T': /* trashed */
 + notmuch_message_add_tag (message, deleted);
 + break;

Same. trashed and deleted are not the same thing.

I don't want to get into an argument over this, because I think this
already exposes a problem: you are putting into global namespace
something not everyone might want, or agree with.

Why not use 'maildirflags::replied' instead? People can always map
that to something in the global namespace.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
geld ist das brecheisen der macht.
 - friedrich nietzsche
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] [PATCH] notmuch: Respect maildir message flags

2010-02-15 Thread martin f krafft
also sprach Stewart Smith stew...@flamingspork.com [2010.02.16.1521 +1300]:
 What about putting them all in there except for the seen tag, with
 the seen tag dictating if it gets marked 'unread' or not? I cannot
 imagine where somebody would want this not to be the case... it
 was bad enough discovering 100,000 unread messages :)

Well, I do keep messages unread even after I saw them, until I can
actually read them...

 What about this patch (just with those few things fixed)?

Thanks!

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
i dislike arguments of any kind. they are always vulgar, and often
 convincing.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] notmuch reply template

2010-02-12 Thread martin f krafft
also sprach Sebastian Spaeth  [2010.02.12.0254 +1300]:
> I've had enough of the 
> 
>  On a sunny Sunday the 30th in timezone , male human  with mail
>  address  hit the reply button  microseconds after 1970 while being 
> at latitude :
> 
> type of reply template in notmuch. :-)
> 
> May I suggest to use a simpler:
> 
>  "On 11 Feb 2010, David Edmondson wrote:" 

This should be configurable so everyone can set their own.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

if god had meant for us to be naked,
we would have been born that way.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] patchwork test instance

2010-02-11 Thread martin f krafft
also sprach Sebastian Spaeth  [2010.02.10.2225 +1300]:
> "notmuch dump tag:notmuch and tag:patch" could be used to construct a
> list of "accepted", "willnottakeit","superseded" and "pending" lists or
> whatever. If that list were made accessible somewhere, this would be
> super useful. At least it would help me see whether Carl just hasn't
> gotten around to including my "press 'd' for delete" patch or whether he
> is not interested in merging it. :)

Carl, would you consider bouncing messages to addresses like
patchwork+rejected at patchwork.notmuchmail.org? That would make it
trivial for me to write glue to update patchwork automatically.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"es ist gut, eine sache doppelt auszudr?cken und ihr einen
 rechten und linken fu? zu geben. auf einem bein kann die wahrheit
 zwar stehen; mit zweien aber wird sie gehen und herumkommen."
-- friedrich nietzsche

spamtraps: madduck.bogus at madduck.net


[notmuch] patchwork test instance

2010-02-11 Thread martin f krafft
also sprach David Bremner  [2010.02.10.2149 +1300]:
> I'm not sure what merging patches means here; some kind of squash
> operation?  Anyway it seemed to me that every every patch series
> that I looked at was broken into individual patches. Maybe I am
> just unlucky, or does patchwork really not understand the concept
> of series of patches in a thread?

I don't think it does, this is what bundles are for, but these need
to be created manually at the moment. Patch series are pretty easy
to detect, so maybe the bundle could be automatically generated.

http://lists.ozlabs.org/pipermail/patchwork/2010-February/000226.html

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

because light travels faster than sound,
some people appear to be intelligent,
until you hear them speak.

spamtraps: madduck.bogus at madduck.net


Re: [notmuch] notmuch reply template

2010-02-11 Thread martin f krafft
also sprach Sebastian Spaeth sebast...@sspaeth.de [2010.02.12.0254 +1300]:
 I've had enough of the 
 
  On a sunny Sunday the 30th in timezone wurgs, male human X with mail
  address z hit the reply button bar microseconds after 1970 while being 
 at latitude foo:
 
 type of reply template in notmuch. :-)
 
 May I suggest to use a simpler:
 
  On 11 Feb 2010, David Edmondson wrote: 

This should be configurable so everyone can set their own.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if god had meant for us to be naked,
we would have been born that way.
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] patchwork test instance

2010-02-10 Thread martin f krafft
also sprach martin f krafft  [2010.02.02.1131 +1300]:
> I investigated some patch/issue trackers over the weekend. Here's my
> summary/reply.
> 
> The executive summary is that
> http://patchwork.madduck.net/project/notmuch/list/ now exists.
> I have not really used it for anything real, so if some of you feel
> inclined to give it a shot, sign up and triage away! Feedback
> welcome.

Are people actually using it? I know that merging patches is
impossible, and that sucks, but otherwise: is this something to keep
around, or should I take the site offline again?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"time flies like an arrow. fruit flies like a banana."
   -- groucho marx

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100210/d5dc4c1c/attachment.pgp>


Re: [notmuch] patchwork test instance

2010-02-10 Thread martin f krafft
also sprach David Bremner brem...@unb.ca [2010.02.10.2149 +1300]:
 I'm not sure what merging patches means here; some kind of squash
 operation?  Anyway it seemed to me that every every patch series
 that I looked at was broken into individual patches. Maybe I am
 just unlucky, or does patchwork really not understand the concept
 of series of patches in a thread?

I don't think it does, this is what bundles are for, but these need
to be created manually at the moment. Patch series are pretty easy
to detect, so maybe the bundle could be automatically generated.

http://lists.ozlabs.org/pipermail/patchwork/2010-February/000226.html

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
because light travels faster than sound,
some people appear to be intelligent,
until you hear them speak.
 
spamtraps: madduck.bo...@madduck.net
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] patchwork test instance

2010-02-09 Thread martin f krafft
also sprach martin f krafft madd...@madduck.net [2010.02.02.1131 +1300]:
 I investigated some patch/issue trackers over the weekend. Here's my
 summary/reply.
 
 The executive summary is that
 http://patchwork.madduck.net/project/notmuch/list/ now exists.
 I have not really used it for anything real, so if some of you feel
 inclined to give it a shot, sign up and triage away! Feedback
 welcome.

Are people actually using it? I know that merging patches is
impossible, and that sucks, but otherwise: is this something to keep
around, or should I take the site offline again?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
time flies like an arrow. fruit flies like a banana.
   -- groucho marx
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] hack to retag a directory

2010-02-07 Thread martin f krafft
also sprach David Bremner  [2010.02.07.1238 +1300]:
> I have to say that merge conflicts are not very much fun. I tend
> to do a certain amount of oh, take all the changes from the
> server.  I wonder if the approach that someone else mentioned of
> keeping a file tags/message-id with the appropriate tags in it
> might make merging less painful.

Merge conflicts at the file level are even less fun. This reminds me
of my plan to introduce .gitignore.d/*:

http://kerneltrap.org/mailarchive/git/2007/10/1/326425

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"there was silence for a moment, and then out of the scrambled mess
 of arthur's brain crawled some words."
 -- hitchhiker's guide to the galaxy

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-07 Thread martin f krafft
also sprach Carl Worth  [2010.02.07.1156 +1300]:
> > Sorry to burden you, but since you want to continue maintaining
> > the infrastructure, that's just what I have to do. ;)
> 
> No burden at all. I don't really care to do everything alone.
> I just want to ensure we keep things centralized enough that
> people can actually find things easily.

This is what DNS was designed for, right? ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

to err is human - to moo, bovine

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-06 Thread martin f krafft
also sprach Carl Worth cwo...@cworth.org [2010.02.07.1156 +1300]:
  Sorry to burden you, but since you want to continue maintaining
  the infrastructure, that's just what I have to do. ;)
 
 No burden at all. I don't really care to do everything alone.
 I just want to ensure we keep things centralized enough that
 people can actually find things easily.

This is what DNS was designed for, right? ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
to err is human - to moo, bovine
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] patchwork now auto-updates patches from Git (was: Git feature branch)

2010-02-05 Thread martin f krafft
also sprach martin f krafft  [2010.02.04.1650 +1300]:
> - ? which brings me to my second point: there are certain things
>   that patchwork can do, at least in theory:
> 
>   * mark patches accepted when they hit your (canonical) master
> branch
>   * mark patches rfc when they hit e.g. my (canonical) next branch
>   * mark patches "under review" when they hit the all-patches (or
> pu) branch.
> 
>   I have not yet tried any of these, and I am basing this theory
>   only on the idea that git-patch-id can come to the rescue, for
>   there is no other linkage between the patch on the mailing list
>   (and thus known to patchwork), and the commit in the repo.

Patchwork now marks patches Accepted once they hit Carl's master
branch (up to 10 minutes delay due to Cron). It uses an algorithm
similar (but not equal) to git-patch-id in that it hashes the diff.
This means that the commit message can be amended when patches are
applied/cherry-picked, but the patch itself must be verbatim.

I ran it on all history thus far and it found 99 patches.

It'll be trivial to set it up to mark other states when the
corresponding commits hit another branch.

Let me know if there are any problems, and feedback welcome.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"if they can get you asking the wrong questions,
 they don't have to worry about answers."
 -- thomas pynchon

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100205/a05b53e1/attachment.pgp>


[notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-04 Thread martin f krafft
also sprach Servilio Afre Puentes  [2010.02.04.1918 
+1300]:
> > If there is indeed an RSS/Atom feed on gitweb that includes the
> > diffs inline, please give me the URL; I can't fine ond.
> 
> Can't see that even in the code.

Carl, could you set up notmuch-commits at notmuchmail.org and enable
the commit and enable the mail hook as per

  http://notmuchmail.org/pipermail/notmuch/2010/001373.html

Sorry to burden you, but since you want to continue maintaining the
infrastructure, that's just what I have to do. ;)

Thanks,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

beware of bugs in the above code;
i have only proved it correct, not tried it.
-- donald e. knuth

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-04 Thread martin f krafft
also sprach Servilio Afre Puentes  [2010.02.04.1714 
+1300]:
> In the second link, the links with the text "commitdiff" provide
> it, and you have the Atom and RSS feeds at the bottom.

As I see it, the feeds have links to the commitdiffs, but that's not
quite the same as having the diffs inline. You might be offline
without having pulled before, then the items from RSS/Atom are
useless.

If there is indeed an RSS/Atom feed on gitweb that includes the
diffs inline, please give me the URL; I can't fine ond.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"es ist immer etwas wahnsinn in der liebe.
 es ist aber auch immer etwas vernunft im wahnsinn."
 - friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git feature branch

2010-02-04 Thread martin f krafft
also sprach Carl Worth  [2010.02.04.1605 +1300]:
> >   maint/? the stable release
> >   master/   ? the stablising head
> >   next/ ? testing branch
> >   pu/   ? patch integration branch (proposed updates)
> 
> I'm not a fan of this scheme, (or maybe I've just never quite understood
> it).
> 
> When I first encountered this scheme, (when first using git), it was
> never clear to me what branch I should actually run or test as a
> user. Nor was it obvious which branches would be regularly rebased or
> not---nor which branches had features being discussed on the mailing
> list.

I'll happily explain if you want. I think that this workflow, in
combination with the distributed functionality of Git, makes for
really powerful collaboration.

To answer your two (implicit) questions directly:

- as a user, you'd probably be tracking master, if you aren't
  running a stable release anyway. if you are ready to deal with the
  occasional bug and want some juicy stuff, go with next. if you are
  a developer ready to fend of the sh*t as it hits the fan, use pu.

  Basically, it's a bit like Debian testing/unstable/experiemental,
  except that the default entry point for new patches (packages in
  that analogy) is pu (experimental), not next (unstable, as it is
  in Debian).

  maint is then the stable branch, see below.

- nothing is ever rebased. patches are cherry-picked downwards
  (pu?next?master) and branches are occasionally merged upwards
  (master into next, next into pu).

I don't think we will need 'next' anytime soon, not until we have
a situation where we are e.g. working on the next 1.x release by
already have 2.0 on the horizon.

The important distinction is between pu (proposed-updates) and
master. The goal for master should always be that it's usable.
Features that are too new to ensure that go to pu until they
matured.

> Instead of "maint" I'd much rather just have branches that are
> named directly after the stable releases being maintained. We've
> done this with the cairo repository with branch names like "1.2",
> "1.4", "1.6", etc. That way it's very clear what the branch
> represents and it's possible to have multiple concurrent "live"
> maintenance branches. But of course, until we actually have
> releases, this doesn't really matter. :-)

This is all possible to do. It has no impact on the pu/next/master
distinction. Basically, once a release is made, master is merged
into maint (I think) and tagged. If a maintenance release has to be
made, a maint/1.0 branch is created. I don't think Git/Linux do
that, but I do.

> I want to maintain a branch myself, (where I'm the only person
> pushing to the branch). [This is different than what I've done
> with the cairo repository where we have all core maintainer's
> pushing to a central repository. I'm intentionally trying
> something new here.]

Do you want to maintain that branch yourself for your own purposes,
or do you want to be the sole maintainer of the branch that is
advertised as *the* notmuch branch, and from which future releases
are made?

> Obviously, that branch that I maintain is currently called
> "master", but I wouldn't mind (and might actually prefer) to have
> it be called "~cworth" or so. Though we have the problem that we
> need "master" to point to *something*.

Actually, there is no reason for master to exist at all. ;)
It's just the default, but it's not intrinsic.

> Beyond that, I'm quite happy to have any number of branches similarly
> maintained by any other individuals. I want to get things setup so that
> those will be hosted and listed alongside my branch on
> notmuchmail.org.

So you are talking repos, not branches. I know they are almost the
same, but branches live in repos, and if you want to be the only
person pushing to a branch, you need to be the only one with write
access to the containing repo ? unless you use gitolite, which can
do in-repo per-branch access control.

> > What patch tracking workflow should we adopt? Keep sending
> > patches to the mailing list
> 
> I definitely like the patches on the list. I find that with
> notmuch, I can maintain a queue of outstanding patches very
> effectively, (meaning, that the queue is usable and doesn't forget
> even if I do get very far behind like I am currently).

The reason why I set up patchwork is because you might have spent
hours triaging some patches already, e.g. bringing the count from
400 to 100. Since I cannot really "pull" your tags, I am forced to
go through the entire list myself.

> > master or pu (or even maint/next), as appropriate? Use the
> > Debian bug tracker? Use Sebastian's Roundup instance? Set up
> > a patch queue manager on notmuchmail.org? Use patchwork [1]?
>
> I'm personally not interested in any system that requires me to
> push any additional buttons outside of notmuch and git itself.
> I am interested in tools that can generate reports and help
> provide visibility into things. So patchwork fixed to
> automatically notice 

[notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-04 Thread martin f krafft
also sprach Marten Veldthuis  [2010.02.04.0353 +1300]:
> > Like this? http://notmuchmail.org/recentchanges/
> 
> No, more like this: http://git.notmuchmail.org/git/notmuch-wiki

To my knowledge, neither of these two give you diffs. This is what
a post-receive hook offers.

It's trivial to do with Git. Assuming a recent Git-on-Debian, it's
just (assuming you are in the bare repository)

  test -d refs || echo not in repo
  sed -e 's,^#\.,,' hooks/post-receive.sample > hooks/post-receive
  chmod 755 !$
  git config hooks.mailinglist notmuch at notmuchmail.org

One might also have to send hooks.envelopesender and
hooks.emailprefix (without trailing space) could be nice.

PS: speaking of prefixes, how about remving the subject prefix of
this list in general? ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"arguments are extremely vulgar,
 for everyone in good society
 holds exactly the same opinion."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] patchwork now auto-updates patches from Git (was: Git feature branch)

2010-02-04 Thread martin f krafft
also sprach martin f krafft madd...@madduck.net [2010.02.04.1650 +1300]:
 - … which brings me to my second point: there are certain things
   that patchwork can do, at least in theory:
 
   * mark patches accepted when they hit your (canonical) master
 branch
   * mark patches rfc when they hit e.g. my (canonical) next branch
   * mark patches under review when they hit the all-patches (or
 pu) branch.
 
   I have not yet tried any of these, and I am basing this theory
   only on the idea that git-patch-id can come to the rescue, for
   there is no other linkage between the patch on the mailing list
   (and thus known to patchwork), and the commit in the repo.

Patchwork now marks patches Accepted once they hit Carl's master
branch (up to 10 minutes delay due to Cron). It uses an algorithm
similar (but not equal) to git-patch-id in that it hashes the diff.
This means that the commit message can be amended when patches are
applied/cherry-picked, but the patch itself must be verbatim.

I ran it on all history thus far and it found 99 patches.

It'll be trivial to set it up to mark other states when the
corresponding commits hit another branch.

Let me know if there are any problems, and feedback welcome.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if they can get you asking the wrong questions,
 they don't have to worry about answers.
 -- thomas pynchon
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-03 Thread martin f krafft
also sprach Servilio Afre Puentes servi...@gmail.com [2010.02.04.1714 +1300]:
 In the second link, the links with the text commitdiff provide
 it, and you have the Atom and RSS feeds at the bottom.

As I see it, the feeds have links to the commitdiffs, but that's not
quite the same as having the diffs inline. You might be offline
without having pulled before, then the items from RSS/Atom are
useless.

If there is indeed an RSS/Atom feed on gitweb that includes the
diffs inline, please give me the URL; I can't fine ond.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
es ist immer etwas wahnsinn in der liebe.
 es ist aber auch immer etwas vernunft im wahnsinn.
 - friedrich nietzsche
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git commit mails (was: New wiki instance on the notmuchmail.org website)

2010-02-03 Thread martin f krafft
also sprach Servilio Afre Puentes servi...@gmail.com [2010.02.04.1918 +1300]:
  If there is indeed an RSS/Atom feed on gitweb that includes the
  diffs inline, please give me the URL; I can't fine ond.
 
 Can't see that even in the code.

Carl, could you set up notmuch-comm...@notmuchmail.org and enable
the commit and enable the mail hook as per

  http://notmuchmail.org/pipermail/notmuch/2010/001373.html

Sorry to burden you, but since you want to continue maintaining the
infrastructure, that's just what I have to do. ;)

Thanks,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
beware of bugs in the above code;
i have only proved it correct, not tried it.
-- donald e. knuth
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] [PATCH] Enforce foldmethod=manual when showing messages in vim

2010-02-02 Thread martin f. krafft
Vim's NotMuch mode relies on manual markers when rendering/showing
a message. If foldmethod is set to something else (marker in my case) by
default, then there are numerous errors, and folds don't work. Hence,
set foldmethod=manual for the local buffer upon showing a message.

Signed-off-by: martin f. krafft 
---
 vim/plugin/notmuch.vim |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/vim/plugin/notmuch.vim b/vim/plugin/notmuch.vim
index a226f20..2f9b05c 100644
--- a/vim/plugin/notmuch.vim
+++ b/vim/plugin/notmuch.vim
@@ -421,6 +421,7 @@ function! s:NM_cmd_show(words)
 let b:nm_raw_info = info
 let b:nm_prev_bufnr = prev_bufnr

+setlocal foldmethod=manual
 call NM_cmd_show_mkfolds()
 call NM_cmd_show_mksyntax()
 call NM_set_map('n', g:notmuch_show_maps)
-- 
1.6.6




[notmuch] Request for high-priority improvements to notmuch

2010-02-02 Thread martin f krafft
also sprach Scott Robinson  [2010.02.02.1043 +1300]:
> > YES PLEASE :-). notmuch seems designed to work in an ecosystem of
> > surrounding scripts, feeding data in and out. But we are all currently
> > limited to regexes for that. And heck, I hard a hard time understanding
> > why all hell broke out until I found that i had added a tag containing
> > parentheses which made my regex fail. :-). XML, JSON, any structured
> > output would be nice.
> > 
> > And as for filtering: YES, PLEASE :-). notmuchsync and many other 3rd
> > party apps would love that. As father of notmuchsync, I can tell you my
> > little script hickups very badly when slurping in 200k mails (including
> > text bodies) just to find out the maildir tags of those mails.
> > 
> 
> There's been a JSON patch waiting for a month now.

The last month has been busy for everyone, mainly due to LCA. We
should now all work together to help Carl go through the patch
queue. Maybe http://patchwork.madduck.net can help.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

beauty, brains, availability, personality; pick any two.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Request for high-priority improvements to notmuch

2010-02-02 Thread martin f krafft
also sprach sebastian at sspaeth.de  [2010.02.02.0929 
+1300]:
> YES PLEASE :-). notmuch seems designed to work in an ecosystem of
> surrounding scripts, feeding data in and out. But we are all currently
> limited to regexes for that. And heck, I hard a hard time understanding
> why all hell broke out until I found that i had added a tag containing
> parentheses which made my regex fail. :-). XML, JSON, any structured
> output would be nice.

Shoot me, but I'd say mbox output would be nice too.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"time flies like an arrow. fruit flies like a banana."
   -- groucho marx

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] patchwork test instance (was: Git feature branch)

2010-02-02 Thread martin f krafft
I investigated some patch/issue trackers over the weekend. Here's my
summary/reply.

The executive summary is that
http://patchwork.madduck.net/project/notmuch/list/ now exists.
I have not really used it for anything real, so if some of you feel
inclined to give it a shot, sign up and triage away! Feedback
welcome.

also sprach James Rowe  [2010.01.28.2005 +1300]:
>   Roundup has command line and email interfaces.  The email interface is
> quite similar to debian's.  I've never used a launchpad hosted project
> so I can't compare it.

Roundup is an issue tracker, while Patchwork is a patch tracker.
They are fundamentally distinct, but there are overlaps. What led me
to go the Patchwork-path is that projects like the kernel and Git
don't use issue trackers but work entirely patch-based.

I don't know if that is the right way to do things, but having an
issue tracker that fills up with bugs and wishlist items lacking
patches is no better in the long run than not having an issue
tracker.

Arguably, being patch-centric means that a project has a higher
barrier of entry, but it also means that if someone wants something,
they know that they'll have to somehow end up with a patch. The way
this happens on Git is that you either write it yourself and bring
it up to discussion (which is what patchwork facilitates), or
constructively theorise the functionality until someone else
submits a patch.

>   Google's codereview tool has a nice interface for collecting and
> commenting on patches, but I suspect that suggestion will also meet with
> a degree of friction.  To me codereview feels like patchwork with
> polish.

Maybe you could take some ideas from codereview and inform the
patchwork people about them?

>   Both gitorious and github have commenting functionality built in.
> Commenting on commits in a fork is as easy as opening the commit in
> a browser.  I use something along the lines of the following script to
> open commits on github:
> 
> #! /bin/sh
> BASE=$(git config remote.${2:-origin}.url | sed 
> 's,git\(@\|://\)\([^:/]*\)[:/]\(.*\).git,http://\2/\3/commit,')
> COMMIT=$(git rev-parse ${1:-HEAD})
> sensible-browser ${BASE}/${COMMIT}
> 
>   Using github or gitorious you can easily find and track forks from one
> place as well, which makes discovering new work much easier.  Github
> even provides a pretty single page interface to the work going on in
> other forks, gitorious requires a little more leg work to do the same
> but not much.

Git now has commit notes, but it doesn't seem like that's integrated
with Github/Gitorious.

Mind you, patchwork isn't integrated at all with Git. It should be
possible to set it up to automatically flag patches that are
accepted into mainline, next, or pu.

The benefit of patchwork is that discussion isn't moved to the web,
but patchwork hooks into the mailing list, so discussion can stay
where it should IMHO be.

>   For a couple of hosted projects we use at the office we email the
> individual entries from http://github.com/$user/$project/comments.atom
> to the mailing list so they're /forcibly/ seen by everybody :)

Right, but replying requires them to open a browser and be online at
the time, right?



Anyway, I suggest we give patchwork a try. It occurs to me that
notmuch can pretty much do all of what patchwork is doing ? after
all, it's just tagging patches/threads, but until we have
synchronisable tags and a mailing list archive based on notmuch
(which could then replace patchwork), I think we'll need to employ
a third tool.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"what's your conceptual continuity? --
 well, it should be easy to see:
 the crux of the bisquit is the apopstrophe!"
-- frank zappa

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] [PATCH] Enforce foldmethod=manual when showing messages in vim

2010-02-02 Thread martin f. krafft
Vim's NotMuch mode relies on manual markers when rendering/showing
a message. If foldmethod is set to something else (marker in my case) by
default, then there are numerous errors, and folds don't work. Hence,
set foldmethod=manual for the local buffer upon showing a message.

Signed-off-by: martin f. krafft madd...@debian.org
---
 vim/plugin/notmuch.vim |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/vim/plugin/notmuch.vim b/vim/plugin/notmuch.vim
index a226f20..2f9b05c 100644
--- a/vim/plugin/notmuch.vim
+++ b/vim/plugin/notmuch.vim
@@ -421,6 +421,7 @@ function! s:NM_cmd_show(words)
 let b:nm_raw_info = info
 let b:nm_prev_bufnr = prev_bufnr
 
+setlocal foldmethod=manual
 call SIDNM_cmd_show_mkfolds()
 call SIDNM_cmd_show_mksyntax()
 call SIDNM_set_map('n', g:notmuch_show_maps)
-- 
1.6.6


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Some thoughts about notmuch sync with other agents

2010-02-01 Thread martin f krafft
also sprach Paul R  [2010.01.28.0316 +1300]:
> As you see, I advocate a NotMuch <-> IMAP synchronisation ASAP :)

Given the limitations of IMAP (non-transactional, non-standard
keywords implementation, ?), I think chances of this are rapidly
diminishing, but I'd love to be proven wrong.

> First of all, processing mail with MUA involves some simple logic that
> is shared by most MUA. This is about receiving *new* mails, *reading*
> mail, *replying* to mail and so on... IMHO, this really belongs to the
> mail processing logic and not to the user logic. Hence my first
> request :
> 
>   1: Don't use user tags space to store MUA flags.
> 
>  That means no more "seen", "unread", "replied" as tags. These are
>  MUA processing *flags*, that must belong to an established set.
>  Tags, on the other hand, are user-land information. So no more
>  [seen, replied, grandma, important] tag sets :)

I disagree.

The MUA actually doesn't (shouldn't) care at all about any of these
flags, at least not for core functionality. Sure, a MUA should
probably hide messages tagged 'deleted', and it would be nice if it
could be configured to highlight messages tagged 'important', but
none of the others ? "seen", "unread", "replied", ? ? have any role
in the core functionality. They are purely user-specific.

They are leftovers from days when some MUA designer decided that
having these flags would be a useful way to organise e-mail
handling, but that person probably dealt with a dozen messages
a day, and didn't have half his/her life organised through
electronic mail.

Point being, times and needs have changed. And while we're in the
process of finding the technology that can suit those needs (in the
most generic way), we might just as well (and should) rid ourselves
from these leftovers.

Any solution must be generic enough so that if you rely on the
functionality provided by these tags, you can trivially make it
happen, e.g. with hooks to add/remove flags on certain actions (such
as sending a reply, or reading a message).

Neither IMAP nor Maildir is capable of storing an extensible,
freely-configurable set of tags (keywords). Therefore, we need a new
method anyway.

> Once this is done, notmuch will know, in a robust a predictable
> way, what happened to a mail. Simply put, NotMuch will store and
> expose MUA flags (Passed, Replied, Seen, Trashed, Draft, and
> Flagged [1]). For each , notmuch should associate
> a _synced flag. When changing  from notmuch, it should
> set the _synced bit to 0. These are MUA mail flags.

I think the semantics would be clearer the other way around: setting
*_changed when a flag is changed.

> Additionally, in a third ? space ?, notmuch should store its ? new ?
> bit, as well as a ? missing ? bit probably. Again, this is neither MUA
> logic or user logic, so this should not interfer with user
> classification facility provided by tags, nor with MUA flags. It,
> really, is something else, let's name it "status".

Or "lifetime".

> Once this is done, the 'notmuch new' command should find new mails
> and set the 'new' bit for them, and find the missing mails and set
> the 'missing' bit for them. This will allow for robust external
> processing.

Why would I want to keep around a record in the database when the
physical file is no longer present?

> # 1/ Sync from NotMuch to MailDir
> 
> notmuch list flags:(seen and not seen_synced) 
>   | notmuch-maildir --mark-mail seen
>   | notmuch move --stdin
>   | notmuch set flags:+seen_synced --stdin

Have you seen/look at notmuchsync?

> The output of the first command would be a list of filenames for mails
> 'seen' since last sync. The second one, an external notmuch--maildir
> helper, would propagate this logic to the MailDir store (easy, this is
> simply a rename), and output the list of (old-name ! new-name). Then
> notmuch would use this information, via a generic 'move' switch, to know
> that mail has been moved, and would output the list of the new places.
> Finaly, notmuch would set the seen_synced flag.

What happens if notmuch-move gets killed due to out-of-memory half
way through?

> As we have seen, this would allow most IMAP <-> MailDir <-> NotMuch
> synchronisation, without having to implement any kind of
> MailDir-specific logic inside notmuch.

It would not take care of any tags beyond the very strictly defined
and limited set of tags you listed in your mail. My point is that we
need to solve this problem more generally anyway ? why should
a "replied" tag be synchronised, but a "no-need-to-reply" tag not?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

#include 

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] patchwork test instance (was: Git feature branch)

2010-02-01 Thread martin f krafft
I investigated some patch/issue trackers over the weekend. Here's my
summary/reply.

The executive summary is that
http://patchwork.madduck.net/project/notmuch/list/ now exists.
I have not really used it for anything real, so if some of you feel
inclined to give it a shot, sign up and triage away! Feedback
welcome.

also sprach James Rowe jnr...@gmail.com [2010.01.28.2005 +1300]:
   Roundup has command line and email interfaces.  The email interface is
 quite similar to debian's.  I've never used a launchpad hosted project
 so I can't compare it.

Roundup is an issue tracker, while Patchwork is a patch tracker.
They are fundamentally distinct, but there are overlaps. What led me
to go the Patchwork-path is that projects like the kernel and Git
don't use issue trackers but work entirely patch-based.

I don't know if that is the right way to do things, but having an
issue tracker that fills up with bugs and wishlist items lacking
patches is no better in the long run than not having an issue
tracker.

Arguably, being patch-centric means that a project has a higher
barrier of entry, but it also means that if someone wants something,
they know that they'll have to somehow end up with a patch. The way
this happens on Git is that you either write it yourself and bring
it up to discussion (which is what patchwork facilitates), or
constructively theorise the functionality until someone else
submits a patch.

   Google's codereview tool has a nice interface for collecting and
 commenting on patches, but I suspect that suggestion will also meet with
 a degree of friction.  To me codereview feels like patchwork with
 polish.

Maybe you could take some ideas from codereview and inform the
patchwork people about them?

   Both gitorious and github have commenting functionality built in.
 Commenting on commits in a fork is as easy as opening the commit in
 a browser.  I use something along the lines of the following script to
 open commits on github:
 
 #! /bin/sh
 BASE=$(git config remote.${2:-origin}.url | sed 
 's,git\(@\|://\)\([^:/]*\)[:/]\(.*\).git,http://\2/\3/commit,')
 COMMIT=$(git rev-parse ${1:-HEAD})
 sensible-browser ${BASE}/${COMMIT}
 
   Using github or gitorious you can easily find and track forks from one
 place as well, which makes discovering new work much easier.  Github
 even provides a pretty single page interface to the work going on in
 other forks, gitorious requires a little more leg work to do the same
 but not much.

Git now has commit notes, but it doesn't seem like that's integrated
with Github/Gitorious.

Mind you, patchwork isn't integrated at all with Git. It should be
possible to set it up to automatically flag patches that are
accepted into mainline, next, or pu.

The benefit of patchwork is that discussion isn't moved to the web,
but patchwork hooks into the mailing list, so discussion can stay
where it should IMHO be.

   For a couple of hosted projects we use at the office we email the
 individual entries from http://github.com/$user/$project/comments.atom
 to the mailing list so they're /forcibly/ seen by everybody :)

Right, but replying requires them to open a browser and be online at
the time, right?



Anyway, I suggest we give patchwork a try. It occurs to me that
notmuch can pretty much do all of what patchwork is doing — after
all, it's just tagging patches/threads, but until we have
synchronisable tags and a mailing list archive based on notmuch
(which could then replace patchwork), I think we'll need to employ
a third tool.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
what's your conceptual continuity? --
 well, it should be easy to see:
 the crux of the bisquit is the apopstrophe!
-- frank zappa
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Request for high-priority improvements to notmuch

2010-02-01 Thread martin f krafft
also sprach sebast...@sspaeth.de sebast...@sspaeth.de [2010.02.02.0929 +1300]:
 YES PLEASE :-). notmuch seems designed to work in an ecosystem of
 surrounding scripts, feeding data in and out. But we are all currently
 limited to regexes for that. And heck, I hard a hard time understanding
 why all hell broke out until I found that i had added a tag containing
 parentheses which made my regex fail. :-). XML, JSON, any structured
 output would be nice.

Shoot me, but I'd say mbox output would be nice too.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
time flies like an arrow. fruit flies like a banana.
   -- groucho marx
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-29 Thread martin f krafft
also sprach Ben Gamari  [2010.01.29.0949 +1300]:
> > I guess I just dislike having to run notmuch new regularly,
> > rather than integrating the database more closely with the mail
> > flow.
> > 
> Sounds like you need to add a line to crontab.

It still feels like a hack. It's a bit like making many changes to
a source code repository (new mails get delivered) and committing
only once every hour (notmuch new), rather than making and
committing transactional changes (delivering and catalogueing mails
individually).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

a Hooloovoo is a superintelligent shade of the color blue.
-- douglas adams, "the hitchhiker's guide to the galaxy"

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-28 Thread martin f krafft
also sprach martin f krafft  [2010.01.28.1834 +1300]:
> > That's not going to work so well if so.
> 
> Why not? Works fine for me with the vim plugin...

Now I get it. I was talking about
id:20100114084713.GA22273 at harikalardiyari

Sorry, I *am* new to notmuch ;)

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"when zarathustra was alone... he said to his heart: 'could it be
 possible! this old saint in the forest hath not yet heard of it, that
 god is dead!'"
 - friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100128/c2bedb2b/attachment.pgp>


[notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-28 Thread martin f krafft
also sprach Jameson Rollins  [2010.01.27.0603 
+1300]:
> > This is getting involved. 
> > 
> > Maybe I'm missing something in this thread; but, why couldn't these complex 
> > and
> > context-sensitive decisions be delegated to sub-processes? ala git hooks?
> 
> I think this idea is completely independent of anything having to do
> with using git as a mail store.  That's why I was trying to separate it
> into a separate thread.

I think he meant "notmuch hooks like you have hooks for Git too",
e.g. thread:755741d13573c7642761d2a175cb146d

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"if i am occasionally a little overdressed, i make up for it by being
 always immensely over-educated."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-28 Thread martin f krafft
also sprach Jameson Rollins  [2010.01.26.1046 
+1300]:
> > For example, I might have:
> > 
> > ~/.notmuch-config:
> > 
> > [database]
> > path=/home/pioto/mail
> > ...
> > [tags]
> > pioto at pioto.org/INBOX.ListMail.notmuch = notmuch
> > 
> > So, a 'tags' section, where each key is the folder name, relative to the
> > db path, and the value is one or more tag names
> 
> I think this idea is a really good one and I would like to pursue it as
> a tangent thread here.  I was going to propose something very similar to
> this.  I think it's a very flexible idea that would help in a lot of
> ways.

I think we need to carefully distinguish here. The above seems to
suggest a mapping from folder to tag, but we don't actually need
tags for folder locations, because those can (and should) be
implicitly determined from the database and storing the tag in
addition would just run the risk of getting out of sync: if I moved
a message, I would also have to remember to delete old and add new
tags, which is just asking for trouble.

> [tags]
> inbox = +inbox,+unread
> sent = +sent
> drafts = +draft
> archive = -inbox

This proposal, on the other hand, is an interesting one, but when is
it supposed to happen? It just feels wrong to make this happen as
part of 'notmuch new'.

What I would like to see is a notmuch-aware MDA, e.g. a programme
which reads an incoming mail on stdin and can do all this kind of
stuff, e.g. assign tags based on such rules (or take tags as
arguments, so that I could trivially set tags from procmail too),
write the message to the message store, and update the database.

This would allow us to get rid of 'notmuch new' altogether, at least
conceptually. We'd still need it if mail is being delivered
independently, e.g. with offlineimap.

On the performance side, it might make sense to write to a journal
instead of updating the database every time. SpamAssassin does this
with its Bayesian database, and it only merges the journal every
X updates (or when the user manually requests it). I am not sure
whether this is possible with Xapian. On the other hand, I think
notmuch needs to learn to journal anyway so that we can keep
different instances in sync.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"the only way to get rid of a temptation is to yield to it."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git feature branch

2010-01-28 Thread martin f krafft
also sprach micah anderson  [2010.01.27.1124 +1300]:
> Couldn't all of this be done without moving the existing git
> repository (don't forget that transition is a cost)? Those who
> wish to put together these proposed branches go ahead and do so,
> publishing those wherever they like (git.debian.org, gitorious,
> their own hosted git repositories, or even github) and then Carl
> can just add those as remotes as he sees fit.

I brought this up, so I should make the motivation explicit: I just
tried to relieve Carl from the burden of bottleneck. Since passing
out SSH accounts on his own machine does not really scale, I looked
elsewhere.

However, I agree that the easiest solution is just to add more
users. Potentially, Carl can make use of gitolite, I can investigate
that some time this week.

> Personally, I've found mailing lists that have patches sent to
> them tends to totally kill the list for anything else. It seems
> a bit weird to use Debian's bug tracker for a non-Debian native
> program (but using it for the Debian package of notmuch does make
> sense). I am not so familiar with Roundup, patch queue trackers or
> patchwork to have anything to say about those.

patchwork integrates with the mailing list and slurps patches and
related discussion and threads them into a webpage, where they can
be workflow-managed.

The Debian bug tracker has the benefit of being usable with e-mail
(and this is notmuch we're developing, don't forget). The others are
all exclusively web-based, with the exception of launchpad, AFAIK.

The idea of having a tracker is quite simply to make it easier for
Carl to stay on top, and for e.g. you and I to assist him by vetting
patches in our free minutes.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

there's an old proverb that says just about whatever you want it to.

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-28 Thread martin f krafft
also sprach Servilio Afre Puentes servi...@gmail.com [2010.01.29.0132 +1300]:
  [tags]
  inbox = +inbox,+unread
  sent = +sent
  drafts = +draft
  archive = -inbox
 
  This proposal, on the other hand, is an interesting one, but when is
  it supposed to happen? It just feels wrong to make this happen as
  part of 'notmuch new'.
 
 Why so?

I guess I just dislike having to run notmuch new regularly, rather
than integrating the database more closely with the mail flow.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
to get back my youth i would do anything in the world, except take
 exercise, get up early, or be respectable.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git feature branch

2010-01-27 Thread martin f krafft
also sprach micah anderson mi...@riseup.net [2010.01.27.1124 +1300]:
 Couldn't all of this be done without moving the existing git
 repository (don't forget that transition is a cost)? Those who
 wish to put together these proposed branches go ahead and do so,
 publishing those wherever they like (git.debian.org, gitorious,
 their own hosted git repositories, or even github) and then Carl
 can just add those as remotes as he sees fit.

I brought this up, so I should make the motivation explicit: I just
tried to relieve Carl from the burden of bottleneck. Since passing
out SSH accounts on his own machine does not really scale, I looked
elsewhere.

However, I agree that the easiest solution is just to add more
users. Potentially, Carl can make use of gitolite, I can investigate
that some time this week.

 Personally, I've found mailing lists that have patches sent to
 them tends to totally kill the list for anything else. It seems
 a bit weird to use Debian's bug tracker for a non-Debian native
 program (but using it for the Debian package of notmuch does make
 sense). I am not so familiar with Roundup, patch queue trackers or
 patchwork to have anything to say about those.

patchwork integrates with the mailing list and slurps patches and
related discussion and threads them into a webpage, where they can
be workflow-managed.

The Debian bug tracker has the benefit of being usable with e-mail
(and this is notmuch we're developing, don't forget). The others are
all exclusively web-based, with the exception of launchpad, AFAIK.

The idea of having a tracker is quite simply to make it easier for
Carl to stay on top, and for e.g. you and I to assist him by vetting
patches in our free minutes.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
there's an old proverb that says just about whatever you want it to.
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-27 Thread martin f krafft
also sprach Jameson Rollins jroll...@finestructure.net [2010.01.26.1046 
+1300]:
  For example, I might have:
  
  ~/.notmuch-config:
  
  [database]
  path=/home/pioto/mail
  ...
  [tags]
  pi...@pioto.org/INBOX.ListMail.notmuch = notmuch
  
  So, a 'tags' section, where each key is the folder name, relative to the
  db path, and the value is one or more tag names
 
 I think this idea is a really good one and I would like to pursue it as
 a tangent thread here.  I was going to propose something very similar to
 this.  I think it's a very flexible idea that would help in a lot of
 ways.

I think we need to carefully distinguish here. The above seems to
suggest a mapping from folder to tag, but we don't actually need
tags for folder locations, because those can (and should) be
implicitly determined from the database and storing the tag in
addition would just run the risk of getting out of sync: if I moved
a message, I would also have to remember to delete old and add new
tags, which is just asking for trouble.

 [tags]
 inbox = +inbox,+unread
 sent = +sent
 drafts = +draft
 archive = -inbox

This proposal, on the other hand, is an interesting one, but when is
it supposed to happen? It just feels wrong to make this happen as
part of 'notmuch new'.

What I would like to see is a notmuch-aware MDA, e.g. a programme
which reads an incoming mail on stdin and can do all this kind of
stuff, e.g. assign tags based on such rules (or take tags as
arguments, so that I could trivially set tags from procmail too),
write the message to the message store, and update the database.

This would allow us to get rid of 'notmuch new' altogether, at least
conceptually. We'd still need it if mail is being delivered
independently, e.g. with offlineimap.

On the performance side, it might make sense to write to a journal
instead of updating the database every time. SpamAssassin does this
with its Bayesian database, and it only merges the journal every
X updates (or when the user manually requests it). I am not sure
whether this is possible with Xapian. On the other hand, I think
notmuch needs to learn to journal anyway so that we can keep
different instances in sync.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
the only way to get rid of a temptation is to yield to it.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-27 Thread martin f krafft
also sprach Jameson Rollins jroll...@finestructure.net [2010.01.27.0603 
+1300]:
  This is getting involved. 
  
  Maybe I'm missing something in this thread; but, why couldn't these complex 
  and
  context-sensitive decisions be delegated to sub-processes? ala git hooks?
 
 I think this idea is completely independent of anything having to do
 with using git as a mail store.  That's why I was trying to separate it
 into a separate thread.

I think he meant notmuch hooks like you have hooks for Git too,
e.g. thread:755741d13573c7642761d2a175cb146d

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if i am occasionally a little overdressed, i make up for it by being
 always immensely over-educated.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

2010-01-27 Thread martin f krafft
also sprach James Westby jw+deb...@jameswestby.net [2010.01.28.1828 +1300]:
 Are you trying to use thread: such that it could be passed to
 notmuch show to see the conversation?
 
 That's not going to work so well if so.

Why not? Works fine for me with the vim plugin...

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
perfection is achieved, not when there is nothing more to add, but
 when there is nothing left to take away.
 -- antoine de saint-exupéry
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Git feature branch

2010-01-26 Thread martin f krafft
also sprach Carl Worth  [2010.01.23.1010 +1300]:
> Anyway, I'll be on vacation for the next few days, so will likely
> not have much, (likely have not much?), time for patch merging.
> 
> But I *am* anxious to get back to the backlog. And in the
> meantime, I really appreciate others merging and sharing patches.

I discussed this with Carl at LCA a bit and ideally we should come
up with a way to relieve Carl of the bottleneck burden (obviously
without stealing away his dictator hat ;)

I think it would make sense to move the mainline to git.debian.org
for now, or another place where everyone can easily get an account.
As alternatives I propose repo.or.cz. I'd prefer to stay away from
commercial services like Github.

Once we're there, how about copying the branch structure from
Git development[0]:

  maint/? the stable release
  master/   ? the stablising head
  next/ ? testing branch
  pu/   ? patch integration branch (proposed updates)

Each of the four branches is usually a direct descendant of the one
above it. Conceptually, the feature enters at an unstable branch
(usually 'next' or 'pu'), and "graduates" to 'master' for the next
release once it is considered stable enough.

0. http://repo.or.cz/w/git.git/blob/HEAD:/Documentation/gitworkflows.txt

Sebastian, would it make sense to migrate your work into a 'pu'
branch?

What patch tracking workflow should we adopt? Keep sending patches
to the mailing list and have a few people who integrate them into
master or pu (or even maint/next), as appropriate? Use the Debian
bug tracker? Use Sebastian's Roundup instance? Set up a patch queue
manager on notmuchmail.org? Use patchwork [1]?

Cheers,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"in all unimportant matters, style, not sincerity, is the essential.
 in all important matters, style, not sincerity, is the essential."
-- oscar wilde

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] proposal for more streamlined mail flow in notmuch

2010-01-26 Thread martin f krafft
also sprach Jameson Rollins  [2010.01.17.0949 
+1300]:
> I would like to put forth here a proposal for a couple of changes
> to notmuch that I believe will considerably streamline message
> handling and new message flow through notmuch.  Notmuch is still
> new and I believe it hasn't quite figured out the best way to
> handle message flow in the new mail handling paradigm that it is
> working.  This is a proposal to fix that.
>
> I believe that most people are syncing their mail from remote IMAP or
> POP servers to local maildirs (via offlineimap for instance), and that
> most people's IMAP servers have limited storage capacity.  This
> practically means that people can not keep all of their mail in a
> single directory.  However, notmuch currently basically assumes that
> this is what is happening.  In order to keep synced maildirs from
> growing out of hand, messages need to be either deleted or moved out
> of the "inbox" where they initially show up.

We should keep this in mind when designing an object store, whether
or not that's based on Git. If we use Git, then a local branch is
all we really need to support this. \o/

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

above all, we should not wish to divest
our existence of its rich ambiguity.
 --friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-26 Thread martin f krafft
also sprach Sebastian Spaeth  [2010.01.26.0249 +1300]:
> While notmuchsync fullfils my needs, it is a kludge. It needs to
> call "notmuch" for each mail where a MailDir flag has changed
> (which can be quite often on an initial run, where most mails are
> likely to be read), this can take a long, long time. It would
> makes sense IMHO to at least pick pioto's "don't set unread if 'S'
> flag is set" on notmuch new[1].

I am sure this could be implemented with libnotmuch if it proves to
be useful.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"it isn't pollution that's harming the environment.
 it's the impurities in our air and water that are doing it."
  - dan quayle

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-25 Thread martin f krafft
also sprach Asheesh Laroia  [2010.01.25.1819 +1300]:
> You say "Ouch" but you should know Dovecot *already* does this. I
> don't mind interoperating with that.
>
> See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues
> with the specification", subsection "Locking". I term this theQ
> famous readdir() race.

Yikes. IMAP (including dovecot) just SUCKS.

> Without this lock, Maildir is fundamentally incompatible with IMAP
> -- one Maildir-using process modifying message flags could make
> a different Maildir-using process think said message is actually
> deleted. In the case of temporary disappearing mails in Mutt
> locally, that's not the end of the world. For IMAP, it will make
> the IMAP daemon (one of the Maildir-using processes) send a note
> to IMAP clients saying that the message has been deleted and
> expunged.
[?]
> Just don't fall into the trap of thinking Maildir is compatible
> with IMAP. It's not, because as I understand things, the
> filesystem doesn't guarantee that you can actually iterate across
> a directory's files if another process is modifying the list of
> files.

This is all perfect reason to concentrate even more on designing
a store that could potentially make IMAP obsolete once and for all!

The current idea is to sync Git downstream only, and find a way to
keep multiple copies of a tagstore in sync, by way of the "server
instance" (where mail is received/delivered). Deleting messages
would then be something like setting the notmuch::deleted tag, which
clients would honour; on the server, a cleanup process would run
regularly to actually delete the blobs associated with deleted
messages. This would then propogate the next time one pulls from
Git.

Whether to store history (commit objects) or just collections (tree
objects) needs to be investigated.

> >But there are still good reasons why you'd want to have IMAP
> >capability too, e.g. Webmail. Given the atomicity problems that
> >come from Git, maybe an IMAP server reading from the Git store
> >would make sense.
> 
> It wouldn't be too hard to write a FUSE filesystem that presented
> an interface to a Git repository that didn't allow the contents of
> files to be modified. Then Dovecot could think it's interacting
> with the filesystem.

Yes, a FUSE layer (which adds a daemon), or a lightweight access
API via libnotmuch. Probably the former using the latter. ;)

> Aww, I like Maildir flags, but if there's a sync tool, I'm fine
> with that.
[?]
> I'm not sure, but maybe it's safe if you refuse to ever modify
> a message's flags in the filename.

The main point is that there is nothing really in Maildir filenames
that you couldn't equally (and possibly better) represent in the
notmuch::* tag namespace, and then there is benefit in only having
one used primarily (which means notmuchsync can do whatever it
wants without affecting or messing with notmuch).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"if I can't dance, i don't want to be part of your revolution."
- emma goldman

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-25 Thread martin f krafft
also sprach Asheesh Laroia  [2010.01.21.1928 +1300]:
> >I suppose that I never actually considered merges on the IMAP
> >server side, but obviously the IMAP server has to work off a clone,
> >and that means it needs to merge.
> 
> It's not "merge" that's unsafe; that just builds a tree in the git
> index (assuming no conflicts). It's the ensuing process of git
> writing a tree to the filesystem that is problematic.

There is no way to make that atomic, I am afraid. As you say.

> I could probably actually write a wrapper that locks the Maildir
> while git is operating. It would probably be specific to each IMAP
> server.

Ouch! I'd really rather not go there.

> Note that this mean git is fundamentally incompatible with
> Maildir, not just IMAP servers.

We had an idea about using Git to replace IMAP altogether, along
with making notmuch use a bare Git repository as object store. The
idea is that notmuch uses low-level Git commands to access the .git
repository (from which you can still checkout a tree tying the blobs
into a Maildir). The benefit would be compression, lower inode count
(due to packs), and backups using clones/merges.

You could either have the MDA write to a Git repo on the server side
and use git packs to download mail to a local clone, or one could
have e.g. offlineimap grow a Git storage backend. The interface to
notmuch would be the same.

If we used this, all the rename and delete code would be refactored
into Git and could be removed from notmuch. In addition, notmuch
could actually use Git tree objects to represent the results of
searches, and you could checkout these trees. However, deleting
messages from search results would not have any effect on the
message or its existence in other search results, much like what
happens with mairix nowadays.

I think we all kinda agreed that the Maildir flags should not be
used by notmuch and that things like Sebastian's notmuchsync should
be used if people wanted flags represented in Maildir filenames.

Instead of a Maildir checkout, notmuch could provide an interface to
browse the store contents in a way that could make it accessible to
mutt. The argument is that with 'notmuch {ls,cat,rm,?}', a mutt
backend could be trivially written. I am not sure about that, but
it's worth a try.

But there are still good reasons why you'd want to have IMAP
capability too, e.g. Webmail. Given the atomicity problems that come
from Git, maybe an IMAP server reading from the Git store would make
sense.

However, this all sounds like a lot of NIH and reinvention. It's
a bit like the marriage between the hypothetical Maildir2 and Git,
which is definitely worth pursuing. Before we embark on any of this,
however, we'd need to define the way in which Git stores mail.

Stewart, you've worked most on this so far. Would you like to share
your thoughts?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"reife des mannes, das ist es,
 den ernst wiedergefunden zu haben, den
 man hatte als kind beim spiel."
-- friedrich nietzsche

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-25 Thread martin f krafft
also sprach Sebastian Spaeth sebast...@sspaeth.de [2010.01.26.0249 +1300]:
 While notmuchsync fullfils my needs, it is a kludge. It needs to
 call notmuch for each mail where a MailDir flag has changed
 (which can be quite often on an initial run, where most mails are
 likely to be read), this can take a long, long time. It would
 makes sense IMHO to at least pick pioto's don't set unread if 'S'
 flag is set on notmuch new[1].

I am sure this could be implemented with libnotmuch if it proves to
be useful.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
it isn't pollution that's harming the environment.
 it's the impurities in our air and water that are doing it.
  - dan quayle
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] proposal for more streamlined mail flow in notmuch

2010-01-25 Thread martin f krafft
also sprach Jameson Rollins jroll...@finestructure.net [2010.01.17.0949 
+1300]:
 I would like to put forth here a proposal for a couple of changes
 to notmuch that I believe will considerably streamline message
 handling and new message flow through notmuch.  Notmuch is still
 new and I believe it hasn't quite figured out the best way to
 handle message flow in the new mail handling paradigm that it is
 working.  This is a proposal to fix that.

 I believe that most people are syncing their mail from remote IMAP or
 POP servers to local maildirs (via offlineimap for instance), and that
 most people's IMAP servers have limited storage capacity.  This
 practically means that people can not keep all of their mail in a
 single directory.  However, notmuch currently basically assumes that
 this is what is happening.  In order to keep synced maildirs from
 growing out of hand, messages need to be either deleted or moved out
 of the inbox where they initially show up.

We should keep this in mind when designing an object store, whether
or not that's based on Git. If we use Git, then a local branch is
all we really need to support this. \o/

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
above all, we should not wish to divest
our existence of its rich ambiguity.
 --friedrich nietzsche
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git feature branch

2010-01-25 Thread martin f krafft
also sprach Carl Worth cwo...@cworth.org [2010.01.23.1010 +1300]:
 Anyway, I'll be on vacation for the next few days, so will likely
 not have much, (likely have not much?), time for patch merging.
 
 But I *am* anxious to get back to the backlog. And in the
 meantime, I really appreciate others merging and sharing patches.

I discussed this with Carl at LCA a bit and ideally we should come
up with a way to relieve Carl of the bottleneck burden (obviously
without stealing away his dictator hat ;)

I think it would make sense to move the mainline to git.debian.org
for now, or another place where everyone can easily get an account.
As alternatives I propose repo.or.cz. I'd prefer to stay away from
commercial services like Github.

Once we're there, how about copying the branch structure from
Git development[0]:

  maint/— the stable release
  master/   — the stablising head
  next/ — testing branch
  pu/   — patch integration branch (proposed updates)

Each of the four branches is usually a direct descendant of the one
above it. Conceptually, the feature enters at an unstable branch
(usually 'next' or 'pu'), and graduates to 'master' for the next
release once it is considered stable enough.

0. http://repo.or.cz/w/git.git/blob/HEAD:/Documentation/gitworkflows.txt

Sebastian, would it make sense to migrate your work into a 'pu'
branch?

What patch tracking workflow should we adopt? Keep sending patches
to the mailing list and have a few people who integrate them into
master or pu (or even maint/next), as appropriate? Use the Debian
bug tracker? Use Sebastian's Roundup instance? Set up a patch queue
manager on notmuchmail.org? Use patchwork [1]?

Cheers,

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
in all unimportant matters, style, not sincerity, is the essential.
 in all important matters, style, not sincerity, is the essential.
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-24 Thread martin f krafft
also sprach Asheesh Laroia ashe...@asheesh.org [2010.01.21.1928 +1300]:
 I suppose that I never actually considered merges on the IMAP
 server side, but obviously the IMAP server has to work off a clone,
 and that means it needs to merge.
 
 It's not merge that's unsafe; that just builds a tree in the git
 index (assuming no conflicts). It's the ensuing process of git
 writing a tree to the filesystem that is problematic.

There is no way to make that atomic, I am afraid. As you say.

 I could probably actually write a wrapper that locks the Maildir
 while git is operating. It would probably be specific to each IMAP
 server.

Ouch! I'd really rather not go there.

 Note that this mean git is fundamentally incompatible with
 Maildir, not just IMAP servers.

We had an idea about using Git to replace IMAP altogether, along
with making notmuch use a bare Git repository as object store. The
idea is that notmuch uses low-level Git commands to access the .git
repository (from which you can still checkout a tree tying the blobs
into a Maildir). The benefit would be compression, lower inode count
(due to packs), and backups using clones/merges.

You could either have the MDA write to a Git repo on the server side
and use git packs to download mail to a local clone, or one could
have e.g. offlineimap grow a Git storage backend. The interface to
notmuch would be the same.

If we used this, all the rename and delete code would be refactored
into Git and could be removed from notmuch. In addition, notmuch
could actually use Git tree objects to represent the results of
searches, and you could checkout these trees. However, deleting
messages from search results would not have any effect on the
message or its existence in other search results, much like what
happens with mairix nowadays.

I think we all kinda agreed that the Maildir flags should not be
used by notmuch and that things like Sebastian's notmuchsync should
be used if people wanted flags represented in Maildir filenames.

Instead of a Maildir checkout, notmuch could provide an interface to
browse the store contents in a way that could make it accessible to
mutt. The argument is that with 'notmuch {ls,cat,rm,…}', a mutt
backend could be trivially written. I am not sure about that, but
it's worth a try.

But there are still good reasons why you'd want to have IMAP
capability too, e.g. Webmail. Given the atomicity problems that come
from Git, maybe an IMAP server reading from the Git store would make
sense.

However, this all sounds like a lot of NIH and reinvention. It's
a bit like the marriage between the hypothetical Maildir2 and Git,
which is definitely worth pursuing. Before we embark on any of this,
however, we'd need to define the way in which Git stores mail.

Stewart, you've worked most on this so far. Would you like to share
your thoughts?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
reife des mannes, das ist es,
 den ernst wiedergefunden zu haben, den
 man hatte als kind beim spiel.
-- friedrich nietzsche
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

2010-01-24 Thread martin f krafft
also sprach Asheesh Laroia ashe...@asheesh.org [2010.01.25.1819 +1300]:
 You say Ouch but you should know Dovecot *already* does this. I
 don't mind interoperating with that.

 See http://wiki.dovecot.org/MailboxFormat/Maildir, section Issues
 with the specification, subsection Locking. I term this theQ
 famous readdir() race.

Yikes. IMAP (including dovecot) just SUCKS.

 Without this lock, Maildir is fundamentally incompatible with IMAP
 -- one Maildir-using process modifying message flags could make
 a different Maildir-using process think said message is actually
 deleted. In the case of temporary disappearing mails in Mutt
 locally, that's not the end of the world. For IMAP, it will make
 the IMAP daemon (one of the Maildir-using processes) send a note
 to IMAP clients saying that the message has been deleted and
 expunged.
[…]
 Just don't fall into the trap of thinking Maildir is compatible
 with IMAP. It's not, because as I understand things, the
 filesystem doesn't guarantee that you can actually iterate across
 a directory's files if another process is modifying the list of
 files.

This is all perfect reason to concentrate even more on designing
a store that could potentially make IMAP obsolete once and for all!

The current idea is to sync Git downstream only, and find a way to
keep multiple copies of a tagstore in sync, by way of the server
instance (where mail is received/delivered). Deleting messages
would then be something like setting the notmuch::deleted tag, which
clients would honour; on the server, a cleanup process would run
regularly to actually delete the blobs associated with deleted
messages. This would then propogate the next time one pulls from
Git.

Whether to store history (commit objects) or just collections (tree
objects) needs to be investigated.

 But there are still good reasons why you'd want to have IMAP
 capability too, e.g. Webmail. Given the atomicity problems that
 come from Git, maybe an IMAP server reading from the Git store
 would make sense.
 
 It wouldn't be too hard to write a FUSE filesystem that presented
 an interface to a Git repository that didn't allow the contents of
 files to be modified. Then Dovecot could think it's interacting
 with the filesystem.

Yes, a FUSE layer (which adds a daemon), or a lightweight access
API via libnotmuch. Probably the former using the latter. ;)

 Aww, I like Maildir flags, but if there's a sync tool, I'm fine
 with that.
[…]
 I'm not sure, but maybe it's safe if you refuse to ever modify
 a message's flags in the filename.

The main point is that there is nothing really in Maildir filenames
that you couldn't equally (and possibly better) represent in the
notmuch::* tag namespace, and then there is benefit in only having
one used primarily (which means notmuchsync can do whatever it
wants without affecting or messing with notmuch).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
if I can't dance, i don't want to be part of your revolution.
- emma goldman
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Thoughts on notmuch and Lua

2010-01-15 Thread martin f krafft
also sprach Carl Worth  [2010.01.15.1200 +1300]:
> > Lua has many advantages over other scripting languages when it
> > comes to integration with a C program. It has a very clean and
> > easy C API, the overhead of running Lua scripts is not noticable
> > among other things.
> 
> I've definitely heard lots of good things about "lua
> embedability". So if we do decide to provide hooks, then lua would
> seem like a logical option to look at first. I've never looked at
> it closely myself though.

Lua for hooks has the advantage that the hooks can be executed in
the context of manipulateable objects. On the other hand, hooks in
the style of run-parts directories are more flexible and accessible,
and could always be invoked as filters for the manipulateable data.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"imagine if every thursday your shoes exploded if you
 tied them the usual way. this happens to us all the time
 with computers, and nobody thinks of complaining."
-- jeff raskin

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



[notmuch] Threading

2010-01-15 Thread martin f krafft
also sprach Carl Worth  [2010.01.15.1108 +1300]:
> > Reading is one thing. Information storage and organisation is
> > another. After a message is delivered (and read) to my mailbox,
> > it's really mine and I can (and should be able) to affix it and
> > integrate it into my organisational scheme any way I want, don't
> > you think?
> 
> A fair point.
> 
> I don't see this being something I'm going to spend any time
> implementing. I just wouldn't use the functionality myself. But
> I would be happy to integrate patches if someone came up with
> some.

Maybe I should try to persuade you in person.

Just today I referenced a discussion I had with a client's ISP,
which was done via a web-based support system (custhelp.com). They
send you e-mail for every post you or they make to the thread, but
those e-mails do not reference each other. Fortunately, I stitched
them together and when I searched for the correspondence in my
mailstore, I had the entire thread available to me, which was handy
(thanks to mutt's useful thread handling abilities).

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

"this week dragged past me so slowly;
 the days fell on their knees..."
-- david bowie

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 



  1   2   >