[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Sebastian Spaeth]

On Mon, 25 Jan 2010 14:49:00 +0100, "Sebastian Spaeth"  wrote:
> While notmuchsync fullfils my needs, it is a kludge. It needs to call
> "notmuch" for each mail where a MailDir flag has changed (which can be
> quite often on an initial run, where most mails are likely to be read),
> this can take a long, long time. It would makes sense IMHO to at least
> pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1].

Once python bindings exist for the notmuch shared library, I am sure we
can speed notmuchsync up a lot by keeping the connection open and
tagging all mails in one go rather than executing a separate binary for
each mail. So, this approach might still be feasible.
Sebastian

[notmuch] [PATCH] Make the date parser nicer

[Sebastian Spaeth]

On Tue, 26 Jan 2010 09:55:00 -0800, Keith Packard  wrote:
> Very cool. Oh, if you've got commits that don't compile on their own,
> you should squash them together (or fix it in some other way). Makes
> bisecting easier in the future.

Makes sense. I am still quite new to git, so excuse those beginner's
lapses. Perhaps Carl could squash commits ec3c79a and 2565fc6 when (if?)
pulling, that would make every step compile IMHO.

> Also, cworth is on vacation this week, so we won't be seeing any
> merging to master...

No hurry :-). cworth will have to do quite some catching up when he
returns.

The one "disadvantage" my integration has over your original approach,
is that we now always require "date:XXX..YYY". A 'date:lastmonth' won't
work, it will need to be "date:lastmonth..today". The reason is that
xapian only seems to invoke the RangeParser when something of the format
'A..B' is passed as a parameter. So while we could get "date:..2005" to
work, "date:2005.." is not passed to the RangeParser handler, it seems.

We could ditch the "date:" prefix, but imho it is more consistent with
the other keywords to use it. I have no strong feelings about this.

It also still has the same limitation, in that it will not find emails with
a future timestamp (I use date:lastweek..5000 to get all mails with a
future stamp).

Sebastian

[notmuch] Internal error no thread ID

[Matthias Teege]

Hello,

I have tried to add some mails to the index with "notmuch new" but it
gives me:

Internal error: Message with document ID of 62004 has no thread ID.
 (lib/message.cc:353).

Is it possible to get the filename for the document 62004? Is there any
way to exclude some files from indexing? I compiled notmuch from the
git repo with xapian 1.0.17.

Many thanks
Matthias

[notmuch] Indexing without files and directories

[Simon Cozens]

Hi all,
I don't want to throw a spanner in the works, but I am intending to use
notmuch for indexing in a scenario where the email doesn't hit the disk
- the notmuch model seems to rely pretty heavily on directories and
files but I'm working with email being delivered to a script for
indexing and then stored into a database. So far I've been hacking it by
feeding "/dev/stdin" as a filename but that leads to interesting fun
when I forget and try to access something via a notmuch_message object.
With all this talk of a new object store, can I put in a plea for - at
least at the API level - some way of indexing based on a user-defined
location identifier? (In my case, a database row ID.)

Thanks,
Simon

[notmuch] Fcc, Maildir, and Emacs message-mode -- a bit of code

[Jameson Rollins]

Hey, folks.  Following up on this thread about better fcc handling,
Jesse passed on a simple python script he wrote that uses the python
"mailbox" module to deliver a message on stdin to a specified maildir
directory.  It's very a simple, elegant and general purpose script
(attached).

I then put the following in my notmuch .emacs to use the new script in
message-mode to fcc sent mail to my ~/.mail/sent directory:

;; fcc handler
(defun maildir-deliver-region(destdir)
  (shell-command-on-region
   (point-min) (point-max)
   (concat "maildir-deliver.py -c -s -d " destdir)))
(setq message-fcc-handler-function 'maildir-deliver-region)
(defun my-message-header-setup ()
  (message-add-header "Fcc: ~/.mail/sent"))
(add-hook 'message-send-hook 'my-message-header-setup)

Works like a charm.  Thanks Jesse!

I think we should look at packaging this in a set of notmuch helper
scripts, hopefully including notmuchsync.

jamie.

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100127/bde1acc7/attachment.pgp>
-- next part --
A non-text attachment was scrubbed...
Name: maildir-deliver.py
Type: text/x-python
Size: 1090 bytes
Desc: python maildir delivery script
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100127/bde1acc7/attachment.py>

[notmuch] Git feature branch

[micah anderson]

On Mon, 25 Jan 2010 16:46:43 -0800, sebastian at sspaeth.de wrote:
> > I think it would make sense to move the mainline to git.debian.org
> > for now, or another place where everyone can easily get an account.
> > As alternatives I propose repo.or.cz. I'd prefer to stay away from
> > commercial services like Github.

I too prefer to stay away from commercial,non-free services like GitHub.
Gitorious (http://www.gitorious.org/) is a great service, similar to
Github, but with added freedom.

Currently the git repository is at git://notmuchmail.org/git/notmuch
which seems like a fine location for it, but perhaps I missed the
motivation for switching to a centralized repository with the added
overhead of giving people access to commit?

Couldn't all of this be done without moving the existing git repository
(don't forget that transition is a cost)? Those who wish to put together
these proposed branches go ahead and do so, publishing those wherever
they like (git.debian.org, gitorious, their own hosted git repositories,
or even github) and then Carl can just add those as remotes as he sees
fit.

> > What patch tracking workflow should we adopt? Keep sending patches
> > to the mailing list and have a few people who integrate them into
> > master or pu (or even maint/next), as appropriate? Use the Debian
> > bug tracker? Use Sebastian's Roundup instance? Set up a patch queue
> > manager on notmuchmail.org? Use patchwork [1]?

Personally, I've found mailing lists that have patches sent to them
tends to totally kill the list for anything else. It seems a bit weird
to use Debian's bug tracker for a non-Debian native program (but using
it for the Debian package of notmuch does make sense). I am not so
familiar with Roundup, patch queue trackers or patchwork to have
anything to say about those.

micah
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100127/297ea735/attachment.pgp>

[notmuch] Git feature branch

[Ben Gamari]

Excerpts from micah anderson's message of Tue Jan 26 17:24:15 -0500 2010:
> On Mon, 25 Jan 2010 16:46:43 -0800, sebastian at sspaeth.de wrote:
> > > I think it would make sense to move the mainline to git.debian.org
> > > for now, or another place where everyone can easily get an account.
> > > As alternatives I propose repo.or.cz. I'd prefer to stay away from
> > > commercial services like Github.
> 
> I too prefer to stay away from commercial,non-free services like GitHub.
> Gitorious (http://www.gitorious.org/) is a great service, similar to
> Github, but with added freedom.
> 
> Currently the git repository is at git://notmuchmail.org/git/notmuch
> which seems like a fine location for it, but perhaps I missed the
> motivation for switching to a centralized repository with the added
> overhead of giving people access to commit?
> 
> Couldn't all of this be done without moving the existing git repository
> (don't forget that transition is a cost)? Those who wish to put together
> these proposed branches go ahead and do so, publishing those wherever
> they like (git.debian.org, gitorious, their own hosted git repositories,
> or even github) and then Carl can just add those as remotes as he sees
> fit.
> 
I agree. There is no good reason to switch away from the existing
infrastructure. If he wants, Carl can give regular contributors their
own repositories on notmuchmail.org if some people have difficulties
providing it themselves. After all, this is one of the strengths of
distributed version control.

- Ben

[notmuch] Git feature branch

[Jameson Rollins]

On Wed, 27 Jan 2010 11:24:15 +1300, micah anderson  wrote:
> Couldn't all of this be done without moving the existing git repository
> (don't forget that transition is a cost)? Those who wish to put together
> these proposed branches go ahead and do so, publishing those wherever
> they like (git.debian.org, gitorious, their own hosted git repositories,
> or even github) and then Carl can just add those as remotes as he sees
> fit.

I second this idea.  I've been working on quite a few projects that work
in this way, with nothing but personal distributed repos, and it works
quite well.  Especially if Carl is going to continue to be benevolent
dictator (as I agree that he should).  Everyone who has patches can just
post them to their personal repos, making sure that they're well rebased
to Carl's master, and Carl can just pull from them as he sees fit.  It's
what git is designed to do, and it actually works really well.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100127/fcded64c/attachment.pgp>

[notmuch] Some thoughts about notmuch sync with other agents

[Paul R]

Hi,

before going into details, I'd like to tell you how much I like the
notmuch workflow, thank you for producing this nice piece of software !

Like most potential users, I can not really fully jump into notmuch
because of the current synchronisation issues with others MUA. One may
or may not like IMAP for good reasons, the fact is that it is here and
has allowed users to read mails from various places and terminals,
keeping important information synced. So I think that notmuch will have
to live with that, and provide what is required to integrate smoothly
into this environment. Redefining a new mail retrieval protocol and
a mail storing format are both really exciting projects, but they are
projects on their own that could only distract notmuch from its most
beautiful goal : giving *today* users the power to process mail in an
efficient way.

As you see, I advocate a NotMuch <-> IMAP synchronisation ASAP :)

At the moment, notmuch input are mail-as-file + user-defined tags.
OfflineIMAP allows to do the IMAP <-> mail-as-file transition, in both
directions, mail-as-file being namely MailDir. So we can simply aim at
a NotMuch <-> MailDir synchronisation, offlineimap will take care of
IMAP itself.

Of course, my proposal does not require to implement any MailDir
specific logic inside NotMuch, but rather describes how should notmuch
evolve to allow easy third-party-tool jobs.


Preliminary thoughts :
--

First of all, processing mail with MUA involves some simple logic that
is shared by most MUA. This is about receiving *new* mails, *reading*
mail, *replying* to mail and so on... IMHO, this really belongs to the
mail processing logic and not to the user logic. Hence my first
request :

  1: Don't use user tags space to store MUA flags.

 That means no more "seen", "unread", "replied" as tags. These are
 MUA processing *flags*, that must belong to an established set.
 Tags, on the other hand, are user-land information. So no more
 [seen, replied, grandma, important] tag sets :)

Once this is done, notmuch will know, in a robust a predictable way,
what happened to a mail. Simply put, NotMuch will store and expose MUA
flags (Passed, Replied, Seen, Trashed, Draft, and Flagged [1]). For each
, notmuch should associate a _synced flag. When changing
 from notmuch, it should set the _synced bit to 0. These are
MUA mail flags.

Additionally, in a third ? space ?, notmuch should store its ? new ?
bit, as well as a ? missing ? bit probably. Again, this is neither MUA
logic or user logic, so this should not interfer with user
classification facility provided by tags, nor with MUA flags. It,
really, is something else, let's name it "status". Once this is done,
the 'notmuch new' command should find new mails and set the 'new' bit
for them, and find the missing mails and set the 'missing' bit for them.
This will allow for robust external processing.

Finally, notmuch should provide a switch to output a list of filenames
to stdout and to process a list of filenames from stdin.


NotMuch <-> MailDir synchronisation :
-

Provided the behaviour described above, notmuch <-> MailDir
synchronisation could be done fully externally, by a 'notmuch-maildir'
adapter.

Here is some pseudo code, that could be wrapped into a single
'notmuch-sync' command. The | are unix stream pipes, and everything
should be on a single line.

# 1/ Sync from NotMuch to MailDir

notmuch list flags:(seen and not seen_synced) 
  | notmuch-maildir --mark-mail seen
  | notmuch move --stdin
  | notmuch set flags:+seen_synced --stdin

The output of the first command would be a list of filenames for mails
'seen' since last sync. The second one, an external notmuch--maildir
helper, would propagate this logic to the MailDir store (easy, this is
simply a rename), and output the list of (old-name ! new-name). Then
notmuch would use this information, via a generic 'move' switch, to know
that mail has been moved, and would output the list of the new places.
Finaly, notmuch would set the seen_synced flag.

Same would apply for the Replied, Trashed, Flagged and Passed flags.

# 2/ Then lets do the MailDir <-> IMAP sync

 offlineimap

... done ! that was easy :)

# 3/ notmuch new

 notmuch new

At this point, notmuch should set the 'new' or the 'missing' status bit
to the mails. Let's forget how to deal with the missing bit, that should
be easy to do.

# 4/ Sync from MailDir to NotMuch

  notmuch list status:new 
   | notmuch-maildir --filter seen
   | notmuch set flags:+seen+seen_synced --stdin

First command outputs newly discovered mail. Second one reads stdin and
output only files that are already seen (again, this is as easy as
a filter based on a regular expression). Third one reads stdin and
applies the seen and seen_synced flags.

Same applies for the Replied, Trashed, Flagged and Passed flags.


Conclusion:
---

As we have seen, this would allow most IMAP <-

Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)

[Sebastian Spaeth]

On Mon, 25 Jan 2010 14:49:00 +0100, "Sebastian Spaeth"  
wrote:
> While notmuchsync fullfils my needs, it is a kludge. It needs to call
> "notmuch" for each mail where a MailDir flag has changed (which can be
> quite often on an initial run, where most mails are likely to be read),
> this can take a long, long time. It would makes sense IMHO to at least
> pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1].

Once python bindings exist for the notmuch shared library, I am sure we
can speed notmuchsync up a lot by keeping the connection open and
tagging all mails in one go rather than executing a separate binary for
each mail. So, this approach might still be feasible.
Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] [PATCH] Make the date parser nicer

[Sebastian Spaeth]

On Tue, 26 Jan 2010 09:55:00 -0800, Keith Packard  wrote:
> Very cool. Oh, if you've got commits that don't compile on their own,
> you should squash them together (or fix it in some other way). Makes
> bisecting easier in the future.

Makes sense. I am still quite new to git, so excuse those beginner's
lapses. Perhaps Carl could squash commits ec3c79a and 2565fc6 when (if?)
pulling, that would make every step compile IMHO.

> Also, cworth is on vacation this week, so we won't be seeing any
> merging to master...

No hurry :-). cworth will have to do quite some catching up when he
returns.

The one "disadvantage" my integration has over your original approach,
is that we now always require "date:XXX..YYY". A 'date:lastmonth' won't
work, it will need to be "date:lastmonth..today". The reason is that
xapian only seems to invoke the RangeParser when something of the format
'A..B' is passed as a parameter. So while we could get "date:..2005" to
work, "date:2005.." is not passed to the RangeParser handler, it seems.
 
We could ditch the "date:" prefix, but imho it is more consistent with
the other keywords to use it. I have no strong feelings about this.

It also still has the same limitation, in that it will not find emails with
a future timestamp (I use date:lastweek..5000 to get all mails with a
future stamp).

Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Internal error no thread ID

[Matthias Teege]

Hello,

I have tried to add some mails to the index with "notmuch new" but it
gives me:

Internal error: Message with document ID of 62004 has no thread ID.
 (lib/message.cc:353).

Is it possible to get the filename for the document 62004? Is there any
way to exclude some files from indexing? I compiled notmuch from the
git repo with xapian 1.0.17.

Many thanks
Matthias

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Indexing without files and directories

[Simon Cozens]

Hi all,
I don't want to throw a spanner in the works, but I am intending to use
notmuch for indexing in a scenario where the email doesn't hit the disk
- the notmuch model seems to rely pretty heavily on directories and
files but I'm working with email being delivered to a script for
indexing and then stored into a database. So far I've been hacking it by
feeding "/dev/stdin" as a filename but that leads to interesting fun
when I forget and try to access something via a notmuch_message object.
With all this talk of a new object store, can I put in a plea for - at
least at the API level - some way of indexing based on a user-defined
location identifier? (In my case, a database row ID.)

Thanks,
Simon
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Fcc, Maildir, and Emacs message-mode -- a bit of code

[Jameson Rollins]

Hey, folks.  Following up on this thread about better fcc handling,
Jesse passed on a simple python script he wrote that uses the python
"mailbox" module to deliver a message on stdin to a specified maildir
directory.  It's very a simple, elegant and general purpose script
(attached).

I then put the following in my notmuch .emacs to use the new script in
message-mode to fcc sent mail to my ~/.mail/sent directory:

;; fcc handler
(defun maildir-deliver-region(destdir)
  (shell-command-on-region
   (point-min) (point-max)
   (concat "maildir-deliver.py -c -s -d " destdir)))
(setq message-fcc-handler-function 'maildir-deliver-region)
(defun my-message-header-setup ()
  (message-add-header "Fcc: ~/.mail/sent"))
(add-hook 'message-send-hook 'my-message-header-setup)

Works like a charm.  Thanks Jesse!

I think we should look at packaging this in a set of notmuch helper
scripts, hopefully including notmuchsync.

jamie.



pgpS1ojP8FF63.pgp
Description: PGP signature
#!/usr/bin/env python

import mailbox
import sys
import optparse

def maildir_deliver(msg, maildir, mark_read=False, mark_cur=False):
if mark_read:
msg.set_flags("S")
if mark_cur:
msg.set_subdir('cur')
md = mailbox.Maildir(maildir)
key = md.add(msg)
md.close

if __name__ == '__main__':
parser = optparse.OptionParser()
parser.add_option("-d", "--destdir",
  dest="maildir", 
  help="destination maildir")
parser.add_option("-s", "--seen",
  action="store_true", 
  dest="mark_read",
  default=False,
  help="mark message as read")
parser.add_option("-c", "--cur",
  action="store_true", 
  dest="mark_cur",
  default=False,
  help="deliver message to cur instead of new")
(options, args) = parser.parse_args()
msg = mailbox.MaildirMessage(sys.stdin)
maildir_deliver(msg, options.maildir, options.mark_read, options.mark_cur)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git feature branch

[micah anderson]

On Mon, 25 Jan 2010 16:46:43 -0800, sebast...@sspaeth.de wrote:
> > I think it would make sense to move the mainline to git.debian.org
> > for now, or another place where everyone can easily get an account.
> > As alternatives I propose repo.or.cz. I'd prefer to stay away from
> > commercial services like Github.

I too prefer to stay away from commercial,non-free services like GitHub.
Gitorious (http://www.gitorious.org/) is a great service, similar to
Github, but with added freedom.

Currently the git repository is at git://notmuchmail.org/git/notmuch
which seems like a fine location for it, but perhaps I missed the
motivation for switching to a centralized repository with the added
overhead of giving people access to commit?

Couldn't all of this be done without moving the existing git repository
(don't forget that transition is a cost)? Those who wish to put together
these proposed branches go ahead and do so, publishing those wherever
they like (git.debian.org, gitorious, their own hosted git repositories,
or even github) and then Carl can just add those as remotes as he sees
fit.

> > What patch tracking workflow should we adopt? Keep sending patches
> > to the mailing list and have a few people who integrate them into
> > master or pu (or even maint/next), as appropriate? Use the Debian
> > bug tracker? Use Sebastian's Roundup instance? Set up a patch queue
> > manager on notmuchmail.org? Use patchwork [1]?

Personally, I've found mailing lists that have patches sent to them
tends to totally kill the list for anything else. It seems a bit weird
to use Debian's bug tracker for a non-Debian native program (but using
it for the Debian package of notmuch does make sense). I am not so
familiar with Roundup, patch queue trackers or patchwork to have
anything to say about those.

micah


pgp3gYDNVEZ4g.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] [patch] store folder information

[micah anderson]

Hi Andreas,

I'm just writing because of the patch you sent to the notmuch list on
December 15th. It seems like many people are wanting this functionality,
I know I am myself and Carl has also indicated the same. However, there
were a couple of minor suggestions for improvements for your patch that
have not seen a reply from you yet. I'm particularly eager to see this
get accepted upstream, and it sounds like the changes necessary to do so
are relatively minor.

I'm wondering what your plans are for addressing these issues? I've come
to depend on this functionality, and would love to see it incorporated
upstream! 

Specifically these were:

1. Unrelated whitespace:

On December 16th,2009 Ruben Pollan  wrote:

> [mes...@blackspot:src/notmuch.orig]$ git apply 
> ~/0001-Preseve-folder-information-when-indexing.patch
> /home/meskio/0001-Preseve-folder-information-when-indexing.patch:136: 
> trailing whitespace.
>status notmuch_database_add_message (notmuch, next,
> /home/meskio/0001-Preseve-folder-information-when-indexing.patch:137: 
> trailing whitespace.
>   folder_base_name,
> warning: 2 lines add whitespace errors.
>
> It's just whitespaces at the end of the lines.

2. An unrelated hunk creeping in:

On Tue, 15 Dec 2009 13:22:19 -0800, Carl Worth  wrote:
> On Mon, 14 Dec 2009 14:21:50 -0500, Andreas Kl=C3=B6ckner  wrote:
> >
> > @@ -116,6 +116,8 @@ skip_re_in_subject (const char *subject)
> > s++;
> > if (strncasecmp (s, "re:", 3) =3D=3D 0)
> > s +=3D 3;
> > +else if (strncasecmp (s, "aw:", 3) =3D=3D 0)
> > +   s +=3D 3;
> > else
> > break;
> >  }
>=20
> This hunk looks unrelated to the rest. Could you submit that separately,
> please?


3. Redundant trailing directory name traversal:

> > +gchar *full_folder_name =3D NULL;
> > +gchar *folder_base_name =3D NULL;
> > +
> > +/* Find name of "folder" containing the email. */
> > +full_folder_name =3D g_strdup(path);
> > +while (1)
> > +{
> > +folder_base_name =3D g_path_get_basename(full_folder_name);
>
> The trailing directory name is available already during the
> traversal. So you don't need to search it back out again. See the patch
> in the following message:
>
>   id:87fx8bygi7@linux.vnet.ibm.com
>
> which simply passes the trailing directory name along, (but skipping a
> name of "cur" or "new" while traversing).

4. supporting hierarchical folders (perhaps this is a later improvement
that does not need to be added before the original patch is accepted?):

> Beyond that, though, I imagine some people have hierarchical folders as
> well, so it probably makes sense to store them as well.
>
> To do that, it's probably just a matter of calling gen_terms on the
> complete filename. I haven't tested, but doing that should allow
> Xapian's phrase searching to do the right thing for something like:
>
>   filename:portion/of/the/path/name

5. Probably the patch needs to be rebased off of master at this point.

Micah


pgpA2MoNAVnNu.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git feature branch

[Ben Gamari]

Excerpts from micah anderson's message of Tue Jan 26 17:24:15 -0500 2010:
> On Mon, 25 Jan 2010 16:46:43 -0800, sebast...@sspaeth.de wrote:
> > > I think it would make sense to move the mainline to git.debian.org
> > > for now, or another place where everyone can easily get an account.
> > > As alternatives I propose repo.or.cz. I'd prefer to stay away from
> > > commercial services like Github.
> 
> I too prefer to stay away from commercial,non-free services like GitHub.
> Gitorious (http://www.gitorious.org/) is a great service, similar to
> Github, but with added freedom.
> 
> Currently the git repository is at git://notmuchmail.org/git/notmuch
> which seems like a fine location for it, but perhaps I missed the
> motivation for switching to a centralized repository with the added
> overhead of giving people access to commit?
> 
> Couldn't all of this be done without moving the existing git repository
> (don't forget that transition is a cost)? Those who wish to put together
> these proposed branches go ahead and do so, publishing those wherever
> they like (git.debian.org, gitorious, their own hosted git repositories,
> or even github) and then Carl can just add those as remotes as he sees
> fit.
> 
I agree. There is no good reason to switch away from the existing
infrastructure. If he wants, Carl can give regular contributors their
own repositories on notmuchmail.org if some people have difficulties
providing it themselves. After all, this is one of the strengths of
distributed version control.

- Ben
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git feature branch

[Jameson Rollins]

On Wed, 27 Jan 2010 11:24:15 +1300, micah anderson  wrote:
> Couldn't all of this be done without moving the existing git repository
> (don't forget that transition is a cost)? Those who wish to put together
> these proposed branches go ahead and do so, publishing those wherever
> they like (git.debian.org, gitorious, their own hosted git repositories,
> or even github) and then Carl can just add those as remotes as he sees
> fit.

I second this idea.  I've been working on quite a few projects that work
in this way, with nothing but personal distributed repos, and it works
quite well.  Especially if Carl is going to continue to be benevolent
dictator (as I agree that he should).  Everyone who has patches can just
post them to their personal repos, making sure that they're well rebased
to Carl's master, and Carl can just pull from them as he sees fit.  It's
what git is designed to do, and it actually works really well.

jamie.


pgpthEyczMsGP.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git feature branch

[martin f krafft]

also sprach micah anderson  [2010.01.27.1124 +1300]:
> Couldn't all of this be done without moving the existing git
> repository (don't forget that transition is a cost)? Those who
> wish to put together these proposed branches go ahead and do so,
> publishing those wherever they like (git.debian.org, gitorious,
> their own hosted git repositories, or even github) and then Carl
> can just add those as remotes as he sees fit.

I brought this up, so I should make the motivation explicit: I just
tried to relieve Carl from the burden of bottleneck. Since passing
out SSH accounts on his own machine does not really scale, I looked
elsewhere.

However, I agree that the easiest solution is just to add more
users. Potentially, Carl can make use of gitolite, I can investigate
that some time this week.

> Personally, I've found mailing lists that have patches sent to
> them tends to totally kill the list for anything else. It seems
> a bit weird to use Debian's bug tracker for a non-Debian native
> program (but using it for the Debian package of notmuch does make
> sense). I am not so familiar with Roundup, patch queue trackers or
> patchwork to have anything to say about those.

patchwork integrates with the mailing list and slurps patches and
related discussion and threads them into a webpage, where they can
be workflow-managed.

The Debian bug tracker has the benefit of being usable with e-mail
(and this is notmuch we're developing, don't forget). The others are
all exclusively web-based, with the exception of launchpad, AFAIK.

The idea of having a tracker is quite simply to make it easier for
Carl to stay on top, and for e.g. you and I to assist him by vetting
patches in our free minutes.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
there's an old proverb that says just about whatever you want it to.
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Backport of Xapian term update optimisation

[Olly Betts]

I've backported the term update optimisation patches
 to Xapian's 1.0 branch, and you can
find snapshot tarballs including these changes here:

http://oligarchy.co.uk/xapian/branches/1.0/

Xapian's testsuite passes (including the additional test coverage which I
also backported), and I looked over each change carefully, but I would be
interested to see some real world testing, particularly in the situation
which these changes are intended to improve (i.e. speed of tagging in
notmuch).

So if you're so inclined, try it and report how you got on (on this list
is fine).

Cheers,
Olly

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

[martin f krafft]

also sprach Jameson Rollins  [2010.01.26.1046 
+1300]:
> > For example, I might have:
> > 
> > ~/.notmuch-config:
> > 
> > [database]
> > path=/home/pioto/mail
> > ...
> > [tags]
> > pi...@pioto.org/INBOX.ListMail.notmuch = notmuch
> > 
> > So, a 'tags' section, where each key is the folder name, relative to the
> > db path, and the value is one or more tag names
> 
> I think this idea is a really good one and I would like to pursue it as
> a tangent thread here.  I was going to propose something very similar to
> this.  I think it's a very flexible idea that would help in a lot of
> ways.

I think we need to carefully distinguish here. The above seems to
suggest a mapping from folder to tag, but we don't actually need
tags for folder locations, because those can (and should) be
implicitly determined from the database and storing the tag in
addition would just run the risk of getting out of sync: if I moved
a message, I would also have to remember to delete old and add new
tags, which is just asking for trouble.

> [tags]
> inbox = +inbox,+unread
> sent = +sent
> drafts = +draft
> archive = -inbox

This proposal, on the other hand, is an interesting one, but when is
it supposed to happen? It just feels wrong to make this happen as
part of 'notmuch new'.

What I would like to see is a notmuch-aware MDA, e.g. a programme
which reads an incoming mail on stdin and can do all this kind of
stuff, e.g. assign tags based on such rules (or take tags as
arguments, so that I could trivially set tags from procmail too),
write the message to the message store, and update the database.

This would allow us to get rid of 'notmuch new' altogether, at least
conceptually. We'd still need it if mail is being delivered
independently, e.g. with offlineimap.

On the performance side, it might make sense to write to a journal
instead of updating the database every time. SpamAssassin does this
with its Bayesian database, and it only merges the journal every
X updates (or when the user manually requests it). I am not sure
whether this is possible with Xapian. On the other hand, I think
notmuch needs to learn to journal anyway so that we can keep
different instances in sync.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"the only way to get rid of a temptation is to yield to it."
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

[martin f krafft]

also sprach Jameson Rollins  [2010.01.27.0603 
+1300]:
> > This is getting involved. 
> > 
> > Maybe I'm missing something in this thread; but, why couldn't these complex 
> > and
> > context-sensitive decisions be delegated to sub-processes? ala git hooks?
> 
> I think this idea is completely independent of anything having to do
> with using git as a mail store.  That's why I was trying to separate it
> into a separate thread.

I think he meant "notmuch hooks like you have hooks for Git too",
e.g. thread:755741d13573c7642761d2a175cb146d

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"if i am occasionally a little overdressed, i make up for it by being
 always immensely over-educated."
-- oscar wilde
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

[James Westby]

On Thu, 28 Jan 2010 18:12:52 +1300, martin f krafft  wrote:
> also sprach Jameson Rollins  [2010.01.27.0603 
> +1300]:
> > > This is getting involved. 
> > > 
> > > Maybe I'm missing something in this thread; but, why couldn't these 
> > > complex and
> > > context-sensitive decisions be delegated to sub-processes? ala git hooks?
> > 
> > I think this idea is completely independent of anything having to do
> > with using git as a mail store.  That's why I was trying to separate it
> > into a separate thread.
> 
> I think he meant "notmuch hooks like you have hooks for Git too",
> e.g. thread:755741d13573c7642761d2a175cb146d

Are you trying to use thread: such that it could be passed to notmuch
show to see the conversation?

That's not going to work so well if so.

Thanks,

James
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

[martin f krafft]

also sprach James Westby  [2010.01.28.1828 +1300]:
> Are you trying to use thread: such that it could be passed to
> notmuch show to see the conversation?
> 
> That's not going to work so well if so.

Why not? Works fine for me with the vim plugin...

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"perfection is achieved, not when there is nothing more to add, but
 when there is nothing left to take away."
 -- antoine de saint-exupéry
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Fwd: Re: [patch] store folder information

[Andreas Klöckner]

As instructed by Micah below, I'm hereby letting the list know I'm no
longer interesting in shepherding that patch.

Best,
Andreas
--  Forwarded Message  --

Betreff: Re: [notmuch] [patch] store folder information
Datum: Mittwoch 27 Januar 2010
Von: micah anderson 
An: Andreas Klöckner 

On Wed, 27 Jan 2010 14:42:39 -0500, Andreas Klöckner  
wrote:
> Hi Micah,
> 
> On Mittwoch 27 Januar 2010, you wrote:
> > I'm wondering what your plans are for addressing these issues? I've come
> > to depend on this functionality, and would love to see it incorporated
> > upstream! 
> 
> That's easy to answer: I don't use notmuch any more, I've caved and gone
> back to KMail.

Ah! That is too bad :(

Would you mind sending the same to the notmuch list in reply to the
thread? I spoke with Carl at the recent LCA and he said he was waiting
for a response from you on that patch before doing anything with it. If
he knows that you are no longer interested in it, then he may take care
of it himself.

micah

---


signature.asc
Description: This is a digitally signed message part.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] tag dir proposal [was: Re: Git as notmuch object store]

[James Westby]

On Thu, 28 Jan 2010 18:34:21 +1300, martin f krafft  wrote:
> also sprach James Westby  [2010.01.28.1828 +1300]:
> > Are you trying to use thread: such that it could be passed to
> > notmuch show to see the conversation?
> > 
> > That's not going to work so well if so.
> 
> Why not? Works fine for me with the vim plugin...

lib/message.cc:560

static void   
thread_id_generate (thread_id_t *thread_id)   
{
static int seeded = 0;
FILE *dev_random;
uint32_t value;   
char *s;
int i;
  
if (! seeded) {
dev_random = fopen ("/dev/random", "r");  
if (dev_random == NULL) {
srand (time (NULL));  
} else {
fread ((void *) &value, sizeof (value), 1, dev_random);   
srand (value);
fclose (dev_random); 
}
seeded = 1;
} 

s = thread_id->str;
for (i = 0; i < NOTMUCH_THREAD_ID_DIGITS; i += 8) {   
value = rand ();
sprintf (s, "%08x", value);   
s += 8;   
}
}

so it works fine for you, however I have no idea which thread you are
talking about.

Thanks,

James
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] Git feature branch

[James Rowe]

* martin f krafft (madd...@madduck.net) wrote:
> also sprach micah anderson  [2010.01.27.1124 +1300]:
> > Personally, I've found mailing lists that have patches sent to
> > them tends to totally kill the list for anything else. It seems
> > a bit weird to use Debian's bug tracker for a non-Debian native
> > program (but using it for the Debian package of notmuch does make
> > sense). I am not so familiar with Roundup, patch queue trackers or
> > patchwork to have anything to say about those.
>
> patchwork integrates with the mailing list and slurps patches and
> related discussion and threads them into a webpage, where they can
> be workflow-managed.
>
> The Debian bug tracker has the benefit of being usable with e-mail
> (and this is notmuch we're developing, don't forget). The others are
> all exclusively web-based, with the exception of launchpad, AFAIK.

  As I use some of the other options...

  Roundup has command line and email interfaces.  The email interface is
quite similar to debian's.  I've never used a launchpad hosted project
so I can't compare it.

  Google's codereview tool has a nice interface for collecting and
commenting on patches, but I suspect that suggestion will also meet with
a degree of friction.  To me codereview feels like patchwork with
polish.

  Both gitorious and github have commenting functionality built in.
Commenting on commits in a fork is as easy as opening the commit in
a browser.  I use something along the lines of the following script to
open commits on github:

#! /bin/sh
BASE=$(git config remote.${2:-origin}.url | sed 
's,git\(@\|://\)\([^:/]*\)[:/]\(.*\).git,http://\2/\3/commit,')
COMMIT=$(git rev-parse ${1:-HEAD})
sensible-browser ${BASE}/${COMMIT}

  Using github or gitorious you can easily find and track forks from one
place as well, which makes discovering new work much easier.  Github
even provides a pretty single page interface to the work going on in
other forks, gitorious requires a little more leg work to do the same
but not much.

  For a couple of hosted projects we use at the office we email the
individual entries from http://github.com/$user/$project/comments.atom
to the mailing list so they're /forcibly/ seen by everybody :)

Thanks,

James



pgpIpxyv08ZLr.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch