Experimental Tag Sharing

2011-10-30 Thread David Bremner 
On Sat, 29 Oct 2011 17:44:56 -0700, Jameson Graef Rollins  wrote:
> On Sat, 29 Oct 2011 13:45:06 -0300, David Bremner  
> wrote:
> 
> These patches did not apply cleanly for me from the list.  I was able to
> get them from bremner's "nmbug" branch [0], though.
> [0] git://pivot.cs.unb.ca/git/notmuch

I did resend the one patch that we figured out was corrupted, but for
the moment I will just rebase the nmbug branch that jrollins mentioned.

> > $ nmbug dump
> > $ cd $HOME/.nmbug && git add tags && git commit
[snip]

> Could they be merged into a single operation?

Yes, "commit" now (in my git repo) does both the dump and the git
commit.

> While I think having a central shared tag repo is ok to get started, I
> would really like to see this work in a distributed way.

So far I don't see any technical issues with working in a distributed
way, just social ones. Whatever organization we set up, I think it is
important for there to be a way to submit and manage bug reports for
"outsiders". Of course, requiring a notmuch install is already a burden.

> It might be nice to let the user configure what prefix they use
> (ie. what goes before the "::" (currently "notmuch")).

There are two simple-matters-of-programming here. One is how to do the
configuration; probably "nmbug config" can be an alias for "git config
nmbug.$1" or something. The other is translating between prefixes.
Neither is hard, I guess.

> Does this system support the removing of tags?  I guess I need to see
> the documentation for the restore --match option...

Yes, barring bugs, removing a tag in notmuch and running "nmbug commit"
should remove from the files in ~/.nmbug/tags

> It's too bad there's not a way to do scoped tag searches in notmuch
> (ie. notmuch search tag:"notmuch::*").  Any idea how hard it would be to
> support something like that?

I don't know. Carl thought it should work, but it doesn't seem to. 
My uneducated guess is it has something to do with query parsing.

> > On think I'm not sure about is how to handle ideas like "voting", and
> > "signing off" with tags. I suspect we can come up with some

> What issue are you trying to solve here?  Whatever it is, I don't think
> I like enumerating tags as the solution.

I'm trying to handle the issue of sharing non-boolean data. 
For example, 
 - how can we record priorities bugs/patches
 - how can we conveniently find all of the emacs patches (b.t.w I
   started using notmuch::emacs for those) that jrollins has endorsed
   but are not pushed.

Of course, nmbug could share more metadata than tags. OTOH I'd like to
avoid completely reinventing a distributed [0] bug tracker.

David


[0] well, at least potentially distributed, depending on definitions.





-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 315 bytes
Desc: not available
URL:

Experimental Tag Sharing

2011-10-29 Thread David Bremner 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

OK, here is my rough and ready attempt at tag sharing.  I figure we
can smooth out the rough edges if/when we agree on a set of tags and
preferably on an on-disk format.

How to play?

- - Apply all the patches in this thread (starting
  id:"1319383133-11006-1-git-send-email-david at tethera.net")

- - to git master git clone git://pivot.cs.unb.ca/nmbug.git $HOME/.nmbug

- - put the script nmbug somewhere

# suck in the tags
$ nmbug restore

[ do some tagging ]

$ nmbug dump
$ cd $HOME/.nmbug && git add tags && git commit

now you have to get the changes into the "master" repo.  I think
hosting this on git.notmuchmail.org in a seperate repo will make
sense, but for now, if you want to participate in the experiment, send
me your public key (ideally in a gpg signed mail, but if not, oh well), 
and you can push to my repo at

git at pivot.cs.unb.ca:pub/nmbug

Probably the most crucial thing is that we agree on some set of tags.
Here is the set of tags I am working with (output from 
 notmuch search --output=tags "*" | grep ^notmuch:: 
)

notmuch::bug is a bug report
notmuch::feature provides a new feature
notmuch::fix fixes a bug
notmuch::obsoletereplaced by some other patch
notmuch::patch   
notmuch::portability improves portability
notmuch::pushed  is pushed to master
notmuch::review  is a review
notmuch::reviewedhas been (well reviewed)
notmuch::testprovides a new test/or improves testing

The prefix "notmuch::" is hardcoded into the script, but everything
else is easy to change with the usual notmuch operations.  Of course
we can change the prefix too. I decided on :: because it needed less
escaping.

On think I'm not sure about is how to handle ideas like "voting", and
"signing off" with tags. I suspect we can come up with some
conventions like "notmuch::review1", or
"notmuch::vote1".."notmuch::vote5". But maybe people have less kludgy
ideas.

Remember, perfect is the enemy of good ;)

David
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iJwEAQECAAYFAk6sLWQACgkQTiiN/0Um85k5JQP+PZ0ycrNTTZZ72PyXbDQzrv4+
dDKopxh+WpeORjWu8gvj+LNMM3A8n0t4/A12mxrG9cUJxg2HTPUtRo9Zi7dFQzct
uGvSOvQ6GocUretW0BaK/yp53v0R7b00qxvxAvUeoZPgIWGtqp52fXW8d0I2yAUn
079okQwPYdbowaPAX1A=
=+E8F
-END PGP SIGNATURE-

Experimental Tag Sharing

2011-10-29 Thread David Bremner 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

OK, here is my rough and ready attempt at tag sharing.  I figure we
can smooth out the rough edges if/when we agree on a set of tags and
preferably on an on-disk format.

How to play?

- - Apply all the patches in this thread (starting
  id:1319383133-11006-1-git-send-email-da...@tethera.net)

- - to git master git clone git://pivot.cs.unb.ca/nmbug.git $HOME/.nmbug

- - put the script nmbug somewhere

# suck in the tags
$ nmbug restore

[ do some tagging ]

$ nmbug dump
$ cd $HOME/.nmbug  git add tags  git commit

now you have to get the changes into the master repo.  I think
hosting this on git.notmuchmail.org in a seperate repo will make
sense, but for now, if you want to participate in the experiment, send
me your public key (ideally in a gpg signed mail, but if not, oh well), 
and you can push to my repo at

g...@pivot.cs.unb.ca:pub/nmbug

Probably the most crucial thing is that we agree on some set of tags.
Here is the set of tags I am working with (output from 
 notmuch search --output=tags * | grep ^notmuch:: 
)

notmuch::bug is a bug report
notmuch::feature provides a new feature
notmuch::fix fixes a bug
notmuch::obsoletereplaced by some other patch
notmuch::patch   
notmuch::portability improves portability
notmuch::pushed  is pushed to master
notmuch::review  is a review
notmuch::reviewedhas been (well reviewed)
notmuch::testprovides a new test/or improves testing

The prefix notmuch:: is hardcoded into the script, but everything
else is easy to change with the usual notmuch operations.  Of course
we can change the prefix too. I decided on :: because it needed less
escaping.

On think I'm not sure about is how to handle ideas like voting, and
signing off with tags. I suspect we can come up with some
conventions like notmuch::review1, or
notmuch::vote1..notmuch::vote5. But maybe people have less kludgy
ideas.

Remember, perfect is the enemy of good ;)

David
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iJwEAQECAAYFAk6sLWQACgkQTiiN/0Um85k5JQP+PZ0ycrNTTZZ72PyXbDQzrv4+
dDKopxh+WpeORjWu8gvj+LNMM3A8n0t4/A12mxrG9cUJxg2HTPUtRo9Zi7dFQzct
uGvSOvQ6GocUretW0BaK/yp53v0R7b00qxvxAvUeoZPgIWGtqp52fXW8d0I2yAUn
079okQwPYdbowaPAX1A=
=+E8F
-END PGP SIGNATURE-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Experimental Tag Sharing

2011-10-29 Thread Jameson Graef Rollins 
On Sat, 29 Oct 2011 13:45:06 -0300, David Bremner da...@tethera.net wrote:
 OK, here is my rough and ready attempt at tag sharing.  I figure we
 can smooth out the rough edges if/when we agree on a set of tags and
 preferably on an on-disk format.

Great!  Thank you so much for working on this, David.  This is very
exciting work.

Comments below.

 How to play?
 
 - - Apply all the patches in this thread (starting
   id:1319383133-11006-1-git-send-email-da...@tethera.net)

These patches did not apply cleanly for me from the list.  I was able to
get them from bremner's nmbug branch [0], though.

[0] git://pivot.cs.unb.ca/git/notmuch

 $ nmbug dump
 $ cd $HOME/.nmbug  git add tags  git commit

This last command seems to be included in nmbug as commit.  Fwiw, I
don't like having to do two separate operations for dump and commit.
Could they be merged into a single operation?

 now you have to get the changes into the master repo.  I think
 hosting this on git.notmuchmail.org in a seperate repo will make
 sense, but for now, if you want to participate in the experiment, send
 me your public key (ideally in a gpg signed mail, but if not, oh well), 
 and you can push to my repo at
 
 g...@pivot.cs.unb.ca:pub/nmbug

While I think having a central shared tag repo is ok to get started, I
would really like to see this work in a distributed way.  I don't think
it's impossible to extend this model you have here to work in a
distributed way, though, so that will be something good to work on down
the line.

 Probably the most crucial thing is that we agree on some set of tags.
 Here is the set of tags I am working with (output from 
  notmuch search --output=tags * | grep ^notmuch:: 
 )
 
 notmuch::bug   is a bug report
 notmuch::feature   provides a new feature
 notmuch::fix   fixes a bug
 notmuch::obsolete  replaced by some other patch
 notmuch::patch 
 notmuch::portability   improves portability
 notmuch::pushedis pushed to master
 notmuch::reviewis a review
 notmuch::reviewed  has been (well reviewed)
 notmuch::test  provides a new test/or improves testing
 
 The prefix notmuch:: is hardcoded into the script, but everything
 else is easy to change with the usual notmuch operations.  Of course
 we can change the prefix too. I decided on :: because it needed less
 escaping.

I think these are a fine set to start with.

It might be nice to let the user configure what prefix they use
(ie. what goes before the :: (currently notmuch)).

Does this system support the removing of tags?  I guess I need to see
the documentation for the restore --match option...

It's too bad there's not a way to do scoped tag searches in notmuch
(ie. notmuch search tag:notmuch::*).  Any idea how hard it would be to
support something like that?

 On think I'm not sure about is how to handle ideas like voting, and
 signing off with tags. I suspect we can come up with some
 conventions like notmuch::review1, or
 notmuch::vote1..notmuch::vote5. But maybe people have less kludgy
 ideas.

What issue are you trying to solve here?  Whatever it is, I don't think
I like enumerating tags as the solution.

Again, thanks so much for working on this, David.  This is going to be
very cool.  I really look forward to fleshing it out, and using it!

jamie.


pgpTuchttBmoH.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Experimental Tag Sharing

2011-10-29 Thread David Bremner 
On Sat, 29 Oct 2011 17:44:56 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 On Sat, 29 Oct 2011 13:45:06 -0300, David Bremner da...@tethera.net wrote:
 
 These patches did not apply cleanly for me from the list.  I was able to
 get them from bremner's nmbug branch [0], though.
 [0] git://pivot.cs.unb.ca/git/notmuch

I did resend the one patch that we figured out was corrupted, but for
the moment I will just rebase the nmbug branch that jrollins mentioned.

  $ nmbug dump
  $ cd $HOME/.nmbug  git add tags  git commit
[snip]

 Could they be merged into a single operation?

Yes, commit now (in my git repo) does both the dump and the git
commit.

 While I think having a central shared tag repo is ok to get started, I
 would really like to see this work in a distributed way.

So far I don't see any technical issues with working in a distributed
way, just social ones. Whatever organization we set up, I think it is
important for there to be a way to submit and manage bug reports for
outsiders. Of course, requiring a notmuch install is already a burden.

 It might be nice to let the user configure what prefix they use
 (ie. what goes before the :: (currently notmuch)).

There are two simple-matters-of-programming here. One is how to do the
configuration; probably nmbug config can be an alias for git config
nmbug.$1 or something. The other is translating between prefixes.
Neither is hard, I guess.

 Does this system support the removing of tags?  I guess I need to see
 the documentation for the restore --match option...

Yes, barring bugs, removing a tag in notmuch and running nmbug commit
should remove from the files in ~/.nmbug/tags

 It's too bad there's not a way to do scoped tag searches in notmuch
 (ie. notmuch search tag:notmuch::*).  Any idea how hard it would be to
 support something like that?

I don't know. Carl thought it should work, but it doesn't seem to. 
My uneducated guess is it has something to do with query parsing.

  On think I'm not sure about is how to handle ideas like voting, and
  signing off with tags. I suspect we can come up with some

 What issue are you trying to solve here?  Whatever it is, I don't think
 I like enumerating tags as the solution.

I'm trying to handle the issue of sharing non-boolean data. 
For example, 
 - how can we record priorities bugs/patches
 - how can we conveniently find all of the emacs patches (b.t.w I
   started using notmuch::emacs for those) that jrollins has endorsed
   but are not pushed.

Of course, nmbug could share more metadata than tags. OTOH I'd like to
avoid completely reinventing a distributed [0] bug tracker.

David


[0] well, at least potentially distributed, depending on definitions.







pgpzv0CvVuq14.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

tag sharing

2011-10-07 Thread Jesse Rosenthal 
On Fri, 07 Oct 2011 07:36:30 -0400, Jesse Rosenthal  
wrote:
> What's the value added over just keeping a (compressed?) collection of
> diffs for each namespace?

Okay -- please don't bother answering this part. It was early in the
morning, and I forgot some of the obvious advantages of real version
control (we can roll back to a previous moment, etc). We still don't
need most of what git offers, but if it's the best tool for keeping
track of the move from the file in step (5) to the file in step (6),
then sure, let's use it.

tag sharing

2011-10-07 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 11:18:51 -0300, David Bremner  wrote:
Non-text part: multipart/signed
> On Thu, 06 Oct 2011 09:21:48 -0400, Jesse Rosenthal  
> wrote:
> > morning's project. In retrospect, I think the main issue was that I was
> > trying to figure out how history would be kept. By using git, though,
> > your idea would be that we get history for free, right?
> 
> Yeah. Of course, one would have to decide if the history of the current
> dump file format is intelligible, or if some alternative format should
> be used.

I've managed to reconstruct a bit of my thinking. I think it could work
with the current format if we went as follows. I also think we could use
git, but I'm not sure that wouldn't be overkill. Some of the steps are
in extreme slow motion, just to make sure we're on the same page. (I've
also replaced the real msg-ids to cut down on noise).

  1. David dumps the targeted messages (tags in namespace "my"), so we
 get a file like this:

  msg-id-1 at example.1 (my.foo inbox)
  msg-id-2 at example.2 (answered my.to-do inbox signed)

  2. David strips out everything but tags in the namespace and removes
 the namespace:

  msg-id-1 at example.1 (foo)
  msg-id-2 at example.2 (to-do)

  3. David makes it available, Jesse pulls.

  4. Jesse dumps the namespace he has associated with David ("bremner"):

  msg-id-3 at example.3 (bremner.bar)
  msg-id-4 at example.4 (bremner.bar unread)
  msg-id-1 at example.1 (bremner.foo inbox)
  msg-id-5 at example.5 (bremner.wishlist inbox)

  5. Jesse strips the tags:

  msg-id-3 at example.3 (bar)
  msg-id-4 at example.4 (bar)
  msg-id-1 at example.1 (foo)
  msg-id-5 at example.5 (wishlist)

  6. Jesse replaces this list with David's:

  msg-id-1 at example.1 (foo)
  msg-id-2 at example.2 (to-do)

  7. Jesse removes all tags with the "bremner" namespace from his
 database.

  8. Jesse adds the tags from the list in step (6), with the appropriate
 namespace, to his list of tags. The messages of interest now look like:

  msg-id-3 at example.3 ()
  msg-id-4 at example.4 (unread)
  msg-id-1 at example.1 (bremner.foo inbox)
  msg-id-5 at example.5 (inbox)
  msg-id-2 at example.2 (bremner.to-do someothertag)

  9. Jesse runs `notmuch restore`

The reason I went through this slowly is to highlight the fact that the
only thing that needs to be recorded in diffs somehow (in git, let's
say) are the files in steps (5) and (6). So if we integrated this with
git, there would be something along the lines of a commit (on Jesse's
computer) in step (5) and then another commit (again on Jesse's
computer) in step (6). 

Note that we don't actually need (5) to do tag-sharing. The only reason
it's there is for keeping history.

I'm pretty sure that we don't need a commit on David's computer in step
(1), because what the history would record is when the tag-sharing
utility changed your tags, not when you did.

Note also that all we would really be asking git to do here is keep an
ordered collection of diffs of a single file. Honest question: since
this is the barest possible form of version control, is git necessary?
Some of the features, like commit messages, don't seem to fit well with
this model; and others, like branching, seem pretty useless. What's the
value added over just keeping a (compressed?) collection of diffs for
each namespace?

Best,
Jesse

Re: tag sharing

2011-10-07 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 11:18:51 -0300, David Bremner da...@tethera.net wrote:
Non-text part: multipart/signed
 On Thu, 06 Oct 2011 09:21:48 -0400, Jesse Rosenthal jrosent...@jhu.edu 
 wrote:
  morning's project. In retrospect, I think the main issue was that I was
  trying to figure out how history would be kept. By using git, though,
  your idea would be that we get history for free, right?
 
 Yeah. Of course, one would have to decide if the history of the current
 dump file format is intelligible, or if some alternative format should
 be used.

I've managed to reconstruct a bit of my thinking. I think it could work
with the current format if we went as follows. I also think we could use
git, but I'm not sure that wouldn't be overkill. Some of the steps are
in extreme slow motion, just to make sure we're on the same page. (I've
also replaced the real msg-ids to cut down on noise).

  1. David dumps the targeted messages (tags in namespace my), so we
 get a file like this:
  
  msg-id-1@example.1 (my.foo inbox)
  msg-id-2@example.2 (answered my.to-do inbox signed)

  2. David strips out everything but tags in the namespace and removes
 the namespace:

  msg-id-1@example.1 (foo)
  msg-id-2@example.2 (to-do)

  3. David makes it available, Jesse pulls.

  4. Jesse dumps the namespace he has associated with David (bremner):

  msg-id-3@example.3 (bremner.bar)
  msg-id-4@example.4 (bremner.bar unread)
  msg-id-1@example.1 (bremner.foo inbox)
  msg-id-5@example.5 (bremner.wishlist inbox)

  5. Jesse strips the tags:

  msg-id-3@example.3 (bar)
  msg-id-4@example.4 (bar)
  msg-id-1@example.1 (foo)
  msg-id-5@example.5 (wishlist)

  6. Jesse replaces this list with David's:

  msg-id-1@example.1 (foo)
  msg-id-2@example.2 (to-do)

  7. Jesse removes all tags with the bremner namespace from his
 database.

  8. Jesse adds the tags from the list in step (6), with the appropriate
 namespace, to his list of tags. The messages of interest now look like:

  msg-id-3@example.3 ()
  msg-id-4@example.4 (unread)
  msg-id-1@example.1 (bremner.foo inbox)
  msg-id-5@example.5 (inbox)
  msg-id-2@example.2 (bremner.to-do someothertag)

  9. Jesse runs `notmuch restore`

The reason I went through this slowly is to highlight the fact that the
only thing that needs to be recorded in diffs somehow (in git, let's
say) are the files in steps (5) and (6). So if we integrated this with
git, there would be something along the lines of a commit (on Jesse's
computer) in step (5) and then another commit (again on Jesse's
computer) in step (6). 

Note that we don't actually need (5) to do tag-sharing. The only reason
it's there is for keeping history.

I'm pretty sure that we don't need a commit on David's computer in step
(1), because what the history would record is when the tag-sharing
utility changed your tags, not when you did.

Note also that all we would really be asking git to do here is keep an
ordered collection of diffs of a single file. Honest question: since
this is the barest possible form of version control, is git necessary?
Some of the features, like commit messages, don't seem to fit well with
this model; and others, like branching, seem pretty useless. What's the
value added over just keeping a (compressed?) collection of diffs for
each namespace?

Best,
Jesse
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-07 Thread Jesse Rosenthal 
On Fri, 07 Oct 2011 07:36:30 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote:
 What's the value added over just keeping a (compressed?) collection of
 diffs for each namespace?

Okay -- please don't bother answering this part. It was early in the
morning, and I forgot some of the obvious advantages of real version
control (we can roll back to a previous moment, etc). We still don't
need most of what git offers, but if it's the best tool for keeping
track of the move from the file in step (5) to the file in step (6),
then sure, let's use it.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

tag sharing

2011-10-06 Thread David Bremner 
On Thu, 06 Oct 2011 16:40:00 -0400, Jesse Rosenthal  
wrote:
> On Thu, 06 Oct 2011 17:23:21 -0300, David Bremner  
> wrote:
> > What doesn't work is searches for the whole namespace "notmuch search
> > tag:bremner.*" will return nothing, even though "notmuch search
> > tag:bremner.to-fix" does.
> 
> A simple shell way to do this would be
> 
> notmuch search-tags | grep "^bremner\." | xargs -I {} notmuch search 
> tag:{}
> 

Ah thanks, that is not so bad. It turns out to be literally a one line
change to add a query argument for notmuch-dump, _except_ that notmuch
dump already uses the first command line argument as an output file
name. I think I will start a seperate thread about that. If you want to
test, this works OK, except the output is put in a file named after the
query ;).

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..341207a 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -40,7 +40,7 @@ notmuch_dump_command (unused (void *ctx), int argc, char 
*argv[])
 if (notmuch == NULL)
return 1;

-query = notmuch_query_create (notmuch, "");
+query = notmuch_query_create (notmuch, argc > 0 ? argv[0] : "");
 if (query == NULL) {
fprintf (stderr, "Out of memory\n");
return 1;

tag sharing

2011-10-06 Thread David Bremner 
On Thu, 06 Oct 2011 15:49:10 -0400, Jesse Rosenthal  
wrote:
> 
> A period works: `tag:bremner.to-fix`. That would fit with java/python
> conventions. Double-colons also seem to work, for what it's worth:
> `tag:bremner::to-fix`

Right, I was confused and/or unclear. What doesn't work is searches for
the whole namespace "notmuch search tag:bremner.*" will return nothing,
even though "notmuch search tag:bremner.to-fix" does.

d

tag sharing

2011-10-06 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 17:23:21 -0300, David Bremner  wrote:
> What doesn't work is searches for the whole namespace "notmuch search
> tag:bremner.*" will return nothing, even though "notmuch search
> tag:bremner.to-fix" does.

A simple shell way to do this would be

notmuch search-tags | grep "^bremner\." | xargs -I {} notmuch search tag:{}

That's pretty quick over a fair amount of messages. It would be quicker,
I imagine, if it did the grepping inside of a program, but this is still
way down there in the subsecond range.

NB. I don't have that many tags, so maybe the grepping step would bog
down a bit if I had more. But I doubt someone would have that much more
than a few hundred.

tag sharing

2011-10-06 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 11:18:51 -0300, David Bremner  wrote:
> something like that sounds plausible. Currently the query parser doesn't
> handle searches like "tag:bremner/to-fix" very well, because it
> helpfully splits at '/' (aiui; maybe somebody else can explain it
> better).  "notmuch search "tag:notmuch*" does not seem to work either,
> so my first idea to kludge around the problem by using
> e.g. "notmuchPushed" doesn't seem to work either.

A period works: `tag:bremner.to-fix`. That would fit with java/python
conventions. Double-colons also seem to work, for what it's worth:
`tag:bremner::to-fix`

tag sharing

2011-10-06 Thread David Bremner 
On Thu, 06 Oct 2011 09:21:48 -0400, Jesse Rosenthal  
wrote:

> morning's project. In retrospect, I think the main issue was that I was
> trying to figure out how history would be kept. By using git, though,
> your idea would be that we get history for free, right?

Yeah. Of course, one would have to decide if the history of the current
dump file format is intelligible, or if some alternative format should
be used.

> of git? I.e., all we need something to just keep different dumps in
> different namespaces, so that when I pull your dump, the tags come in as
> `bremner/to-fix` (or whatever our separator is), and then when I dump
> all `bremner` tags, it removes the namespace and puts it in
> `bremner.tagfile`.

something like that sounds plausible. Currently the query parser doesn't
handle searches like "tag:bremner/to-fix" very well, because it
helpfully splits at '/' (aiui; maybe somebody else can explain it
better).  "notmuch search "tag:notmuch*" does not seem to work either,
so my first idea to kludge around the problem by using
e.g. "notmuchPushed" doesn't seem to work either.

> So using
> either the library or the bindings would probably speed things up quite
> a bit, by only dumping the tags you care about. Which means it couldn't
> just be a shell script: it would have to be c/ruby/python.

Without looking at the code, I'd guess it wouldn't be too hard to have
notmuch-dump take a standard search term; it's just as I mentioned I
don't see an elegant way of doing what we want with a standard search
term. Of course I'd be happy to be proved wrong.

David



-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 315 bytes
Desc: not available
URL:

tag sharing

2011-10-06 Thread David Bremner 
On Wed, 08 Jun 2011 10:46:57 -0700, Jameson Graef Rollins  wrote:

> On Mon, 06 Jun 2011 13:20:00 -0400, Jesse Rosenthal  
> wrote:
> > After a conversation with David last year about bug-tracking, I worked
> > up a rough python-based prototype of this. It worked in terms of
> > namespaces, so Carl could associate the namespace "public" with a list
> > of tags he publishes to a http-accessable location. And you could
> > associate the namespace "cworth" with those same tags.
> 
> This sounds very cool, Jesse.  Very git-like.  I like it.
> 

Warning: git fanboi-ism ahead.

I like the concept too. But I worry a bit that we are re-inventing
things that could be done by git.  What do people think about a similar
namespace import export tool where we rely on git for a transport layer,
and for merging, conflict-resolution, and history tracking. Of course
there are details to be settled, but before I try to work them out,
would anyone like to convince me the idea is fundementally wrong headed?
Jesse, do you remember why you decided to roll your own?  I agree it is
subjective what is easier, but I think we have enough people comfortable
with git that collaborative maintenence of a bug-tag git repo would be
pretty easy.

d

tag sharing

2011-10-06 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 09:56:43 -0300, David Bremner  wrote:
> Jesse, do you remember why you decided to roll your own?

Only reason I can remember (a year and a half later) of is that it
seemed like the basic illustration of concept would be a saturday
morning's project. In retrospect, I think the main issue was that I was
trying to figure out how history would be kept. By using git, though,
your idea would be that we get history for free, right?

I think my format was:

 
12345  foo at barto-fix
13456  bar at foofeature-request

But using git would get us timestamps for free too, I guess.  And if we
get rid of timestamps, then we just have dumps. So what we would really
just need is a thin namespacing layer in some scripting language on top
of git? I.e., all we need something to just keep different dumps in
different namespaces, so that when I pull your dump, the tags come in as
`bremner/to-fix` (or whatever our separator is), and then when I dump
all `bremner` tags, it removes the namespace and puts it in
`bremner.tagfile`.

One rather obvious thing that I discovered: processing dumps and
restores from the ui is pretty slow, because you have to output a huge
file, and then find the tags in your preferred namespace. So using
either the library or the bindings would probably speed things up quite
a bit, by only dumping the tags you care about. Which means it couldn't
just be a shell script: it would have to be c/ruby/python.

Best,
Jesse

Re: tag sharing

2011-10-06 Thread David Bremner 
On Wed, 08 Jun 2011 10:46:57 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:

 On Mon, 06 Jun 2011 13:20:00 -0400, Jesse Rosenthal jrosent...@jhu.edu 
 wrote:
  After a conversation with David last year about bug-tracking, I worked
  up a rough python-based prototype of this. It worked in terms of
  namespaces, so Carl could associate the namespace public with a list
  of tags he publishes to a http-accessable location. And you could
  associate the namespace cworth with those same tags.
 
 This sounds very cool, Jesse.  Very git-like.  I like it.
 

Warning: git fanboi-ism ahead.

I like the concept too. But I worry a bit that we are re-inventing
things that could be done by git.  What do people think about a similar
namespace import export tool where we rely on git for a transport layer,
and for merging, conflict-resolution, and history tracking. Of course
there are details to be settled, but before I try to work them out,
would anyone like to convince me the idea is fundementally wrong headed?
Jesse, do you remember why you decided to roll your own?  I agree it is
subjective what is easier, but I think we have enough people comfortable
with git that collaborative maintenence of a bug-tag git repo would be
pretty easy.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-06 Thread David Bremner 
On Thu, 06 Oct 2011 09:21:48 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote:

 morning's project. In retrospect, I think the main issue was that I was
 trying to figure out how history would be kept. By using git, though,
 your idea would be that we get history for free, right?

Yeah. Of course, one would have to decide if the history of the current
dump file format is intelligible, or if some alternative format should
be used.

 of git? I.e., all we need something to just keep different dumps in
 different namespaces, so that when I pull your dump, the tags come in as
 `bremner/to-fix` (or whatever our separator is), and then when I dump
 all `bremner` tags, it removes the namespace and puts it in
 `bremner.tagfile`.

something like that sounds plausible. Currently the query parser doesn't
handle searches like tag:bremner/to-fix very well, because it
helpfully splits at '/' (aiui; maybe somebody else can explain it
better).  notmuch search tag:notmuch* does not seem to work either,
so my first idea to kludge around the problem by using
e.g. notmuchPushed doesn't seem to work either.

 So using
 either the library or the bindings would probably speed things up quite
 a bit, by only dumping the tags you care about. Which means it couldn't
 just be a shell script: it would have to be c/ruby/python.

Without looking at the code, I'd guess it wouldn't be too hard to have
notmuch-dump take a standard search term; it's just as I mentioned I
don't see an elegant way of doing what we want with a standard search
term. Of course I'd be happy to be proved wrong.

David





pgprd6P3nNjJz.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-06 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 09:56:43 -0300, David Bremner da...@tethera.net wrote:
 Jesse, do you remember why you decided to roll your own?

Only reason I can remember (a year and a half later) of is that it
seemed like the basic illustration of concept would be a saturday
morning's project. In retrospect, I think the main issue was that I was
trying to figure out how history would be kept. By using git, though,
your idea would be that we get history for free, right?

I think my format was:

TIMESTAMPMESSAGID TAG
12345  foo@barto-fix
13456  bar@foofeature-request

But using git would get us timestamps for free too, I guess.  And if we
get rid of timestamps, then we just have dumps. So what we would really
just need is a thin namespacing layer in some scripting language on top
of git? I.e., all we need something to just keep different dumps in
different namespaces, so that when I pull your dump, the tags come in as
`bremner/to-fix` (or whatever our separator is), and then when I dump
all `bremner` tags, it removes the namespace and puts it in
`bremner.tagfile`.

One rather obvious thing that I discovered: processing dumps and
restores from the ui is pretty slow, because you have to output a huge
file, and then find the tags in your preferred namespace. So using
either the library or the bindings would probably speed things up quite
a bit, by only dumping the tags you care about. Which means it couldn't
just be a shell script: it would have to be c/ruby/python.

Best,
Jesse


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-06 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 11:18:51 -0300, David Bremner da...@tethera.net wrote:
 something like that sounds plausible. Currently the query parser doesn't
 handle searches like tag:bremner/to-fix very well, because it
 helpfully splits at '/' (aiui; maybe somebody else can explain it
 better).  notmuch search tag:notmuch* does not seem to work either,
 so my first idea to kludge around the problem by using
 e.g. notmuchPushed doesn't seem to work either.

A period works: `tag:bremner.to-fix`. That would fit with java/python
conventions. Double-colons also seem to work, for what it's worth:
`tag:bremner::to-fix`
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-06 Thread David Bremner 
On Thu, 06 Oct 2011 15:49:10 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote:
 
 A period works: `tag:bremner.to-fix`. That would fit with java/python
 conventions. Double-colons also seem to work, for what it's worth:
 `tag:bremner::to-fix`

Right, I was confused and/or unclear. What doesn't work is searches for
the whole namespace notmuch search tag:bremner.* will return nothing,
even though notmuch search tag:bremner.to-fix does.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-06 Thread Jesse Rosenthal 
On Thu, 06 Oct 2011 17:23:21 -0300, David Bremner da...@tethera.net wrote:
 What doesn't work is searches for the whole namespace notmuch search
 tag:bremner.* will return nothing, even though notmuch search
 tag:bremner.to-fix does.

A simple shell way to do this would be

notmuch search-tags | grep ^bremner\. | xargs -I {} notmuch search tag:{}

That's pretty quick over a fair amount of messages. It would be quicker,
I imagine, if it did the grepping inside of a program, but this is still
way down there in the subsecond range.

NB. I don't have that many tags, so maybe the grepping step would bog
down a bit if I had more. But I doubt someone would have that much more
than a few hundred.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing

2011-10-06 Thread David Bremner 
On Thu, 06 Oct 2011 16:40:00 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote:
 On Thu, 06 Oct 2011 17:23:21 -0300, David Bremner da...@tethera.net wrote:
  What doesn't work is searches for the whole namespace notmuch search
  tag:bremner.* will return nothing, even though notmuch search
  tag:bremner.to-fix does.
 
 A simple shell way to do this would be
 
 notmuch search-tags | grep ^bremner\. | xargs -I {} notmuch search 
 tag:{}
 

Ah thanks, that is not so bad. It turns out to be literally a one line
change to add a query argument for notmuch-dump, _except_ that notmuch
dump already uses the first command line argument as an output file
name. I think I will start a seperate thread about that. If you want to
test, this works OK, except the output is put in a file named after the
query ;).

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..341207a 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -40,7 +40,7 @@ notmuch_dump_command (unused (void *ctx), int argc, char 
*argv[])
 if (notmuch == NULL)
return 1;
 
-query = notmuch_query_create (notmuch, );
+query = notmuch_query_create (notmuch, argc  0 ? argv[0] : );
 if (query == NULL) {
fprintf (stderr, Out of memory\n);
return 1;
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-08 Thread Jesse Rosenthal 
On Wed, 08 Jun 2011 10:46:57 -0700, Jameson Graef Rollins  wrote:
> Did you guys try to address the issue of tag removal at all?  I've been
> trying to decide if this is something we need to worry about or not.
> For instance, if cworth pushed a tag ".needs-review", you would probably
> want to have that tag removed if cworth removed it.  I guess
> alternatively he could just push the tag ".reviewed" to nullify the
> meaning of the previous one.  I'm not sure that would work in all cases,
> though.

Yes, if he deletes the tag "public.needs-review" in his notmuch, and
then pushes it, it will delete the tag "cworth.needs-review" from yours.

A couple points about this:

1. If you added your own "needs-review" tag to things that have a
"cworth.needs-review" tag, that wouldn't be deleted. In other words,
cworth's actions will only affect that namespace. Now, if you delete it,
and push it, and Carl pulls back from you (meaning he trusts you), then
it would be deleted on his. When he pushes next, it will be deleted on
everyone who pulls from him.

I'm sure there are resolution paradoxes here. It only had a ten-minute
trial run one day on IRC with me and someone else.

2. The history is available. If you run `nm-remote history msg-id`, it
will show you its history of being added and deleted. I think it will
show by whom it was added or deleted as well, but I don't quite remember
(I hacked this up over a year ago).

tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-08 Thread Jameson Graef Rollins 
On Mon, 06 Jun 2011 13:20:00 -0400, Jesse Rosenthal  
wrote:
> After a conversation with David last year about bug-tracking, I worked
> up a rough python-based prototype of this. It worked in terms of
> namespaces, so Carl could associate the namespace "public" with a list
> of tags he publishes to a http-accessable location. And you could
> associate the namespace "cworth" with those same tags.

This sounds very cool, Jesse.  Very git-like.  I like it.

> You type `whatevercommand pull cworth` and the tags come down from that
> URL as "cworth.*".
> 
> So what he has as "public.to-push" comes down to your notmuch as
> "cworth.to-push".

Did you guys try to address the issue of tag removal at all?  I've been
trying to decide if this is something we need to worry about or not.
For instance, if cworth pushed a tag ".needs-review", you would probably
want to have that tag removed if cworth removed it.  I guess
alternatively he could just push the tag ".reviewed" to nullify the
meaning of the previous one.  I'm not sure that would work in all cases,
though.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL:

Re: tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-08 Thread Jameson Graef Rollins 
On Mon, 06 Jun 2011 13:20:00 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote:
 After a conversation with David last year about bug-tracking, I worked
 up a rough python-based prototype of this. It worked in terms of
 namespaces, so Carl could associate the namespace public with a list
 of tags he publishes to a http-accessable location. And you could
 associate the namespace cworth with those same tags.

This sounds very cool, Jesse.  Very git-like.  I like it.

 You type `whatevercommand pull cworth` and the tags come down from that
 URL as cworth.*.
 
 So what he has as public.to-push comes down to your notmuch as
 cworth.to-push.

Did you guys try to address the issue of tag removal at all?  I've been
trying to decide if this is something we need to worry about or not.
For instance, if cworth pushed a tag .needs-review, you would probably
want to have that tag removed if cworth removed it.  I guess
alternatively he could just push the tag .reviewed to nullify the
meaning of the previous one.  I'm not sure that would work in all cases,
though.

jamie.


pgpOt4bxk6pSj.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-08 Thread Jesse Rosenthal 
On Wed, 08 Jun 2011 10:46:57 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 Did you guys try to address the issue of tag removal at all?  I've been
 trying to decide if this is something we need to worry about or not.
 For instance, if cworth pushed a tag .needs-review, you would probably
 want to have that tag removed if cworth removed it.  I guess
 alternatively he could just push the tag .reviewed to nullify the
 meaning of the previous one.  I'm not sure that would work in all cases,
 though.

Yes, if he deletes the tag public.needs-review in his notmuch, and
then pushes it, it will delete the tag cworth.needs-review from yours.

A couple points about this:

1. If you added your own needs-review tag to things that have a
cworth.needs-review tag, that wouldn't be deleted. In other words,
cworth's actions will only affect that namespace. Now, if you delete it,
and push it, and Carl pulls back from you (meaning he trusts you), then
it would be deleted on his. When he pushes next, it will be deleted on
everyone who pulls from him.

I'm sure there are resolution paradoxes here. It only had a ten-minute
trial run one day on IRC with me and someone else.

2. The history is available. If you run `nm-remote history msg-id`, it
will show you its history of being added and deleted. I think it will
show by whom it was added or deleted as well, but I don't quite remember
(I hacked this up over a year ago).

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-06 Thread Jesse Rosenthal 

On Mon, 06 Jun 2011 09:28:13 -0700, Jameson Graef Rollins  wrote:
> I've been thinking about this more and it really seems we need a way to
> just share tags.  What if we had a way to export all the tags for a set
> of messages as a notmuch dump file, that could just be piped into
> notmuch to modify tags?  This would be a great way for lots of people to
> keep tags synced on a set of messages.

After a conversation with David last year about bug-tracking, I worked
up a rough python-based prototype of this. It worked in terms of
namespaces, so Carl could associate the namespace "public" with a list
of tags he publishes to a http-accessable location. And you could
associate the namespace "cworth" with those same tags.

He types `whatevercommand push public` and all tags "public.*" go to an
config-associated URL.

You type `whatevercommand pull cworth` and the tags come down from that
URL as "cworth.*".

So what he has as "public.to-push" comes down to your notmuch as
"cworth.to-push".

It's available here:

http://commonmeasure.org/~jkr/git/nm-remote.git

See these emails for more (but note that the repo url has changed to the
above).

id:"m1k4rkkchy.fsf at watt.gilman.jhu.edu"
id:"m1hbmokbxj.fsf at watt.gilman.jhu.edu"

There were some details and inherent ambiguities about conflict
resolution, and the above emails explain how I dealt with them. 

Note also that it uses python's configparser, which will overwrite your
config -- which means it'll get rid of your comments. So if you use any
of the config-writing commands, make sure you back up your config first.

Best,
Jesse

tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-06 Thread Jameson Graef Rollins 
On Mon, 06 Jun 2011 05:17:27 -0700, Carl Worth  wrote:
> Hopefully, someone will provide me with a good way to publish my queue
> soon, ("notmuch search --output=html" ?), and then communication like
> this will be a bit easier. ;-)

I've been thinking about this more and it really seems we need a way to
just share tags.  What if we had a way to export all the tags for a set
of messages as a notmuch dump file, that could just be piped into
notmuch to modify tags?  This would be a great way for lots of people to
keep tags synced on a set of messages.

The main difficulty (seems to me) would be the sharing of -tags.  You
wouldn't want the absence of a tag on a message in the tag file to mean
that the tag should be removed.  So we would need to represent both
+tags and -tags in the dump file [0].

It seems a little crazy, but would it be possible to store -tags in the
database somehow?

jamie.

[0] This would actually help with tags applied when the message is
indexed as well, like "signed" and "encrypted", since it would allow for
tagging messages that were indexed before the "signed"/"encrypted"
tagging was a feature.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL:

tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-06 Thread Jameson Graef Rollins 
On Mon, 06 Jun 2011 05:17:27 -0700, Carl Worth cwo...@cworth.org wrote:
 Hopefully, someone will provide me with a good way to publish my queue
 soon, (notmuch search --output=html ?), and then communication like
 this will be a bit easier. ;-)

I've been thinking about this more and it really seems we need a way to
just share tags.  What if we had a way to export all the tags for a set
of messages as a notmuch dump file, that could just be piped into
notmuch to modify tags?  This would be a great way for lots of people to
keep tags synced on a set of messages.

The main difficulty (seems to me) would be the sharing of -tags.  You
wouldn't want the absence of a tag on a message in the tag file to mean
that the tag should be removed.  So we would need to represent both
+tags and -tags in the dump file [0].

It seems a little crazy, but would it be possible to store -tags in the
database somehow?

jamie.

[0] This would actually help with tags applied when the message is
indexed as well, like signed and encrypted, since it would allow for
tagging messages that were indexed before the signed/encrypted
tagging was a feature.


pgp5zi4sZlKn7.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: tag sharing [was: Re: release-candidate/0.6 redux]

2011-06-06 Thread Jesse Rosenthal 

On Mon, 06 Jun 2011 09:28:13 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 I've been thinking about this more and it really seems we need a way to
 just share tags.  What if we had a way to export all the tags for a set
 of messages as a notmuch dump file, that could just be piped into
 notmuch to modify tags?  This would be a great way for lots of people to
 keep tags synced on a set of messages.

After a conversation with David last year about bug-tracking, I worked
up a rough python-based prototype of this. It worked in terms of
namespaces, so Carl could associate the namespace public with a list
of tags he publishes to a http-accessable location. And you could
associate the namespace cworth with those same tags.

He types `whatevercommand push public` and all tags public.* go to an
config-associated URL.

You type `whatevercommand pull cworth` and the tags come down from that
URL as cworth.*.

So what he has as public.to-push comes down to your notmuch as
cworth.to-push.

It's available here:

http://commonmeasure.org/~jkr/git/nm-remote.git

See these emails for more (but note that the repo url has changed to the
above).

id:m1k4rkkchy@watt.gilman.jhu.edu
id:m1hbmokbxj@watt.gilman.jhu.edu

There were some details and inherent ambiguities about conflict
resolution, and the above emails explain how I dealt with them. 

Note also that it uses python's configparser, which will overwrite your
config -- which means it'll get rid of your comments. So if you use any
of the config-writing commands, make sure you back up your config first.

Best,
Jesse
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch