Re: tag sharing
On Thu, 06 Oct 2011 11:18:51 -0300, David Bremner da...@tethera.net wrote: Non-text part: multipart/signed On Thu, 06 Oct 2011 09:21:48 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote: morning's project. In retrospect, I think the main issue was that I was trying to figure out how history would be kept. By using git, though, your idea would be that we get history for free, right? Yeah. Of course, one would have to decide if the history of the current dump file format is intelligible, or if some alternative format should be used. I've managed to reconstruct a bit of my thinking. I think it could work with the current format if we went as follows. I also think we could use git, but I'm not sure that wouldn't be overkill. Some of the steps are in extreme slow motion, just to make sure we're on the same page. (I've also replaced the real msg-ids to cut down on noise). 1. David dumps the targeted messages (tags in namespace my), so we get a file like this: msg-id-1@example.1 (my.foo inbox) msg-id-2@example.2 (answered my.to-do inbox signed) 2. David strips out everything but tags in the namespace and removes the namespace: msg-id-1@example.1 (foo) msg-id-2@example.2 (to-do) 3. David makes it available, Jesse pulls. 4. Jesse dumps the namespace he has associated with David (bremner): msg-id-3@example.3 (bremner.bar) msg-id-4@example.4 (bremner.bar unread) msg-id-1@example.1 (bremner.foo inbox) msg-id-5@example.5 (bremner.wishlist inbox) 5. Jesse strips the tags: msg-id-3@example.3 (bar) msg-id-4@example.4 (bar) msg-id-1@example.1 (foo) msg-id-5@example.5 (wishlist) 6. Jesse replaces this list with David's: msg-id-1@example.1 (foo) msg-id-2@example.2 (to-do) 7. Jesse removes all tags with the bremner namespace from his database. 8. Jesse adds the tags from the list in step (6), with the appropriate namespace, to his list of tags. The messages of interest now look like: msg-id-3@example.3 () msg-id-4@example.4 (unread) msg-id-1@example.1 (bremner.foo inbox) msg-id-5@example.5 (inbox) msg-id-2@example.2 (bremner.to-do someothertag) 9. Jesse runs `notmuch restore` The reason I went through this slowly is to highlight the fact that the only thing that needs to be recorded in diffs somehow (in git, let's say) are the files in steps (5) and (6). So if we integrated this with git, there would be something along the lines of a commit (on Jesse's computer) in step (5) and then another commit (again on Jesse's computer) in step (6). Note that we don't actually need (5) to do tag-sharing. The only reason it's there is for keeping history. I'm pretty sure that we don't need a commit on David's computer in step (1), because what the history would record is when the tag-sharing utility changed your tags, not when you did. Note also that all we would really be asking git to do here is keep an ordered collection of diffs of a single file. Honest question: since this is the barest possible form of version control, is git necessary? Some of the features, like commit messages, don't seem to fit well with this model; and others, like branching, seem pretty useless. What's the value added over just keeping a (compressed?) collection of diffs for each namespace? Best, Jesse ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Fri, 07 Oct 2011 07:36:30 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote: What's the value added over just keeping a (compressed?) collection of diffs for each namespace? Okay -- please don't bother answering this part. It was early in the morning, and I forgot some of the obvious advantages of real version control (we can roll back to a previous moment, etc). We still don't need most of what git offers, but if it's the best tool for keeping track of the move from the file in step (5) to the file in step (6), then sure, let's use it. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Wed, 08 Jun 2011 10:46:57 -0700, Jameson Graef Rollins jroll...@finestructure.net wrote: On Mon, 06 Jun 2011 13:20:00 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote: After a conversation with David last year about bug-tracking, I worked up a rough python-based prototype of this. It worked in terms of namespaces, so Carl could associate the namespace public with a list of tags he publishes to a http-accessable location. And you could associate the namespace cworth with those same tags. This sounds very cool, Jesse. Very git-like. I like it. Warning: git fanboi-ism ahead. I like the concept too. But I worry a bit that we are re-inventing things that could be done by git. What do people think about a similar namespace import export tool where we rely on git for a transport layer, and for merging, conflict-resolution, and history tracking. Of course there are details to be settled, but before I try to work them out, would anyone like to convince me the idea is fundementally wrong headed? Jesse, do you remember why you decided to roll your own? I agree it is subjective what is easier, but I think we have enough people comfortable with git that collaborative maintenence of a bug-tag git repo would be pretty easy. d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Thu, 06 Oct 2011 09:21:48 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote: morning's project. In retrospect, I think the main issue was that I was trying to figure out how history would be kept. By using git, though, your idea would be that we get history for free, right? Yeah. Of course, one would have to decide if the history of the current dump file format is intelligible, or if some alternative format should be used. of git? I.e., all we need something to just keep different dumps in different namespaces, so that when I pull your dump, the tags come in as `bremner/to-fix` (or whatever our separator is), and then when I dump all `bremner` tags, it removes the namespace and puts it in `bremner.tagfile`. something like that sounds plausible. Currently the query parser doesn't handle searches like tag:bremner/to-fix very well, because it helpfully splits at '/' (aiui; maybe somebody else can explain it better). notmuch search tag:notmuch* does not seem to work either, so my first idea to kludge around the problem by using e.g. notmuchPushed doesn't seem to work either. So using either the library or the bindings would probably speed things up quite a bit, by only dumping the tags you care about. Which means it couldn't just be a shell script: it would have to be c/ruby/python. Without looking at the code, I'd guess it wouldn't be too hard to have notmuch-dump take a standard search term; it's just as I mentioned I don't see an elegant way of doing what we want with a standard search term. Of course I'd be happy to be proved wrong. David pgprd6P3nNjJz.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Thu, 06 Oct 2011 09:56:43 -0300, David Bremner da...@tethera.net wrote: Jesse, do you remember why you decided to roll your own? Only reason I can remember (a year and a half later) of is that it seemed like the basic illustration of concept would be a saturday morning's project. In retrospect, I think the main issue was that I was trying to figure out how history would be kept. By using git, though, your idea would be that we get history for free, right? I think my format was: TIMESTAMPMESSAGID TAG 12345 foo@barto-fix 13456 bar@foofeature-request But using git would get us timestamps for free too, I guess. And if we get rid of timestamps, then we just have dumps. So what we would really just need is a thin namespacing layer in some scripting language on top of git? I.e., all we need something to just keep different dumps in different namespaces, so that when I pull your dump, the tags come in as `bremner/to-fix` (or whatever our separator is), and then when I dump all `bremner` tags, it removes the namespace and puts it in `bremner.tagfile`. One rather obvious thing that I discovered: processing dumps and restores from the ui is pretty slow, because you have to output a huge file, and then find the tags in your preferred namespace. So using either the library or the bindings would probably speed things up quite a bit, by only dumping the tags you care about. Which means it couldn't just be a shell script: it would have to be c/ruby/python. Best, Jesse ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Thu, 06 Oct 2011 11:18:51 -0300, David Bremner da...@tethera.net wrote: something like that sounds plausible. Currently the query parser doesn't handle searches like tag:bremner/to-fix very well, because it helpfully splits at '/' (aiui; maybe somebody else can explain it better). notmuch search tag:notmuch* does not seem to work either, so my first idea to kludge around the problem by using e.g. notmuchPushed doesn't seem to work either. A period works: `tag:bremner.to-fix`. That would fit with java/python conventions. Double-colons also seem to work, for what it's worth: `tag:bremner::to-fix` ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Thu, 06 Oct 2011 15:49:10 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote: A period works: `tag:bremner.to-fix`. That would fit with java/python conventions. Double-colons also seem to work, for what it's worth: `tag:bremner::to-fix` Right, I was confused and/or unclear. What doesn't work is searches for the whole namespace notmuch search tag:bremner.* will return nothing, even though notmuch search tag:bremner.to-fix does. d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Thu, 06 Oct 2011 17:23:21 -0300, David Bremner da...@tethera.net wrote:
What doesn't work is searches for the whole namespace notmuch search
tag:bremner.* will return nothing, even though notmuch search
tag:bremner.to-fix does.
A simple shell way to do this would be
notmuch search-tags | grep ^bremner\. | xargs -I {} notmuch search tag:{}
That's pretty quick over a fair amount of messages. It would be quicker,
I imagine, if it did the grepping inside of a program, but this is still
way down there in the subsecond range.
NB. I don't have that many tags, so maybe the grepping step would bog
down a bit if I had more. But I doubt someone would have that much more
than a few hundred.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing
On Thu, 06 Oct 2011 16:40:00 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote:
On Thu, 06 Oct 2011 17:23:21 -0300, David Bremner da...@tethera.net wrote:
What doesn't work is searches for the whole namespace notmuch search
tag:bremner.* will return nothing, even though notmuch search
tag:bremner.to-fix does.
A simple shell way to do this would be
notmuch search-tags | grep ^bremner\. | xargs -I {} notmuch search
tag:{}
Ah thanks, that is not so bad. It turns out to be literally a one line
change to add a query argument for notmuch-dump, _except_ that notmuch
dump already uses the first command line argument as an output file
name. I think I will start a seperate thread about that. If you want to
test, this works OK, except the output is put in a file named after the
query ;).
diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..341207a 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -40,7 +40,7 @@ notmuch_dump_command (unused (void *ctx), int argc, char
*argv[])
if (notmuch == NULL)
return 1;
-query = notmuch_query_create (notmuch, );
+query = notmuch_query_create (notmuch, argc 0 ? argv[0] : );
if (query == NULL) {
fprintf (stderr, Out of memory\n);
return 1;
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing [was: Re: release-candidate/0.6 redux]
On Mon, 06 Jun 2011 13:20:00 -0400, Jesse Rosenthal jrosent...@jhu.edu wrote: After a conversation with David last year about bug-tracking, I worked up a rough python-based prototype of this. It worked in terms of namespaces, so Carl could associate the namespace public with a list of tags he publishes to a http-accessable location. And you could associate the namespace cworth with those same tags. This sounds very cool, Jesse. Very git-like. I like it. You type `whatevercommand pull cworth` and the tags come down from that URL as cworth.*. So what he has as public.to-push comes down to your notmuch as cworth.to-push. Did you guys try to address the issue of tag removal at all? I've been trying to decide if this is something we need to worry about or not. For instance, if cworth pushed a tag .needs-review, you would probably want to have that tag removed if cworth removed it. I guess alternatively he could just push the tag .reviewed to nullify the meaning of the previous one. I'm not sure that would work in all cases, though. jamie. pgpOt4bxk6pSj.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing [was: Re: release-candidate/0.6 redux]
On Wed, 08 Jun 2011 10:46:57 -0700, Jameson Graef Rollins jroll...@finestructure.net wrote: Did you guys try to address the issue of tag removal at all? I've been trying to decide if this is something we need to worry about or not. For instance, if cworth pushed a tag .needs-review, you would probably want to have that tag removed if cworth removed it. I guess alternatively he could just push the tag .reviewed to nullify the meaning of the previous one. I'm not sure that would work in all cases, though. Yes, if he deletes the tag public.needs-review in his notmuch, and then pushes it, it will delete the tag cworth.needs-review from yours. A couple points about this: 1. If you added your own needs-review tag to things that have a cworth.needs-review tag, that wouldn't be deleted. In other words, cworth's actions will only affect that namespace. Now, if you delete it, and push it, and Carl pulls back from you (meaning he trusts you), then it would be deleted on his. When he pushes next, it will be deleted on everyone who pulls from him. I'm sure there are resolution paradoxes here. It only had a ten-minute trial run one day on IRC with me and someone else. 2. The history is available. If you run `nm-remote history msg-id`, it will show you its history of being added and deleted. I think it will show by whom it was added or deleted as well, but I don't quite remember (I hacked this up over a year ago). ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: tag sharing [was: Re: release-candidate/0.6 redux]
On Mon, 06 Jun 2011 09:28:13 -0700, Jameson Graef Rollins jroll...@finestructure.net wrote: I've been thinking about this more and it really seems we need a way to just share tags. What if we had a way to export all the tags for a set of messages as a notmuch dump file, that could just be piped into notmuch to modify tags? This would be a great way for lots of people to keep tags synced on a set of messages. After a conversation with David last year about bug-tracking, I worked up a rough python-based prototype of this. It worked in terms of namespaces, so Carl could associate the namespace public with a list of tags he publishes to a http-accessable location. And you could associate the namespace cworth with those same tags. He types `whatevercommand push public` and all tags public.* go to an config-associated URL. You type `whatevercommand pull cworth` and the tags come down from that URL as cworth.*. So what he has as public.to-push comes down to your notmuch as cworth.to-push. It's available here: http://commonmeasure.org/~jkr/git/nm-remote.git See these emails for more (but note that the repo url has changed to the above). id:m1k4rkkchy@watt.gilman.jhu.edu id:m1hbmokbxj@watt.gilman.jhu.edu There were some details and inherent ambiguities about conflict resolution, and the above emails explain how I dealt with them. Note also that it uses python's configparser, which will overwrite your config -- which means it'll get rid of your comments. So if you use any of the config-writing commands, make sure you back up your config first. Best, Jesse ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
