[notmuch] [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show.

[Carl Worth]

On Fri, 18 Dec 2009 08:59:55 -0400, david at tethera.net wrote:
> From: Scott Robinson 

Hi Scott (and David),

I'm finally getting around to looking closely at the JSON patches
(hurrah!). Here are some comments:

> In the case of notmuch-show, "--output=json" also implies
> "--entire-thread" as the thread structure is implicit in the emitted
> document tree.

I think we've all agreed to use --output for selecting what to print and
to instead use --format for how to format it. I also think David's got a
patch to change that, but if we have to change the current patch anyway,
we might change that at the same time.

> --- /dev/null
> +++ b/json.c
> @@ -0,0 +1,73 @@
> +/* notmuch - Not much of an email program, (just index and search)
> + *
> + * Copyright ? 2009 Carl Worth
> + * Copyright ? 2009 Keith Packard

I know I didn't contribute any code to this file, so my name shouldn't
be here. Scott, I imagine you have made some non-trivial contributions
so your name (or suitable copyright-claiming entity) should be here.

> + * Authors: Carl Worth 
> + *   Keith Packard 

Same thing here.

> +/*
> + * json_quote_str derived from cJSON's print_string_ptr,
> + * Copyright (c) 2009 Dave Gamble
> + */

Thanks for attributing the source here, but let's please keep all the
copyright statements up at the top of the file.

It would still be reasonable to have a comment at this point that this
particular code came from Dave Gamble and cJSON. But it should also
mention the license under which the code is being integrated.

I suggest leaving the notmuch-standar GPLv3+ blurb at the top of the
file, but then quoting the license itself of the external code, (it's a
short MIT-style license, right?).

> +char *
> +json_quote_str(const void *ctx, const char *str)

It would be nice to have a little comment here describing what the
function does, (what characters get quoted and how, that the return
value is talloced with 'ctx' as a parent, and perhaps a pointer to the
appropriate JSON reference/specification).

But I definitely like this nice little function as opposed to some JSON
library. Thanks!


> +typedef struct search_format {
> +const char *results_start;
> +const char *thread_start;
> +void (*thread) (const void *ctx,
> + const char *id,
> + const time_t date,
> + const int matched,
> + const int total,
> + const char *authors,
> + const char *subject);
...

Definitely missing at least a quick comment for internal documentation
here as well. But I do like the way this works.

> --- a/notmuch.c
> +++ b/notmuch.c
> @@ -162,6 +162,11 @@ command_t commands[] = {
>"\n"
>"\t\tSupported options for search include:\n"
>"\n"
> +  "\t\t--output=(json|text)\n"
> +  "\n"
> +  "\t\t\tPresents the results in either JSON or plain-text\n"
> +  "\t\t\tformat, which is the default.\n"
> +  "\n"

Thanks for adding the documentation here. But please add to notmuch.1 as
well.

This all looks really great. And I can't wait to apply it and play with
it more.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[notmuch] [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show.

[Carl Worth]

On Fri, 18 Dec 2009 20:36:34 -0400, David Bremner  wrote:
> It's a detail, but could you choose two names that are not substrings of
> each other?  Eventually we do want tab completion on the command line to
> work :).

Yes, that's a good point.

> Also, "search --for tags foo" suggests to me that
> searching for tags matching foo.  What about using --output for that?

OK. "--output" sounds good to me here.

> One thing that is not completely clear to me at this point is what the
> difference is between 
> 
> notmuch search --for messages  search-terms
> 
> and 
> 
> notmuch show search-terms

So, "notmuch show " is clear enough---it works as it does
today.

The new command, ("notmuch search --output=messages"), would be quite
different. It would have single-line output for each message, (as
"notmuch search" has single line-output already, but for threads by
default). You can see behavior like this in the "notmuch
search-messages" command for which I sent a patch a while ago, (but have
never merged).

The idea is that "notmuch search" would always give single-line output
suitable for various kinds of processing.

For example. How much mail have I sent?

notmuch search --output=messages tag:sent | wc -l

That's something you can't do with a thread-based search, (and it's not
convenient to get a robust result from "notmuch show").

Once this is combined with a new --format to select what gets printed, I
can imagine a lot of useful things, like collecting email addresses:

notmuch search --output=messages --format="${FROM}" to:cworth at 
cworth.org

or whatever. I can imagine a lot of different queries I'd like to be
able to make of my mail store with things like this.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[notmuch] [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show.

[David Bremner]

On Fri, 18 Dec 2009 09:33:43 -0800, Carl Worth  wrote:

> I think that selecting *what* to emit is orthogonal from selecting *how*
> to format that output. 

I can see that point of view.

> See some ideas in the TODO file, (where I proposed --for and --format
> options for these).

It's a detail, but could you choose two names that are not substrings of
each other?  Eventually we do want tab completion on the command line to
work :).  Also, "search --for tags foo" suggests to me that
searching for tags matching foo.  What about using --output for that?
One thing that is not completely clear to me at this point is what the
difference is between 

notmuch search --for messages  search-terms

and 

notmuch show search-terms

David

[notmuch] [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show.

[Scott Robinson]

Excerpts from Carl Worth's message of Fri Dec 18 09:33:43 -0800 2009:
> On Fri, 18 Dec 2009 08:59:55 -0400, david at tethera.net wrote:
> > It took me a little work to apply Scott's patch, so rather than asking
> > him to resend it from git-send-email, I am just sending. I hope no-one
> > is offended (much).
> 
> I think that's great! Collaboration is what this is all about.

Me too!

I've never used git-send-email. I'll give it a whirl on my next patch.

> > I'm thinking that the patch I sent out last night to only dump message
> > ids could be reworked to use the framework of this patch.  I also
> > think it would be reasonably simple to add an --output=mbox option,
> > for archiving and so on.
> 
> I think that selecting *what* to emit is orthogonal from selecting *how*
> to format that output. See some ideas in the TODO file, (where I
> proposed --for and --format options for these). Having a way to do mbox
> output for export would indeed be very nice.

Haha! I originally used "--format" and changed for some reason that escapes me
now.

Implementing an "mbox" formatted output in the current logic wouldn't be
archive perfect. The message body is emitted on a per-part basis.

What I would do is change the semantics of format->body to be called from
show_message. Then the text and json parts would point at the original
implementation passing off their per-part function pointers. And, a new mbox
implementation would just dump the full message body.
-- 
Scott Robinson | http://quadhome.com/

Q: Why are my replies five sentences or less?
A: http://five.sentenc.es/

[notmuch] [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show.

[Carl Worth]

On Fri, 18 Dec 2009 08:59:55 -0400, david at tethera.net wrote:
> It took me a little work to apply Scott's patch, so rather than asking
> him to resend it from git-send-email, I am just sending. I hope no-one
> is offended (much).

I think that's great! Collaboration is what this is all about.

> I'm thinking that the patch I sent out last night to only dump message
> ids could be reworked to use the framework of this patch.  I also
> think it would be reasonably simple to add an --output=mbox option,
> for archiving and so on.

I think that selecting *what* to emit is orthogonal from selecting *how*
to format that output. See some ideas in the TODO file, (where I
proposed --for and --format options for these). Having a way to do mbox
output for export would indeed be very nice.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 

[notmuch] [PATCH] Add an "--output=(json|text|)" command-line option to both notmuch-search and notmuch-show.

[da...@tethera.net]

From: Scott Robinson 

In the case of notmuch-show, "--output=json" also implies
"--entire-thread" as the thread structure is implicit in the emitted
document tree.

As a coincidence to the implementation, multipart message ID numbers are
now incremented with each part printed. This changes the previous
semantics, which were unclear and not necessary related to the actual
ordering of the message parts.

Edited-By: David Bremner 
Reviewed-By: David Bremner 
---

It took me a little work to apply Scott's patch, so rather than asking
him to resend it from git-send-email, I am just sending. I hope no-one
is offended (much).

Other than manually extracting the patch from the output of notmuch
show (for me the message arrived base64 encoded), I deleted trailing
whitespace on line 465. 

It compiles, it doesn't seem to screw up the original output, and at
least in a few tests, it generates parseable json. Yay!.

I'm thinking that the patch I sent out last night to only dump message
ids could be reworked to use the framework of this patch.  I also
think it would be reasonably simple to add an --output=mbox option,
for archiving and so on.

 Makefile.local   |3 +-
 json.c   |   73 ++
 notmuch-client.h |3 +
 notmuch-search.c |  163 +---
 notmuch-show.c   |  275 ++
 notmuch.c|   24 --
 show-message.c   |4 +-
 7 files changed, 481 insertions(+), 64 deletions(-)
 create mode 100644 json.c

diff --git a/Makefile.local b/Makefile.local
index 933ff4c..53b474b 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -18,7 +18,8 @@ notmuch_client_srcs = \
notmuch-tag.c   \
notmuch-time.c  \
query-string.c  \
-   show-message.c
+   show-message.c  \
+   json.c

 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 notmuch: $(notmuch_client_modules) lib/notmuch.a
diff --git a/json.c b/json.c
new file mode 100644
index 000..ee563d6
--- /dev/null
+++ b/json.c
@@ -0,0 +1,73 @@
+/* notmuch - Not much of an email program, (just index and search)
+ *
+ * Copyright ?? 2009 Carl Worth
+ * Copyright ?? 2009 Keith Packard
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Authors: Carl Worth 
+ * Keith Packard 
+ */
+
+#include "notmuch-client.h"
+
+/*
+ * json_quote_str derived from cJSON's print_string_ptr,
+ * Copyright (c) 2009 Dave Gamble
+ */
+
+char *
+json_quote_str(const void *ctx, const char *str)
+{
+const char *ptr;
+char *ptr2;
+char *out;
+int len = 0;
+
+if (!str)
+   return NULL;
+
+for (ptr = str; *ptr; len++, ptr++) {
+   if (*ptr < 32 || *ptr == '\"' || *ptr == '\\')
+   len++;
+}
+
+out = talloc_array (ctx, char, len + 3);
+
+ptr = str;
+ptr2 = out;
+
+*ptr2++ = '\"';
+while (*ptr) {
+   if (*ptr > 31 && *ptr != '\"' && *ptr != '\\') {
+   *ptr2++ = *ptr++;
+   } else {
+   *ptr2++ = '\\';
+   switch (*ptr++) {
+   case '\"':  *ptr2++ = '\"'; break;
+   case '\\':  *ptr2++ = '\\'; break;
+   case '\b':  *ptr2++ = 'b';  break;
+   case '\f':  *ptr2++ = 'f';  break;
+   case '\n':  *ptr2++ = 'n';  break;
+   case '\r':  *ptr2++ = 'r';  break;
+   case '\t':  *ptr2++ = 't';  break;
+   default: ptr2--;break;
+   }
+   }
+}
+*ptr2++ = '\"';
+*ptr2++ = '\0';
+
+return out;
+}
diff --git a/notmuch-client.h b/notmuch-client.h
index 50a30fe..7b844b9 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -143,6 +143,9 @@ notmuch_status_t
 show_message_body (const char *filename,
   void (*show_part) (GMimeObject *part, int *part_count));

+char *
+json_quote_str (const void *ctx, const char *str);
+
 /* notmuch-config.c */

 typedef struct _notmuch_config notmuch_config_t;
diff --git a/notmuch-search.c b/notmuch-search.c
index dc44eb6..e243747 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -20,8 +20,120 @@

 #include "notmuch-client.h"

+typedef struct search_format {
+const char *results_start;
+const char *thread_start;
+void (*thread) (const void *ctx,
+  

Re: [notmuch] [PATCH] Add an --output=(json|text|) command-line option to both notmuch-search and notmuch-show.

[Carl Worth]

On Fri, 18 Dec 2009 08:59:55 -0400, da...@tethera.net wrote:
 It took me a little work to apply Scott's patch, so rather than asking
 him to resend it from git-send-email, I am just sending. I hope no-one
 is offended (much).

I think that's great! Collaboration is what this is all about.

 I'm thinking that the patch I sent out last night to only dump message
 ids could be reworked to use the framework of this patch.  I also
 think it would be reasonably simple to add an --output=mbox option,
 for archiving and so on.

I think that selecting *what* to emit is orthogonal from selecting *how*
to format that output. See some ideas in the TODO file, (where I
proposed --for and --format options for these). Having a way to do mbox
output for export would indeed be very nice.

-Carl


pgp3wBJvDcTzl.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch