[PATCH] Add --include-duplicates option to a couple of commands.

2011-01-25 Thread Carl Worth
This adds new functionality under the names of:

notmuch search --output=files --include-duplicates
notmuch show --include-duplicates
notmuch show --format=json --include-duplicates

These new commands behave similarly to the existing commands without
the --include-duplicates agument. The difference is that with the new
argument any duplicate mail files will be included in the
output. Here, files are considered duplicates if they contain
identical contents for the Message-Id header, (regardless of any other
differences in the content of the file). Without the
--include-duplicates argument, these commands would emit a single,
arbitrary file in the face of duplicates.

WARNING: This commit is not yet ready to be pushed to the notmuch
repository. There are at least two problems with the commit so far:

1. Nothing has been documented yet.

   Fixing this shouldn't be too hard. It's mostly just taking
   the text from above and shoving it into the
   documentation. I can do this easily enough myself.

2. show --format=json --include-duplicates doesn't work yet

   This is a more serious problem. I believe the JSON output
   with this patch is not correct and will likely break a
   client trying to consume it. It inserts the duplicate
   message into an array next to the existing message. Our
   current JSON schema isn't documented formally that I could
   find, except for a comment in the emacs code that consumes
   it:

A thread is a forest or list of trees. A tree is a two
element list where the first element is a message, and
the second element is a possibly empty forest of
replies.

   I believe this commit breaks the "two-element list"
   expectation. What we would want instead is the duplicate
   message to appear as a peer next to the original message,
   (and then perhaps have replies appear only to one of the
   messages).

My current need for --include-duplicates was recently satisfied, so I
won't likely pursue this further for now. But I wanted to put this
code out rather than losing it.

If someone wants to fix the patch to do the "right thing" with the
JSON output, then that would be great.

ALSO NOTE: I left the
json.expected-output/notmuch-show-thread-format-json-maildir-storage
out of this commit. It has lines in it that are too long to be sent
via git-send-email.
---
 notmuch-search.c   |   30 +-
 notmuch-show.c |   61 +--
 test/basic |2 +-
 test/json  |   33 ++-
 ...-show-thread-include-duplicates-maildir-storage |   94 
 .../notmuch-show-thread-maildir-storage|   47 
 test/search-output |  113 
 7 files changed, 361 insertions(+), 19 deletions(-)
 create mode 100644 
test/json.expected-output/notmuch-show-thread-include-duplicates-maildir-storage
 create mode 100644 
test/json.expected-output/notmuch-show-thread-maildir-storage

diff --git a/notmuch-search.c b/notmuch-search.c
index c628b36..6d032c2 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -247,7 +247,8 @@ static int
 do_search_messages (const void *ctx,
const search_format_t *format,
notmuch_query_t *query,
-   output_t output)
+   output_t output,
+   notmuch_bool_t include_duplicates)
 {
 notmuch_message_t *message;
 notmuch_messages_t *messages;
@@ -269,8 +270,25 @@ do_search_messages (const void *ctx,
fputs (format->item_sep, stdout);

if (output == OUTPUT_FILES) {
-   format->item_id (ctx, "",
-notmuch_message_get_filename (message));
+   if (include_duplicates) {
+   notmuch_filenames_t *filenames;
+   int first_filename = 1;
+
+   for (filenames = notmuch_message_get_filenames (message);
+notmuch_filenames_valid (filenames);
+notmuch_filenames_move_to_next (filenames))
+   {
+   if (! first_filename)
+   fputs (format->item_sep, stdout);
+   first_filename = 0;
+
+   format->item_id (ctx, "",
+notmuch_filenames_get (filenames));
+   }
+   } else {
+   format->item_id (ctx, "",
+notmuch_message_get_filename (message));
+   }
} else { /* output == OUTPUT_MESSAGES */
format->item_id (ctx, "id:",
 notmuch_message_get_message_id (message));
@@ -352,6 +370,7 @@ notmuch_search_command (void *c

[PATCH] Add --include-duplicates option to a couple of commands.

2011-01-24 Thread Carl Worth
This adds new functionality under the names of:

notmuch search --output=files --include-duplicates
notmuch show --include-duplicates
notmuch show --format=json --include-duplicates

These new commands behave similarly to the existing commands without
the --include-duplicates agument. The difference is that with the new
argument any duplicate mail files will be included in the
output. Here, files are considered duplicates if they contain
identical contents for the Message-Id header, (regardless of any other
differences in the content of the file). Without the
--include-duplicates argument, these commands would emit a single,
arbitrary file in the face of duplicates.

WARNING: This commit is not yet ready to be pushed to the notmuch
repository. There are at least two problems with the commit so far:

1. Nothing has been documented yet.

   Fixing this shouldn't be too hard. It's mostly just taking
   the text from above and shoving it into the
   documentation. I can do this easily enough myself.

2. show --format=json --include-duplicates doesn't work yet

   This is a more serious problem. I believe the JSON output
   with this patch is not correct and will likely break a
   client trying to consume it. It inserts the duplicate
   message into an array next to the existing message. Our
   current JSON schema isn't documented formally that I could
   find, except for a comment in the emacs code that consumes
   it:

A thread is a forest or list of trees. A tree is a two
element list where the first element is a message, and
the second element is a possibly empty forest of
replies.

   I believe this commit breaks the "two-element list"
   expectation. What we would want instead is the duplicate
   message to appear as a peer next to the original message,
   (and then perhaps have replies appear only to one of the
   messages).

My current need for --include-duplicates was recently satisfied, so I
won't likely pursue this further for now. But I wanted to put this
code out rather than losing it.

If someone wants to fix the patch to do the "right thing" with the
JSON output, then that would be great.

ALSO NOTE: I left the
json.expected-output/notmuch-show-thread-format-json-maildir-storage
out of this commit. It has lines in it that are too long to be sent
via git-send-email.
---
 notmuch-search.c   |   30 +-
 notmuch-show.c |   61 +--
 test/basic |2 +-
 test/json  |   33 ++-
 ...-show-thread-include-duplicates-maildir-storage |   94 
 .../notmuch-show-thread-maildir-storage|   47 
 test/search-output |  113 
 7 files changed, 361 insertions(+), 19 deletions(-)
 create mode 100644 
test/json.expected-output/notmuch-show-thread-include-duplicates-maildir-storage
 create mode 100644 
test/json.expected-output/notmuch-show-thread-maildir-storage

diff --git a/notmuch-search.c b/notmuch-search.c
index c628b36..6d032c2 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -247,7 +247,8 @@ static int
 do_search_messages (const void *ctx,
const search_format_t *format,
notmuch_query_t *query,
-   output_t output)
+   output_t output,
+   notmuch_bool_t include_duplicates)
 {
 notmuch_message_t *message;
 notmuch_messages_t *messages;
@@ -269,8 +270,25 @@ do_search_messages (const void *ctx,
fputs (format->item_sep, stdout);
 
if (output == OUTPUT_FILES) {
-   format->item_id (ctx, "",
-notmuch_message_get_filename (message));
+   if (include_duplicates) {
+   notmuch_filenames_t *filenames;
+   int first_filename = 1;
+
+   for (filenames = notmuch_message_get_filenames (message);
+notmuch_filenames_valid (filenames);
+notmuch_filenames_move_to_next (filenames))
+   {
+   if (! first_filename)
+   fputs (format->item_sep, stdout);
+   first_filename = 0;
+
+   format->item_id (ctx, "",
+notmuch_filenames_get (filenames));
+   }
+   } else {
+   format->item_id (ctx, "",
+notmuch_message_get_filename (message));
+   }
} else { /* output == OUTPUT_MESSAGES */
format->item_id (ctx, "id:",
 notmuch_message_get_message_id (message));
@@ -352,6 +370,7 @@ notmuch_search_command (void *