Plans for the 0.2 release (this week)

[Anthony Towns]

On Thu, Apr 8, 2010 at 17:47, Sebastian Spaeth  wrote:
> * Prevent data loss caused by SIGINT during notmuch new
> ?mail id:1269937403-8495-1-git-send-email-sojkam1 at fel.cvut.cz
> ?wentasah, commit dcb7a957027837f187f06d31fdcddb3740b2c20b
> ?(preventable data loss, has happened to him at least once)

That seems to be a different id to
http://patchwork.notmuchmail.org/patch/445/ but "me too" on that,
fwiw. (Haven't had data loss, did have confusing behaviour that I
attribute to this, and the logic for the patch seems sound to me)

Cheers,
aj

-- 
Anthony Towns 

Plans for the 0.2 release (this week)

[Mike Kelly]

On Thu, 08 Apr 2010 14:28:20 -0700
Carl Worth  wrote:

>   2. A "list:" prefix to match content from headers that identify
>  mailing list. My perception is that there are likely a handful of
>  different headers that have been used, and they should all be
>  indexed so that "list:" will search any of them.

Almost all modern mailing list software seems to use List-Id:

https://tools.ietf.org/html/rfc2919

-- 
Mike Kelly

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100408/f1e7821e/attachment.pgp>

Plans for the 0.2 release (this week)

[Jameson Rollins]

On Thu, 08 Apr 2010 14:28:20 -0700, Carl Worth  wrote:
> All of that sounds quite easy to do by simply saving the draft within
> the mailstore and giving it a "draft" tag.

The only way I can see to give drafts a "draft" tag is via 'folder:'
indexing.  Do you have any ideas of other ways?

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100408/0de9f341/attachment.pgp>

Plans for the 0.2 release (this week)

[Michal Sojka]

On Thu, 08 Apr 2010, Dirk Hohndel wrote:
> On Thu, 08 Apr 2010 10:03:15 -0400, Jameson Rollins  finestructure.net> wrote:
> > On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
> > > For the upcoming 0.2 release, here are some things that I would like to
> > > have in place:
> > > 
> > >   * Changes to indexing, (addition of body:, folder:, list:, etc.). This
> > > is stuff that I'll work on.
> > 
> > Also, fwiw, the folder: indexing is probably the new feature that I'm
> > most eagerly awaiting.  I've got all these ideas for ways to handle sent
> > mail and drafts if we can get this working.
> 
> Maybe I am missing some context here - what exactly does 'indexing' mean
> here?

Indexing means calling notmuch new. The folder: patch, adds "folder:"
search prefix which allows searching for messages based on the folder
where the message is stored e.g. folder:sent.

-Michal

[PATCH] Add maildir-based mailstore

[Michal Sojka]

This mailstore allows bi-directional synchronization between maildir
flags and certain tags. The flag-to-tag mapping is defined by flag2tag
array.

The synchronization works this way:

1) Whenever notmuch new is executed, the following happens:
   o New messages are tagged with inbox.
   o New messages without maildir info in the file name (typically
 files in new/) are tagged with unread.
   o For new or renamed messages with maildir info present in the
 file name, the tags defined in flag2tag are either added or
 removed depending on the flags from the file name.

2) Whenever notmuch tag (or notmuch restore) is executed, a new set of
   flags based on the tags is constructed for every message and a new
   file name is prepared based on the old file name but with the new
   flags. If the old message file was in 'new' directory then this is
   replaced with 'cur' in the new file name. If the new and old file
   names differ, the file is renamed and notmuch database is updated
   accordingly.

   The rename happens before the database is updated. In case of crash
   between rename and database update, the next run of notmuch new
   brings the database in sync with the mail store again.

This mailstore is enabled by putting the following to your
.notmuch-config:

[mailstore]
type=maildir

Signed-off-by: Michal Sojka 
---
 lib/database.cc   |7 ++
 lib/mailstore-files.c |  210 -
 lib/mailstore.c   |1 +
 lib/message.cc|   41 +-
 lib/notmuch-private.h |4 +
 lib/notmuch.h |1 +
 6 files changed, 260 insertions(+), 4 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index bd64ed3..4f3ce88 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1479,6 +1479,13 @@ notmuch_database_add_message (notmuch_database_t 
*notmuch,

_notmuch_message_add_filename (message, filename);

+   /* This is a new message or it has a new filename and as such,
+* its tags in database either do not exists or might be out
+* of date. Mailstore assigns the tags later in index_new(),
+* but until then we should not synchronize the tags back to
+* the mailstore. */
+   notmuch_message_set_flag(message, NOTMUCH_MESSAGE_FLAG_TAGS_INVALID, 
TRUE);
+
/* Is this a newly created message object? */
if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
_notmuch_message_add_term (message, "type", "mail");
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index f2cb8d7..e3e346f 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -24,6 +24,30 @@
 #include "notmuch.h"
 #include "mailstore-private.h"
 #include 
+#include 
+
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
+struct mailstore_priv {
+void (*tag_new)(notmuch_message_t *message, const char *filename);
+int  (*tag_renamed)(notmuch_message_t *message, const char *filename);
+};
+
+struct maildir_flag_tag {
+char flag;
+const char *tag;
+bool inverse;
+};
+
+/* ASCII ordered table of Maildir flags and assiciated tags */
+struct maildir_flag_tag flag2tag[] = {
+{ 'D', "draft",   false},
+{ 'F', "flagged", false},
+{ 'P', "passed",  false},
+{ 'R', "replied", false},
+{ 'S', "unread",  true },
+{ 'T', "delete",  false},
+};

 typedef struct _filename_node {
 char *filename;
@@ -69,8 +93,9 @@ _filename_list_add (_filename_list_t *list,
 }

 static void
-tag_inbox_and_unread (notmuch_message_t *message)
+tag_inbox_and_unread (notmuch_message_t *message, const char *filename)
 {
+(void)filename;
 notmuch_message_add_tag (message, "inbox");
 notmuch_message_add_tag (message, "unread");
 }
@@ -116,6 +141,163 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
 return 0;
 }

+static int
+tag_from_maildir_flags (notmuch_message_t *message, const char *filename)
+{
+const char *flags, *p;
+char f;
+bool valid, unread;
+unsigned i;
+
+flags = strstr(filename, ":2,");
+if (!flags)
+   return -1;
+flags += 3;
+
+/*  Check that the letters are valid Maildir flags */
+f = 0;
+valid = true;
+for (p=flags; valid && *p; p++) {
+   switch (*p) {
+   case 'P':
+   case 'R':
+   case 'S':
+   case 'T':
+   case 'D':
+   case 'F':
+   if (*p > f) f=*p;
+   else valid = false;
+   break;
+   default:
+   valid = false;
+   }
+}
+if (!valid) {
+   fprintf (stderr, "Warning: Invalid maildir flags in filename %s\n", 
filename);
+   return -1;
+}
+
+notmuch_message_freeze(message);
+unread = true;
+for (i = 0; i < ARRAY_SIZE(flag2tag); i++) {
+   if ((strchr(flags, flag2tag[i].flag) != NULL) ^ flag2tag[i].inverse) {
+   notmuch_message_add_tag (message, flag2tag[i].tag);
+   } else {
+   notmuch_message_remove_tag (message, flag2tag[i].tag);
+ 

[PATCH] Mailstore abstraction v4 - part 2 (maildir synchronization)

[Michal Sojka]

This is the second part of mailstore abstraction patches. I do not
want this to be merged yet, but there might be some people interested
in testing this.

This patch adds a mailstore, which bi-directionally synchronizes
certain tags with maildir flags.

I use it already four weeks and it works quite well. There are the
following know bugs:

1) Viewing/storing of attachments of unread messages doesn't work. The
   reason is that when you view the message its unread tag is removed
   by elisp code. This leads to rename of the file, but Emacs still
   uses the original name to access message with the attachment.

   Workaround: close the message and open it again.

   I'm working on the solution - if the mailstore cannot open the
   message with the name passed, it tries different names with
   different maildir flags.

2) If there several messages with the same ID (e.g. one in sent folder
   and one sent back by mailing list), the flags are synchronized to
   only one of these files.

   I plan to do this:

   - When a tag is added/removed in notmuch, flags of all
 files corresponding to the message will be updated.
   - If we detect (during notmuch new) that flags of one file were
 changed by somebody else, we also change the flags for the other
 files corresponding to the message.
   - If we detect (during notmuch new) that flags of two or more files
 were changed by somebody else, we have to solve the conflict
 somehow, but I didn't invent how, yet.

The full series is available at
http://rtime.felk.cvut.cz/gitweb/notmuch.git/shortlog/refs/heads/mailstore-abstraction-v4
and can be pulled by

   git pull git://rtime.felk.cvut.cz/notmuch.git mailstore-abstraction-v4

Besides the patch sent here, there are also tests for the maildir
mailstore and a not finished implementation of the solution for 1)
above.

--Michal

[PATCH 4/4] Add 'cat' subcommand

[Michal Sojka]

This command dumps raw message identified by filename to standard
output. It uses mail store interface to get the message from the right
place.

notmuch.el was modified to use this command to access the raw message
content (view/save attachments, view raw message and pipe message to
command).

With this patch, it is straightforward to use notmuch emacs interface
remotely over SSH. To do this, it is sufficient to redefine
notmuch-command variable to contain the name of a script containing:

ssh user at host notmuch "$@"

Signed-off-by: Michal Sojka 
---
 NEWS  |3 ++
 emacs/notmuch-show.el |   11 ++--
 notmuch-client.h  |3 ++
 notmuch-show.c|   62 +
 notmuch.c |4 +++
 5 files changed, 80 insertions(+), 3 deletions(-)

diff --git a/NEWS b/NEWS
index f29ac27..3bd21fa 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,6 @@
+A new subcommand 'cat' was added, which simplifies use of Emacs
+interface with remote database (accessed over SSH).
+
 Notmuch 0.1 (2010-04-05)
 
 This is the first release of the notmuch mail system.
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6f5a55d..45d49d1 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -328,7 +328,11 @@ buffer."
 (defun notmuch-show-view-raw-message ()
   "View the raw email of the current message."
   (interactive)
-  (view-file (notmuch-show-get-filename)))
+  (let ((filename (notmuch-show-get-filename)))
+(let ((buf (get-buffer-create (concat "*notmuch-raw-" filename "*"
+  (switch-to-buffer buf)
+  (save-excursion
+   (call-process notmuch-command nil t nil "cat" filename)

 (defmacro with-current-notmuch-show-message ( body)
   "Evaluate body with current buffer set to the text of current message"
@@ -336,7 +340,7 @@ buffer."
  (let ((filename (notmuch-show-get-filename)))
(let ((buf (generate-new-buffer (concat "*notmuch-msg-" filename "*"
  (with-current-buffer buf
-   (insert-file-contents filename nil nil nil t)
+   (call-process notmuch-command nil t nil "cat" filename)
, at body)
 (kill-buffer buf)

@@ -390,7 +394,8 @@ current email message as stdin. Anything printed by the 
command
 to stdout or stderr will appear in the *Messages* buffer."
   (interactive "sPipe message to command: ")
   (apply 'start-process-shell-command "notmuch-pipe-command" "*notmuch-pipe*"
-(list command " < " (shell-quote-argument 
(notmuch-show-get-filename)
+(list notmuch-command "cat"
+  (shell-quote-argument (notmuch-show-get-filename) " | " 
command

 (defun notmuch-show-move-to-current-message-summary-line ()
   "Move to the beginning of the one-line summary of the current message.
diff --git a/notmuch-client.h b/notmuch-client.h
index 728e448..666e70d 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -111,6 +111,9 @@ int
 notmuch_search_tags_command (void *ctx, int argc, char *argv[]);

 int
+notmuch_cat_command (void *ctx, int argc, char *argv[]);
+
+int
 notmuch_part_command (void *ctx, int argc, char *argv[]);

 const char *
diff --git a/notmuch-show.c b/notmuch-show.c
index 66fd773..dbab9a7 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -515,6 +515,68 @@ notmuch_show_command (void *ctx, unused (int argc), unused 
(char *argv[]))
 }

 int
+notmuch_cat_command (void *ctx, unused (int argc), unused (char *argv[]))
+{
+notmuch_config_t *config;
+notmuch_database_t *notmuch;
+notmuch_mailstore_t *mailstore;
+int i;
+FILE *file;
+const char *filename;
+size_t size;
+char buf[4096];
+
+for (i = 0; i < argc && argv[i][0] == '-'; i++) {
+/* if (STRNCMP_LITERAL (argv[i], "--part=") == 0) { */
+/* part = atoi(argv[i] + sizeof ("--part=") - 1); */
+/* } else { */
+   fprintf (stderr, "Unrecognized option: %s\n", argv[i]);
+/* return 1; */
+/* } */
+}
+
+argc -= i;
+argv += i;
+
+if (argc == 0) {
+   fprintf (stderr, "Error: No filename given\n");
+   return 1;
+}
+filename = argv[0];
+
+config = notmuch_config_open (ctx, NULL, NULL);
+if (config == NULL)
+   return 1;
+
+mailstore = notmuch_config_get_mailstore (config);
+
+if (mailstore == NULL) {
+   fprintf (stderr, "Error: I have no mailstore\n");
+   return 1;
+}
+
+notmuch = notmuch_database_open (notmuch_config_get_database_path (config),
+NOTMUCH_DATABASE_MODE_READ_ONLY,
+mailstore);
+
+file = notmuch_mailstore_open_file(mailstore, filename);
+if (file == NULL) {
+   fprintf (stderr, "Error: Cannot open %s in %s: %s\n", filename,
+notmuch_mailstore_get_type (mailstore), strerror (errno));
+   return 1;
+}
+while  (!feof (file)) {
+   size = fread(buf, 1, sizeof(buf), file);
+   

[PATCH 3/4] Access messages through mail store interface

[Michal Sojka]

This patch modifies notmuch binary to access the messages through mail
store interface, so that non-file based mail stores can also be
implemented.

The API of notmuch library was changed. Now,
notmuch_message_get_filename() returns relative file name with respect
to the database path. As a result, notmuch show also outputs relative
paths so that MUAs need to be changed.

Signed-off-by: Michal Sojka 
---
 lib/database.cc   |   14 +++---
 lib/index.cc  |8 ++--
 lib/mailstore-files.c |   18 +-
 lib/message-file.c|8 
 lib/message.cc|   33 +
 lib/notmuch-private.h |6 +++---
 lib/notmuch.h |   16 ++--
 lib/sha1.c|6 +-
 notmuch-client.h  |2 +-
 notmuch-reply.c   |   10 +-
 notmuch-show.c|   14 --
 show-message.c|   14 +-
 12 files changed, 104 insertions(+), 45 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 93c8d0f..bd64ed3 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1374,6 +1374,7 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
 notmuch_message_t *message = NULL;
 notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
 notmuch_private_status_t private_status;
+const char *relative;

 const char *date, *header;
 const char *from, *to, *subject;
@@ -1386,7 +1387,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
 if (ret)
return ret;

-message_file = notmuch_message_file_open (filename);
+relative = _notmuch_database_relative_path (notmuch, filename);
+message_file = notmuch_message_file_open (notmuch->mailstore, relative);
 if (message_file == NULL)
return NOTMUCH_STATUS_FILE_ERROR;

@@ -1438,9 +1440,15 @@ notmuch_database_add_message (notmuch_database_t 
*notmuch,
}

if (message_id == NULL ) {
+   FILE *file;
+   char *sha1 = NULL;
/* No message-id at all, let's generate one by taking a
 * hash over the file's contents. */
-   char *sha1 = notmuch_sha1_of_file (filename);
+   file = notmuch_mailstore_open_file (notmuch->mailstore, relative);
+   if (file) {
+   sha1 = notmuch_sha1_of_file (file);
+   fclose (file);
+   }

/* If that failed too, something is really wrong. Give up. */
if (sha1 == NULL) {
@@ -1483,7 +1491,7 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
date = notmuch_message_file_get_header (message_file, "date");
_notmuch_message_set_date (message, date);

-   _notmuch_message_index_file (message, filename);
+   _notmuch_message_index_file (message, relative);
} else {
ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
}
diff --git a/lib/index.cc b/lib/index.cc
index cf93025..4d8c4dd 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -425,15 +425,19 @@ _notmuch_message_index_file (notmuch_message_t *message,
 const char *from, *subject;
 notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
 static int initialized = 0;
+notmuch_mailstore_t *mailstore;

 if (! initialized) {
g_mime_init (0);
initialized = 1;
 }

-file = fopen (filename, "r");
+mailstore = notmuch_message_get_mailstore(message);
+file = notmuch_mailstore_open_file (mailstore, filename);
 if (! file) {
-   fprintf (stderr, "Error opening %s: %s\n", filename, strerror (errno));
+   fprintf (stderr, "Error opening %s: %s\n",
+notmuch_message_get_filename (message),
+strerror (errno));
ret = NOTMUCH_STATUS_FILE_ERROR;
goto DONE;
 }
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
index 8b5e1d5..f2cb8d7 100644
--- a/lib/mailstore-files.c
+++ b/lib/mailstore-files.c
@@ -602,11 +602,27 @@ index_new(notmuch_mailstore_t *mailstore, const char* 
path,
 return ret;
 }

+static FILE *
+open_file(notmuch_mailstore_t *mailstore, const char *filename)
+{
+const char *db_path;
+char *abs_filename;
+FILE *file;
+
+db_path = notmuch_database_get_path(mailstore->notmuch);
+abs_filename = talloc_asprintf(NULL, "%s/%s", db_path, filename);
+if (unlikely(abs_filename == NULL))
+   return NULL;
+file = fopen (abs_filename, "r");
+talloc_free(abs_filename);
+return file;
+}
+
 /* Original notmuch mail store */
 struct _notmuch_mailstore mailstore_files = {
 .type = "files",
 .count_files = count_files,
 .index_new = index_new,
 .sync_tags = NULL, /* We cannot store tags in this mailstore. */
-.open_file = NULL, /* Currently not implemented */
+.open_file = open_file,
 };
diff --git a/lib/message-file.c b/lib/message-file.c
index 0c152a3..13c9f4c 100644
--- a/lib/message-file.c
+++ b/lib/message-file.c
@@ -94,7 +94,7 @@ 

[PATCH 2/4] Conversion to mailstore abstraction

[Michal Sojka]

The code for detection of new files in the mailstore and their
addition to the database is moved from notmuch-new.c to
lib/mailstore-files.c, where it is called by the abstract mailstore
interface.

The code was only changed to allow the progress reporting functions to
be implemented outside of notmuch library, as can be seen by
git diff HEAD^ -C

Signed-off-by: Michal Sojka 
---
 lib/Makefile.local |1 +
 notmuch-new.c => lib/mailstore-files.c |  406 
 lib/mailstore.c|5 -
 notmuch-new.c  |  660 +++-
 4 files changed, 141 insertions(+), 931 deletions(-)
 copy notmuch-new.c => lib/mailstore-files.c (61%)

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 6243af1..863df4c 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -32,6 +32,7 @@ extra_cflags += -I$(dir) -fPIC
 libnotmuch_c_srcs =\
$(dir)/libsha1.c\
$(dir)/mailstore.c  \
+   $(dir)/mailstore-files.c\
$(dir)/message-file.c   \
$(dir)/messages.c   \
$(dir)/sha1.c   \
diff --git a/notmuch-new.c b/lib/mailstore-files.c
similarity index 61%
copy from notmuch-new.c
copy to lib/mailstore-files.c
index 2d0ba6c..8b5e1d5 100644
--- a/notmuch-new.c
+++ b/lib/mailstore-files.c
@@ -1,4 +1,5 @@
-/* notmuch - Not much of an email program, (just index and search)
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
  *
  * Copyright ?? 2009 Carl Worth
  *
@@ -15,12 +16,14 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see http://www.gnu.org/licenses/ .
  *
- * Author: Carl Worth 
+ * Authors: Carl Worth 
+ * Michal Sojka 
  */

-#include "notmuch-client.h"
-
-#include 
+#define _GNU_SOURCE/* For asprintf() */
+#include "notmuch.h"
+#include "mailstore-private.h"
+#include 

 typedef struct _filename_node {
 char *filename;
@@ -32,38 +35,10 @@ typedef struct _filename_list {
 _filename_node_t **tail;
 } _filename_list_t;

-typedef struct {
-int output_is_a_tty;
-int verbose;
-
-int total_files;
-int processed_files;
-int added_messages;
-struct timeval tv_start;
-
+typedef struct _indexing_context_priv {
 _filename_list_t *removed_files;
 _filename_list_t *removed_directories;
-} add_files_state_t;
-
-static volatile sig_atomic_t do_add_files_print_progress = 0;
-
-static void
-handle_sigalrm (unused (int signal))
-{
-do_add_files_print_progress = 1;
-}
-
-static volatile sig_atomic_t interrupted;
-
-static void
-handle_sigint (unused (int sig))
-{
-ssize_t ignored;
-static char msg[] = "Stopping... \n";
-
-ignored = write(2, msg, sizeof(msg)-1);
-interrupted = 1;
-}
+} _indexing_context_priv_t;

 static _filename_list_t *
 _filename_list_create (const void *ctx)
@@ -100,34 +75,6 @@ tag_inbox_and_unread (notmuch_message_t *message)
 notmuch_message_add_tag (message, "unread");
 }

-static void
-add_files_print_progress (add_files_state_t *state)
-{
-struct timeval tv_now;
-double elapsed_overall, rate_overall;
-
-gettimeofday (_now, NULL);
-
-elapsed_overall = notmuch_time_elapsed (state->tv_start, tv_now);
-rate_overall = (state->processed_files) / elapsed_overall;
-
-printf ("Processed %d", state->processed_files);
-
-if (state->total_files) {
-   double time_remaining;
-
-   time_remaining = ((state->total_files - state->processed_files) /
- rate_overall);
-   printf (" of %d files (", state->total_files);
-   notmuch_time_print_formatted_seconds (time_remaining);
-   printf (" remaining).  \r");
-} else {
-   printf (" files (%d files/sec.)\r", (int) rate_overall);
-}
-
-fflush (stdout);
-}
-
 static int
 dirent_sort_inode (const struct dirent **a, const struct dirent **b)
 {
@@ -169,6 +116,7 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
 return 0;
 }

+
 /* Examine 'path' recursively as follows:
  *
  *   o Ask the filesystem for the mtime of 'path' (fs_mtime)
@@ -205,9 +153,9 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
  *   o Tell the database to update its time of 'path' to 'fs_mtime'
  */
 static notmuch_status_t
-add_files_recursive (notmuch_database_t *notmuch,
+add_files_recursive (notmuch_mailstore_t *mailstore,
 const char *path,
-add_files_state_t *state)
+notmuch_indexing_context_t *state)
 {
 DIR *dir = NULL;
 struct dirent *entry = NULL;
@@ -222,6 +170,8 @@ add_files_recursive (notmuch_database_t *notmuch,
 notmuch_filenames_t *db_subdirs = NULL;
 struct stat st;
 notmuch_bool_t is_maildir, new_directory;
+_indexing_context_priv_t *priv = state->priv;
+notmuch_database_t *notmuch = 

[PATCH 1/4] Mailstore abstraction interface

[Michal Sojka]

The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. Currently,
there exists implementation for two types of mailstore - plain files
and maildir. It is expected that additional git-based mailstore will
be developed later.

This patch contains only the interface changes. No functionality is
added, removed or changed.

A new configuration group [mailstore] is defined. Currently, there is
only one key called 'type' whose value determines the used mailstore.
The default value of this option (files) is the plain-file mailstore
currently implemented in notmuch.

This patch changes libnotmuch API (and ABI). The functions
notmuch_database_create() and notmuch_database_open() need additional
parameter to identify the type of mailstore, which is used to access
the messages.

If we want to preserve the API, it would be necessary to encode the
mailstore type into the 'path' parameter. For example the value
# (or ://?) would mean use the mailstore of
given 'type' located at 'path'. If we cannot find the type we would
assuse the default mailstore.

Signed-off-by: Michal Sojka 
---
 lib/Makefile.local  |1 +
 lib/database-private.h  |1 +
 lib/database.cc |   15 ++--
 lib/mailstore-private.h |   59 
 lib/mailstore.c |   85 +++
 lib/message.cc  |   13 +++
 lib/notmuch.h   |   82 
 notmuch-client.h|7 
 notmuch-config.c|   34 +++
 notmuch-count.c |3 +-
 notmuch-dump.c  |3 +-
 notmuch-new.c   |6 ++-
 notmuch-reply.c |3 +-
 notmuch-restore.c   |3 +-
 notmuch-search-tags.c   |3 +-
 notmuch-search.c|3 +-
 notmuch-show.c  |6 ++-
 notmuch-tag.c   |3 +-
 18 files changed, 307 insertions(+), 23 deletions(-)
 create mode 100644 lib/mailstore-private.h
 create mode 100644 lib/mailstore.c

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 0e3a4d1..6243af1 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -31,6 +31,7 @@ extra_cflags += -I$(dir) -fPIC

 libnotmuch_c_srcs =\
$(dir)/libsha1.c\
+   $(dir)/mailstore.c  \
$(dir)/message-file.c   \
$(dir)/messages.c   \
$(dir)/sha1.c   \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
 Xapian::TermGenerator *term_gen;
 Xapian::ValueRangeProcessor *value_range_processor;

+notmuch_mailstore_t *mailstore;
 };

 /* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
  */

 #include "database-private.h"
+#include "mailstore-private.h"

 #include 

@@ -438,7 +439,7 @@ parse_references (void *ctx,
 }

 notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
 }

 notmuch = notmuch_database_open (path,
-NOTMUCH_DATABASE_MODE_READ_WRITE);
+NOTMUCH_DATABASE_MODE_READ_WRITE,
+mailstore);
 notmuch_database_upgrade (notmuch, NULL, NULL);

   DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t 
*notmuch)

 notmuch_database_t *
 notmuch_database_open (const char *path,
-  notmuch_database_mode_t mode)
+  notmuch_database_mode_t mode,
+  notmuch_mailstore_t *mailstore)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = _PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+   notmuch->mailstore = mailstore;
+   mailstore->notmuch = notmuch;
 } catch (const Xapian::Error ) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
 error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,

   DONE:
 if (message) {
-   if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+   if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy 

[PATCH 0/4] Mailstore abstraction v4

[Michal Sojka]

Hi all,

this is the fourth version of my mailstore abstraction series. I split
it into two parts, with the first part being sent here. I think that
this part becomes mostly ready for merging.

>From the user's point of view, it adds only the 'cat' subcommand and
modifies the .el to use it. Thanks to this change it is easy to use
Emacs client with the database accessed remotely over SSH. The only
additional patch that must be applied is
id:m1my03gsmu.fsf at watt.gilman.jhu.edu. The rebased version is at
http://rtime.felk.cvut.cz/gitweb/notmuch.git/shortlog/refs/heads/jr/quote-args-in-notmuch-show
and can be pulled by 'git pull git://rtime.felk.cvut.cz/notmuch.git
jr/quote-args-in-notmuch-show').

More importantly, there are bigger changes for developers. Since it
might be quite painful to rebase these patches to quickly changing
master (as of the last few days) I'd like to have these merged ASAP.

So the patches here are the following:

Michal Sojka (4):
  Mailstore abstraction interface
  Conversion to mailstore abstraction
  Access messages through mail store interface
  Add 'cat' subcommand

My biggest question relates to the first patch, which does an
incompatible change to libnotmuch API. After reading RELEASING file, I
found that this change is probably not what Carl wants to merge (and I
understand that) so I'd like to get some feedback on my suggestion in
that patch.

The subsequent patches are quite straightforward and the comments
there should describe what these patches do.

These patches are also located at git://rtime.felk.cvut.cz/notmuch.git
and tagged by mailstore-abstraction-v4-part1.

This is the overall diffstat:
 NEWS   |3 +
 emacs/notmuch-show.el  |   11 +-
 lib/Makefile.local |2 +
 lib/database-private.h |1 +
 lib/database.cc|   29 +-
 lib/index.cc   |8 +-
 notmuch-new.c => lib/mailstore-files.c |  418 +
 lib/mailstore-private.h|   59 +++
 lib/mailstore.c|   80 
 lib/message-file.c |8 +-
 lib/message.cc |   46 ++-
 lib/notmuch-private.h  |6 +-
 lib/notmuch.h  |   98 +-
 lib/sha1.c |6 +-
 notmuch-client.h   |   12 +-
 notmuch-config.c   |   34 ++
 notmuch-count.c|3 +-
 notmuch-dump.c |3 +-
 notmuch-new.c  |  658 +++-
 notmuch-reply.c|   13 +-
 notmuch-restore.c  |3 +-
 notmuch-search-tags.c  |3 +-
 notmuch-search.c   |3 +-
 notmuch-show.c |   82 -
 notmuch-tag.c  |3 +-
 notmuch.c  |4 +
 show-message.c |   14 +-
 27 files changed, 621 insertions(+), 989 deletions(-)
 copy notmuch-new.c => lib/mailstore-files.c (61%)
 create mode 100644 lib/mailstore-private.h
 create mode 100644 lib/mailstore.c

Plans for the 0.2 release (this week)

[Michal Sojka]

On Wed, 07 Apr 2010, Carl Worth wrote:
> For the upcoming 0.2 release, here are some things that I would like to
> have in place:
> 
>   * Anything else that people want, (especially things that already
> exist and that you're already using). Support for maildir flags on
> import would be great here. I'm still waiting to see a complete
> solution I think.

This would be useful: id:m1my03gsmu.fsf at watt.gilman.jhu.edu. I use it
occasionally.

-Michal

[PATCH 3/3] document new `notmuch new` behavior

[Mike Kelly]

---
 notmuch.1 |8 +---
 notmuch.c |7 ---
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/notmuch.1 b/notmuch.1
index 0e6a2ed..8b4021e 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -93,9 +93,11 @@ The
 .B new
 command scans all sub-directories of the database, performing
 full-text indexing on new messages that are found. Each new message
-will automatically be tagged with both the
-.BR inbox " and " unread
-tags.
+will automatically be tagged with the
+.BR inbox
+tag, and, unless it was already "seen" by another client, the
+.BR unread
+tag.

 You should run
 .B "notmuch new"
diff --git a/notmuch.c b/notmuch.c
index f5669fc..9002e7d 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -125,9 +125,10 @@ command_t commands[] = {
 { "new", notmuch_new_command,
   "[--verbose]",
   "Find and import new messages to the notmuch database.",
-  "\tScans all sub-directories of the mail directory, performing\n"
-  "\tfull-text indexing on new messages that are found. Each new\n"
-  "\tmessage will be tagged as both \"inbox\" and \"unread\".\n"
+  "\t\tScans all sub-directories of the mail directory, performing\n"
+  "\t\tfull-text indexing on new messages that are found. Each new\n"
+  "\t\tmessage will be tagged as \"inbox\" and, unless it is\n"
+  "\t\tmarked as \"seen\", \"unread\".\n"
   "\n"
   "\tYou should run \"notmuch new\" once after first running\n"
   "\t\"notmuch setup\" to create the initial database. The first\n"
-- 
1.7.0.4

[PATCH 2/3] apply all the other maildir flag->tag conversions

[Mike Kelly]

---
 notmuch-new.c |   10 ++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/notmuch-new.c b/notmuch-new.c
index 511347d..dc33d69 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -99,6 +99,16 @@ tag_inbox_and_unread (notmuch_message_t *message)
 notmuch_message_add_tag (message, "inbox");
 if (! notmuch_message_md_flag(message, 'S'))
 notmuch_message_add_tag (message, "unread");
+if (notmuch_message_md_flag(message, 'T'))
+notmuch_message_add_tag (message, "deleted");
+if (notmuch_message_md_flag(message, 'D'))
+notmuch_message_add_tag (message, "draft");
+if (notmuch_message_md_flag(message, 'F'))
+notmuch_message_add_tag (message, "flagged");
+if (notmuch_message_md_flag(message, 'P'))
+notmuch_message_add_tag (message, "passed");
+if (notmuch_message_md_flag(message, 'R'))
+notmuch_message_add_tag (message, "replied");
 }

 static void
-- 
1.7.0.4

[PATCH 1/3] Initial support for maildir flags.

[Mike Kelly]

When adding new messages, if they have the 'S' (seen) flag, do not add
them to the 'unread' tag.
---
 lib/message.cc |   25 +
 lib/notmuch.h  |5 +
 notmuch-new.c  |3 ++-
 3 files changed, 32 insertions(+), 1 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 721c9a6..2ca1562 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -509,6 +509,31 @@ notmuch_message_set_flag (notmuch_message_t *message,
message->flags &= ~(1 << flag);
 }

+notmuch_bool_t
+notmuch_message_md_flag (notmuch_message_t *message,
+const char flag)
+{
+const char *filename;
+const char *p;
+
+filename = notmuch_message_get_filename (message);
+
+p = strstr (filename, ":2,");
+if (p == NULL) {
+/* Not a valid maildir filename */
+return FALSE;
+}
+
+for (p += 3; *p != '\0'; p++) {
+if (*p == flag) {
+return TRUE;
+}
+}
+
+return FALSE;
+}
+
+
 time_t
 notmuch_message_get_date (notmuch_message_t *message)
 {
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 88da078..018c002 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -763,6 +763,11 @@ void
 notmuch_message_set_flag (notmuch_message_t *message,
  notmuch_message_flag_t flag, notmuch_bool_t value);

+/* See if a given maildir flag is set, based on the message's filename. */
+notmuch_bool_t
+notmuch_message_md_flag (notmuch_message_t *message,
+const char flag);
+
 /* Get the date of 'message' as a time_t value.
  *
  * For the original textual representation of the Date header from the
diff --git a/notmuch-new.c b/notmuch-new.c
index 44b50aa..511347d 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -97,7 +97,8 @@ static void
 tag_inbox_and_unread (notmuch_message_t *message)
 {
 notmuch_message_add_tag (message, "inbox");
-notmuch_message_add_tag (message, "unread");
+if (! notmuch_message_md_flag(message, 'S'))
+notmuch_message_add_tag (message, "unread");
 }

 static void
-- 
1.7.0.4

[PATCH] Have notmuch count default to showing the total.

[Mike Kelly]

If no parameters are given to notmuch-count, or just '' or '*' are
given, return the total number of messages in the database.

update notmuch count help
---
 notmuch-count.c |5 ++---
 notmuch.c   |4 
 2 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/notmuch-count.c b/notmuch-count.c
index 77aa433..97242ab 100644
--- a/notmuch-count.c
+++ b/notmuch-count.c
@@ -90,9 +90,8 @@ notmuch_count_command (void *ctx, int argc, char *argv[])
fprintf (stderr, "Out of memory.\n");
return 1;
 }
-if (*query_str == '\0') {
-   fprintf (stderr, "Error: notmuch count requires at least one count 
term.\n");
-   return 1;
+if (*query_str == '\0' || (*query_str == '*' && *(query_str+1) == '\0')) {
+   query_str = talloc_strdup (ctx, "");
 }

 query = notmuch_query_create (notmuch, query_str);
diff --git a/notmuch.c b/notmuch.c
index f5669fc..8650951 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -220,6 +220,10 @@ command_t commands[] = {
   "\tof messages matching both a specific tag and either inbox\n"
   "\tor unread\n"
   "\n"
+  "\tIf no parameters are given, or the special search terms '' or\n"
+  "\t'*' are given, it will display the total number of messages in\n"
+  "\tthe database.\n"
+  "\n"
   "\tSee \"notmuch help search-terms\" for details of the search\n"
   "\t\tterms syntax." },
 { "reply", notmuch_reply_command,
-- 
1.7.0.4

[notmuch] [Sebastian Spaeth] Pull requests

[Michal Sojka]

On Wed, 07 Apr 2010, micah anderson wrote:
> On 2010-03-27, micah anderson wrote:
> > On Thu, 25 Mar 2010 22:50:52 -0400, micah anderson  
> > wrote:
> > > On Mon, 01 Mar 2010 15:57:00 +0100, "Sebastian Spaeth"  > > SSpaeth.de> wrote:
> > > 
> > > > From git repository git://github.com/spaetz/notmuch-all-feature.git I
> > > > would like to advocate the following branches for quick pulling. Each
> > > > contains 1 or 2 patches. They have all been based on todays
> > > > cworth/master, so it should be really painless.
> > > 
> > > Thanks for pulling these all together! All the ones that you propose I
> > > also use and would really like these to be merged as well.
> > > 
> > > The only other patch that I find absolutely crucial, that you do not
> > > include, is the 'Preserve folder information when indexing' patch which,
> > > although not perfect, does significantly change my life. 
> > 
> > Glad you find it useful. Yes, having the folder information would indeed
> > be nice. Is that patch working well for you? (Can you point me to the
> > mail ID of the patch you are using? There have been several versions
> > around).
> 
> The patch works really well for me. I also had difficulty figuring out
> which was the latest.

I think that the patch you sent was not the latest version. The latest
is in id:1265122868-12133-1-git-send-email-sojkam1 at fel.cvut.cz, but it
is not rebased to the current master.

I do not use this patch right now, since I didn't find time to apply it
on top of my mailstore abstraction patches. What I remember from when I
used it is, that it was really problematic with mail sent by me to
mailing lists. There were two version of the message - one in sent
folder and one in list folder - and the "folder tag" was not assigned
deterministically.

I think that now I know how to implement this correctly, I only need
some time. Also, the problem is very similar to synchronization of
maildir flags to multiple files, which is what I'm solving right now. So
probably, after I finish maildir flag synchronization I can work on this
patch.

-Michal

[notmuch] [PATCH] notmuch-new: Respect maildir flags when importing a new message

[Michal Sojka]

On Wed, 07 Apr 2010, Carl Worth wrote:
> On Mon,  1 Mar 2010 14:28:56 +0100, Sebastian Spaeth  SSpaeth.de> wrote:
> > When importing a new mail do check for maildir tags and assign
> > corresponding notmuch tags.
> 
> I think this is a useful thing to support, as obviously new users have
> *some* state that's interesting to import (which messages have been
> "seen" for example), and simply importing their entire email archive
> with both the "inbox" and "unread" tags is not helpful. So I'd like to
> figure out how to support this.

I'm solving this in my mailstore abstraction patches. I'll send the next
version in a while.

> > Do note that this will only add tags when importing a really new
> > message, and will not do anything when detecting a file rename
> > (although someone should really make it honor file renames as
> > well). Deleteing an existing message in another IMAP client will
> > therefore not trigger tagging (as it counts as a file rename).
> 
> But I think we really need to fix that if we're going to claim that
> notmuch "supports maildir flags" in any sense.
> 
> It's a fairly tricky issue though since we can have multiple files that
> have the same message ID, but then have different maildir flags. And
> it's the matter of arbitrary ordering which one of these files appears
> as "new" and which one appears as a "rename".

Yes, that's a problem. It is not currently solved in my pacthes, but I
have one solution in my mind. Let's discuss this with my patch.

> It's not obvious to me what we can do here unless we make some
> assumptions, (such as "mails always start without the seen flag, and
> once it appears it can't be removed" or so). But I'd love some input
> From someone who's thought harder about this than I have.

-Michal

[PATCH] Derive version numbers from git

[Michal Sojka]

On Wed, 07 Apr 2010, Carl Worth wrote:
> On Tue, 06 Apr 2010 18:11:28 +0200, Michal Sojka  
> wrote:
> >  On Tue, 06 Apr 2010, Sebastian Spaeth wrote:
> >
> >  > But, there are people without git installed that download the release
> >  > tarball. So if this patch makes it, we need to replace this with a
> >  > static version number when baking a release tar.
> > 
> > Right, here is an updated patch. It's more complicated than the previous
> > one, but it solves the issue.
> 
> I like this idea, definitely.
> 
> But the other piece needed here is a way for me to be able to specify a
> version in order to release. In my current release instructions
> (notmuch/RELEASING) I do that by manually incrementing the version
> number in the Makefile. Then a tag is created later when the "make
> release" target runs.

> It would be possible for me to instead create the tag to specify the
> version, but there's a few things I don't like about this:
> 
> 1. After I increment the version number (early in the release process) I
>often find one or two little things I need to change to make the
>release perfect. So there are likely more commits later, but I
>obviously don't want some git-describe version to end up in the final
>tar file.
> 
> 2. I don't actually want to create a tag, (I *especially* don't want to
>push it), until the release actually happens. That's the point of the
>tag in my view, (to say "*this* is what I released"), so making the
>tag early seems wrong, (and leaves the door open to make mistakes).
> 
> Any suggestion for this part?

I have modified the patch slightly and I think that it could solve the
above points. The release process should be modified this way: you skip
point 5 (increment the notmuch version in Makefile.local) and in point
6, you run 'make release VERSION=X.Y'.

> > +include Makefile.version
> > +
> > +.PHONY: Makefile.version
> > +Makefile.version:
> > +   echo VERSION=$(if $(wildcard version),`cat version`,`git describe 
> > --dirty`) > $@
> 
> Could that be simplified to just use $(shell) rather than a separate
> file and an include?

Done. I have removed the --dirty option, because the dirty suffix
appeared every time I built debian package with git-buildpackage, and I
have added --match to match only release tags and not debian release
tags. That would probably mean that if you release the debian package
later than notmuch, the binary in debian package would have wrong
version compiled in. A solution could be that debian releases would live
in a different branch which will contain 'version' file and this branch
(and file) will be updated by make release.

> I suppose what we could do is to just have the creation of the version
> file be part of the release process. That would be simple enough. Then
> we could drop this rule:
> 
> > +version:
> > +   git describe > $@

I put it completely as part of TAR_FILE creation. If it was only
generated during release process, make dist would produce non-compilable
archives.

-Michal

--8<---cut here---start->8---
>From 26b9aad1143910a198c2d7263312f9b631edc022 Mon Sep 17 00:00:00 2001
From: Michal Sojka 
Date: Tue, 6 Apr 2010 18:01:43 +0200
Subject: [PATCH] Derive version numbers from git

I often have several versions of notmuch compiled and it would be very
helpful to be able to distinguish between them. Git has a very nice
feature to make intermediate numbering automatic and unambiguous so
let's use it here.

For tagged versions, the version is the name of the tag, for
intermediate versions, the unique ID of the commit is appended to the
tag name.

When notmuch is compiled from a release tarball, there is no git
repository and therefore the tarball contains a special file 'version',
which contains the version of release tarball.

To create a new release one has to run 'make release VERSION=X.Y'.
---
 Makefile.local |   15 +--
 1 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index 6882324..a26d6a0 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -7,11 +7,10 @@
 # digit when we reach particularly major milestones of usability.
 #
 # Between releases, (such as when compiling notmuch from the git
-# repository), we add a third digit, (0.1.1, 0.1.2, etc.), and
-# increment it occasionally, (such as after a big batch of commits are
-# merged.
+# repository), we let git to append identification of the actual
+# commit.
 PACKAGE=notmuch
-VERSION=0.1.1
+VERSION:=$(shell if [ -f version ]; then cat version; else git describe 
--match '[0-9].[0-9]*'; fi)

 RELEASE_HOST=notmuchmail.org
 RELEASE_DIR=/srv/notmuchmail.org/www/releases
@@ -63,7 +62,11 @@ endif
 endif

 $(TAR_FILE):
-   git archive --format=tar --prefix=$(PACKAGE)-$(VERSION)/ HEAD | gzip > 
$(TAR_FILE)
+   git archive --format=tar --prefix=$(PACKAGE)-$(VERSION)/ HEAD > 
$(TAR_FILE).tmp
+   echo $(VERSION) > version
+   tar 

draft handling [was: Re: Plans for the 0.2 release (this week)]

[Jameson Rollins]

On Thu, 08 Apr 2010 07:58:47 -0700, Dirk Hohndel  
wrote:
> > And as for drafts, they could be easily indexed and managed by the UI if
> > the folder: search term is available.
> 
> Hmm - haven't even thought about drafts, yet. How would the UI deal with
> those? 

My idea is the following:

Currently for me drafts are stored in ~/.mail/drafts.  If we have
folder: searching, then I can starting indexing my drafts directory and
apply the following post-new filter:

notmuch tag +draft -inbox -unread -- folder:drafts

I'll then make a simple elisp "resume" function that is available for
resuming draft messages, that would be available when viewing messages

Plans for the 0.2 release (this week)

[Mike Kelly]

On Thu, Apr 08, 2010 at 10:03:15AM -0400, Jameson Rollins wrote:
> On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
> > For the upcoming 0.2 release, here are some things that I would like to
> > have in place:
> > 
> >   * Changes to indexing, (addition of body:, folder:, list:, etc.). This
> > is stuff that I'll work on.
> 
> Also, fwiw, the folder: indexing is probably the new feature that I'm
> most eagerly awaiting.  I've got all these ideas for ways to handle sent
> mail and drafts if we can get this working.

For me this, along with handling maildir flags, are pretty much my top
"must haves" before I think I would be able to begin to replace
mutt/whatever with notmuch.

-- 
Mike Kelly

Plans for the 0.2 release (this week)

[Mike Kelly]

On Wed, Apr 07, 2010 at 11:23:14PM +0100, James Westby wrote:
> On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
> >   * Anything else that people want, (especially things that already
> > exist and that you're already using). Support for maildir flags on
> > import would be great here. I'm still waiting to see a complete
> > solution I think.
> 
> id:1268515677-12692-1-git-send-email-jw+debian at jameswestby.net

I haven't tested yet, but this is definitely a feature I want/need (due
to the... unique behavior of a certain bug tracker I use at work).

-- 
Mike Kelly

RFC: User-Agent header

[Sebastian Spaeth]

notmuch is (mostly) not responsible for sending email. However, people
using the emacs frontend use notmuch to create the reply.

Am I the only one who is sometimes curious as to what mail agents others
use? Would it be useful to insert a header to notmuch reply analog to:

User-Agent: Mutt/1.5.13 (2006-08-11)

We could reuse the same version string that we use for the release (or
the git string that was used to build notmuch). I can use this to create
nice stats :).

No patch yet, just asking if this is a good idea or not.

Sebastian

Plans for the 0.2 release (this week)

[Jameson Rollins]

On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
> For the upcoming 0.2 release, here are some things that I would like to
> have in place:
> 
>   * Changes to indexing, (addition of body:, folder:, list:, etc.). This
> is stuff that I'll work on.

Also, fwiw, the folder: indexing is probably the new feature that I'm
most eagerly awaiting.  I've got all these ideas for ways to handle sent
mail and drafts if we can get this working.

Presumably others must be annoyed about having to manually "read" and
archive all their sent mail, unless there's some other way that people
having been dealing with this that I'm not aware of.

And as for drafts, they could be easily indexed and managed by the UI if
the folder: search term is available.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100408/dc18aa2f/attachment.pgp>

[PATCH] Fix code extracting the MTA from Received: headers

[Sebastian Spaeth]

On 2010-04-07, Dirk Hohndel wrote:
> 
> The previous code made too many assumptions about the (sadly not
> standardized) format of the Received headers. This version should
> be more robust to deal with different variations.

This code might be useful for some, but I know it is not being useful
for me. I use e.g. dreamhost.com as my mail provider and I never have my
email domain name show up after the Received: by .
See my Received headers for your message below.

On the other hand, it contains "for " stating the
intended email address explicitely. IMHO, we should use this before we
start some hand-wavy guessing.

Also, I have the "X-Original-To: sebastian at sspaeth.de" header. Is that
something that we could make use of before starting to guess?

Sebastian
---
Received: from segal.dreamhost.com (mx1.spunky.mail.dreamhost.com 
[208.97.132.47])
by homiemail-mx12.g.dreamhost.com (Postfix) with ESMTP id 9A6602781BC
for ; Wed,  7 Apr 2010 13:38:48 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
by segal.dreamhost.com (Postfix) with ESMTP id 9CF8A5341BE
for ; Wed,  7 Apr 2010 13:38:48 -0700 (PDT)
Received: from connor.dreamhost.com ([208.97.132.81])
by localhost (segal.dreamhost.com [208.97.132.104]) (amavisd-new, port 
10024)
with ESMTP id S3IlsMcJewY1 for ;
Wed,  7 Apr 2010 13:38:39 -0700 (PDT)
Received: from olra.theworths.org (u15218177.onlinehome-server.com 
[82.165.184.25])
by connor.dreamhost.com (Postfix) with ESMTP id 33B472C9806F
for ; Wed,  7 Apr 2010 13:38:39 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1])
by olra.theworths.org (Postfix) with ESMTP id 1978741733A;
Wed,  7 Apr 2010 13:38:38 -0700 (PDT)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
Received: from olra.theworths.org ([127.0.0.1])
by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id ZbcQaubefNY6; Wed,  7 Apr 2010 13:38:37 -0700 (PDT)
Received: from olra.theworths.org (localhost [127.0.0.1])
by olra.theworths.org (Postfix) with ESMTP id 044574196F4;
Wed,  7 Apr 2010 13:38:35 -0700 (PDT)

Plans for the 0.2 release (this week)

[Jameson Rollins]

Hey, Carl.  I'm very encouraged by the recent patch-apply fest, and the
proposal for more regular releases.  I have some new improvements I'll
try to get in at some point, but in the mean time...

> So keep the patches coming, and the pointers to patches that you want me
> to look at.

I would really like to see the patch in spaetz/issue15-handle-fcc-bcc
applied soon.  This is the lingering issue of bcc'ing the primary email
address in notmuch replies, which I think really needs to be removed.

Let me know if there are any issues.  I'm pretty sure this will apply
cleanly against the head without a rebase.

jamie.

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100408/0acd02d5/attachment-0001.pgp>

Plans for the 0.2 release (this week)

[Sebastian Spaeth]

First of all, thanks for the great work Carl. I have to admit I was
getting nervous about the backlog of patches, but your recent committing
binge (you did say your work patterns are bursty :-)) made me very happy.

That having said, I am glad to meet your expectation: "I expect people
 to remind me of their favorite features that haven't been merged..."
 :-)

>   * Any further changes from the Sebastian's repository. Sebastian, I
> worked through one list I saw recently. Do you have another list to
> propose yet?

One of the things I have seen no comment on yet, is the
"json_quote_str should handle non-ASCII characters" patch in 
id:1267697893-sup-4538 at sam.mediasupervision.de.

Right now, every char > 127 will simply be suppressed, this patch fixes
this for me but I cannot comment on its correctness. I rebased the patch
to current here:
http://github.com/spaetz/notmuch-all-feature/commit/3b5423fbdda5e78414e6f2e441cad5394173e155

Would that be an acceptable fix?

Also, as part of getting the useful backlog down, I propose these smallish
patches that are in my tree in the feature/misc branch (that I all grabbed from 
the mailing list):

* Add count command to manual page, Sandra Snan
  mail id:4ba29058.0f67f10a.2b59.1a73 at mx.google.com
  commit 2f5ea4c76dc9688b2520c8d64ac8617ba8a62f86
  (it was really missing)

* Fix typo in notmuch.h documentation regarding database open modes 
  mail id:1269628757-24857-1-git-send-email-michael at obberon.com
  michaelforney, comit 3262b39f58ab4865c0c0cada69955ad65fccdfd9
  (our beloved notmuch.h is WRONG!!! :-)

* Prevent data loss caused by SIGINT during notmuch new
  mail id:1269937403-8495-1-git-send-email-sojkam1 at fel.cvut.cz
  wentasah, commit dcb7a957027837f187f06d31fdcddb3740b2c20b
  (preventable data loss, has happened to him at least once)

* Do not segfault on empty mime parts, 
  mail id:1267543888-18134-1-git-send-email-madduck at madduck.net
  martin f. krafft, commit bb310c6397a94f6f59b6f93b232bedd833f96090
  (probably a bug in gmime, but we should not crash)

>   * Some library additions (move_to_first for the iterators,
+1!

All the rest souds very good.

Sebastian

[PATCH] notmuch.el: 'F' in search mode takes us to a list of folders.

[David Edmondson]

On Wed, 07 Apr 2010 16:12:36 -0700, Carl Worth  wrote:
> I'm actually currently using super-n (a personal customization I have)
> to get to notmuch-folder view from *any* buffer in emacs.

This is also my current approach, though I don't use it very much (or
'F').

dme.
-- 
David Edmondson, http://dme.org

Plans for the 0.2 release (this week)

[David Edmondson]

On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
>   * The big batch of emacs-client improvements from David E.'s
> repository. David, do you have particular things to recommend here?

It's necessary for me to merge with your latest batch of changes. That
won't happen until next week, when I can set aside a few hours. (The
merge is somewhat painful, as I have modified versions of some of the
recently applied patches in my tree.)

I worry about committing the JSON based Emacs UI and then immediately
producing a release - it would be useful to have more people test it
from git HEAD before dropping it on an unsuspecting public.

dme.
-- 
David Edmondson, http://dme.org

RFC: User-Agent header

[Dirk Hohndel]

On Thu, 08 Apr 2010 10:26:01 +0200, "Sebastian Spaeth"  wrote:
> notmuch is (mostly) not responsible for sending email. However, people
> using the emacs frontend use notmuch to create the reply.
> 
> Am I the only one who is sometimes curious as to what mail agents others
> use? Would it be useful to insert a header to notmuch reply analog to:
> 
> User-Agent: Mutt/1.5.13 (2006-08-11)
> 
> We could reuse the same version string that we use for the release (or
> the git string that was used to build notmuch). I can use this to create
> nice stats :).
> 
> No patch yet, just asking if this is a good idea or not.

I think it's a very good idea. But it should be something that includes
the other components of how you send email...

Like

User-Aget: Emacs 23 Message-mode / notmuch-0.1.1

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center

[PATCH] Fix code extracting the MTA from Received: headers

[Dirk Hohndel]

On Thu, 08 Apr 2010 09:59:14 +0200, "Sebastian Spaeth"  wrote:
> On 2010-04-07, Dirk Hohndel wrote:
> > 
> > The previous code made too many assumptions about the (sadly not
> > standardized) format of the Received headers. This version should
> > be more robust to deal with different variations.
> 
> This code might be useful for some, but I know it is not being useful
> for me. I use e.g. dreamhost.com as my mail provider and I never have my
> email domain name show up after the Received: by .
> See my Received headers for your message below.

That's the funny thing about heuristics - they are always based on the
cases the author has access to. I run my own mail servers and they put
in useful Received lines. Dreamhost doesn't appear to do that - I'm sure
there are many other scenarios that I don't handle, yet.
Please keep them coming.

> On the other hand, it contains "for " stating the
> intended email address explicitely. IMHO, we should use this before we
> start some hand-wavy guessing.
> 
> Also, I have the "X-Original-To: sebastian at sspaeth.de" header. Is that
> something that we could make use of before starting to guess?

It's complicated. Some MTAs put in bogux "for " or "for
UID 1000" into Received headers. I haven't seen any incorrect
"X-Original-To" headers, but wouldn't be surprised to see those be faked
or wrong, either.
Right now my plan is to do something like this:

1) look for my email address in To/Cc
2) look for my email in "for " in Received headers
3) look for my email in X-Original-To
4) look for the domain of my email in Received headers (not just 1st)
5) punt and use default email address

Does that sound sane?

(and thanks for sending the headers - this really helps... can others
for whom the current code or the logic mentioned above wouldn't work
send their headers, too, please?)

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center

Plans for the 0.2 release (this week)

[Dirk Hohndel]

On Thu, 08 Apr 2010 10:03:15 -0400, Jameson Rollins  wrote:
> On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
> > For the upcoming 0.2 release, here are some things that I would like to
> > have in place:
> > 
> >   * Changes to indexing, (addition of body:, folder:, list:, etc.). This
> > is stuff that I'll work on.
> 
> Also, fwiw, the folder: indexing is probably the new feature that I'm
> most eagerly awaiting.  I've got all these ideas for ways to handle sent
> mail and drafts if we can get this working.

Maybe I am missing some context here - what exactly does 'indexing' mean
here?

> Presumably others must be annoyed about having to manually "read" and
> archive all their sent mail, unless there's some other way that people
> having been dealing with this that I'm not aware of.

my import/tagging script deals with +sent on emails that I've sent and
removes -inbox -unread on them. The FCC logic gets them filed into a
Sent folder (right now just one - see previous comment about making this
>From address dependent)

> And as for drafts, they could be easily indexed and managed by the UI if
> the folder: search term is available.

Hmm - haven't even thought about drafts, yet. How would the UI deal with
those? 

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center

Plans for the 0.2 release (this week)

[Dirk Hohndel]

On Thu, 08 Apr 2010 09:52:21 -0400, Jameson Rollins  wrote:
> > So keep the patches coming, and the pointers to patches that you want me
> > to look at.
> 
> I would really like to see the patch in spaetz/issue15-handle-fcc-bcc
> applied soon.  This is the lingering issue of bcc'ing the primary email
> address in notmuch replies, which I think really needs to be removed.

+1

The FCC solution is much more sane. Especially with the From address
based path selection that I proposed (but haven't been able to implement
for lack of Lisp skills)

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center

Plans for the 0.2 release (this week)

[Dirk Hohndel]

On Thu, 08 Apr 2010 09:13:20 +0100, David Edmondson  wrote:
> On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
> >   * The big batch of emacs-client improvements from David E.'s
> > repository. David, do you have particular things to recommend here?
> 
> It's necessary for me to merge with your latest batch of changes. That
> won't happen until next week, when I can set aside a few hours. (The
> merge is somewhat painful, as I have modified versions of some of the
> recently applied patches in my tree.)

I hope that this will be a less frequent thing to happen as Carl is
quicker in picking up patches. I'm very eager to see the emacs client
improvements, though.

> I worry about committing the JSON based Emacs UI and then immediately
> producing a release - it would be useful to have more people test it
> from git HEAD before dropping it on an unsuspecting public.

Not sure how much of an "unsuspecting public" we have at this
point... but in general this is of course valid. The question is "what
are releases". Or what does it mean to "release 0.2". I don't think we
have the notion of a stable branch and a development cycle,
yet. Everything is development (correct me if I'm wrong, Carl)

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center

Plans for the 0.2 release (this week)

[James Westby]

On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth  wrote:
>   * Anything else that people want, (especially things that already
> exist and that you're already using). Support for maildir flags on
> import would be great here. I'm still waiting to see a complete
> solution I think.

id:1268515677-12692-1-git-send-email-jw+debian at jameswestby.net

I've been running with this locally since then with no apparent issues.

It will make anyone who suffers from this problem regularly pretty
happy, as they can return to the world of a thread-based mail client.

Thanks,

James

Re: Plans for the 0.2 release (this week)

[Sebastian Spaeth]

First of all, thanks for the great work Carl. I have to admit I was
getting nervous about the backlog of patches, but your recent committing
binge (you did say your work patterns are bursty :-)) made me very happy.

That having said, I am glad to meet your expectation: I expect people
 to remind me of their favorite features that haven't been merged...
 :-)

   * Any further changes from the Sebastian's repository. Sebastian, I
 worked through one list I saw recently. Do you have another list to
 propose yet?

One of the things I have seen no comment on yet, is the
json_quote_str should handle non-ASCII characters patch in 
id:1267697893-sup-4...@sam.mediasupervision.de.

Right now, every char  127 will simply be suppressed, this patch fixes
this for me but I cannot comment on its correctness. I rebased the patch
to current here:
http://github.com/spaetz/notmuch-all-feature/commit/3b5423fbdda5e78414e6f2e441cad5394173e155

Would that be an acceptable fix?

Also, as part of getting the useful backlog down, I propose these smallish
patches that are in my tree in the feature/misc branch (that I all grabbed from 
the mailing list):

* Add count command to manual page, Sandra Snan
  mail id:4ba29058.0f67f10a.2b59.1...@mx.google.com
  commit 2f5ea4c76dc9688b2520c8d64ac8617ba8a62f86
  (it was really missing)

* Fix typo in notmuch.h documentation regarding database open modes 
  mail id:1269628757-24857-1-git-send-email-mich...@obberon.com
  michaelforney, comit 3262b39f58ab4865c0c0cada69955ad65fccdfd9
  (our beloved notmuch.h is WRONG!!! :-)

* Prevent data loss caused by SIGINT during notmuch new
  mail id:1269937403-8495-1-git-send-email-sojk...@fel.cvut.cz
  wentasah, commit dcb7a957027837f187f06d31fdcddb3740b2c20b
  (preventable data loss, has happened to him at least once)

* Do not segfault on empty mime parts, 
  mail id:1267543888-18134-1-git-send-email-madd...@madduck.net
  martin f. krafft, commit bb310c6397a94f6f59b6f93b232bedd833f96090
  (probably a bug in gmime, but we should not crash)

   * Some library additions (move_to_first for the iterators,
+1!

All the rest souds very good.

Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[David Edmondson]

On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth cwo...@cworth.org wrote:
   * The big batch of emacs-client improvements from David E.'s
 repository. David, do you have particular things to recommend here?

It's necessary for me to merge with your latest batch of changes. That
won't happen until next week, when I can set aside a few hours. (The
merge is somewhat painful, as I have modified versions of some of the
recently applied patches in my tree.)

I worry about committing the JSON based Emacs UI and then immediately
producing a release - it would be useful to have more people test it
from git HEAD before dropping it on an unsuspecting public.

dme.
-- 
David Edmondson, http://dme.org
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] notmuch.el: 'F' in search mode takes us to a list of folders.

[David Edmondson]

On Wed, 07 Apr 2010 16:12:36 -0700, Carl Worth cwo...@cworth.org wrote:
 I'm actually currently using super-n (a personal customization I have)
 to get to notmuch-folder view from *any* buffer in emacs.

This is also my current approach, though I don't use it very much (or
'F').

dme.
-- 
David Edmondson, http://dme.org
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

RFC: User-Agent header

[Sebastian Spaeth]

notmuch is (mostly) not responsible for sending email. However, people
using the emacs frontend use notmuch to create the reply.

Am I the only one who is sometimes curious as to what mail agents others
use? Would it be useful to insert a header to notmuch reply analog to:

User-Agent: Mutt/1.5.13 (2006-08-11)

We could reuse the same version string that we use for the release (or
the git string that was used to build notmuch). I can use this to create
nice stats :).

No patch yet, just asking if this is a good idea or not.

Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[Anthony Towns]

On Thu, Apr 8, 2010 at 17:47, Sebastian Spaeth sebast...@sspaeth.de wrote:
 * Prevent data loss caused by SIGINT during notmuch new
  mail id:1269937403-8495-1-git-send-email-sojk...@fel.cvut.cz
  wentasah, commit dcb7a957027837f187f06d31fdcddb3740b2c20b
  (preventable data loss, has happened to him at least once)

That seems to be a different id to
http://patchwork.notmuchmail.org/patch/445/ but me too on that,
fwiw. (Haven't had data loss, did have confusing behaviour that I
attribute to this, and the logic for the patch seems sound to me)

Cheers,
aj

-- 
Anthony Towns a...@erisian.com.au
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [notmuch] [Sebastian Spaeth] Pull requests

[Michal Sojka]

On Wed, 07 Apr 2010, micah anderson wrote:
 On 2010-03-27, micah anderson wrote:
  On Thu, 25 Mar 2010 22:50:52 -0400, micah anderson mi...@riseup.net wrote:
   On Mon, 01 Mar 2010 15:57:00 +0100, Sebastian Spaeth 
   sebast...@sspaeth.de wrote:
   
From git repository git://github.com/spaetz/notmuch-all-feature.git I
would like to advocate the following branches for quick pulling. Each
contains 1 or 2 patches. They have all been based on todays
cworth/master, so it should be really painless.
   
   Thanks for pulling these all together! All the ones that you propose I
   also use and would really like these to be merged as well.
   
   The only other patch that I find absolutely crucial, that you do not
   include, is the 'Preserve folder information when indexing' patch which,
   although not perfect, does significantly change my life. 
  
  Glad you find it useful. Yes, having the folder information would indeed
  be nice. Is that patch working well for you? (Can you point me to the
  mail ID of the patch you are using? There have been several versions
  around).
 
 The patch works really well for me. I also had difficulty figuring out
 which was the latest.

I think that the patch you sent was not the latest version. The latest
is in id:1265122868-12133-1-git-send-email-sojk...@fel.cvut.cz, but it
is not rebased to the current master.

I do not use this patch right now, since I didn't find time to apply it
on top of my mailstore abstraction patches. What I remember from when I
used it is, that it was really problematic with mail sent by me to
mailing lists. There were two version of the message - one in sent
folder and one in list folder - and the folder tag was not assigned
deterministically.

I think that now I know how to implement this correctly, I only need
some time. Also, the problem is very similar to synchronization of
maildir flags to multiple files, which is what I'm solving right now. So
probably, after I finish maildir flag synchronization I can work on this
patch.

-Michal
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[Michal Sojka]

On Wed, 07 Apr 2010, Carl Worth wrote:
 For the upcoming 0.2 release, here are some things that I would like to
 have in place:
 
   * Anything else that people want, (especially things that already
 exist and that you're already using). Support for maildir flags on
 import would be great here. I'm still waiting to see a complete
 solution I think.

This would be useful: id:m1my03gsmu@watt.gilman.jhu.edu. I use it
occasionally.

-Michal

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/4] Conversion to mailstore abstraction

[Michal Sojka]

The code for detection of new files in the mailstore and their
addition to the database is moved from notmuch-new.c to
lib/mailstore-files.c, where it is called by the abstract mailstore
interface.

The code was only changed to allow the progress reporting functions to
be implemented outside of notmuch library, as can be seen by
git diff HEAD^ -C

Signed-off-by: Michal Sojka sojk...@fel.cvut.cz
---
 lib/Makefile.local |1 +
 notmuch-new.c = lib/mailstore-files.c |  406 
 lib/mailstore.c|5 -
 notmuch-new.c  |  660 +++-
 4 files changed, 141 insertions(+), 931 deletions(-)
 copy notmuch-new.c = lib/mailstore-files.c (61%)

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 6243af1..863df4c 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -32,6 +32,7 @@ extra_cflags += -I$(dir) -fPIC
 libnotmuch_c_srcs =\
$(dir)/libsha1.c\
$(dir)/mailstore.c  \
+   $(dir)/mailstore-files.c\
$(dir)/message-file.c   \
$(dir)/messages.c   \
$(dir)/sha1.c   \
diff --git a/notmuch-new.c b/lib/mailstore-files.c
similarity index 61%
copy from notmuch-new.c
copy to lib/mailstore-files.c
index 2d0ba6c..8b5e1d5 100644
--- a/notmuch-new.c
+++ b/lib/mailstore-files.c
@@ -1,4 +1,5 @@
-/* notmuch - Not much of an email program, (just index and search)
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
  *
  * Copyright © 2009 Carl Worth
  *
@@ -15,12 +16,14 @@
  * You should have received a copy of the GNU General Public License
  * along with this program.  If not, see http://www.gnu.org/licenses/ .
  *
- * Author: Carl Worth cwo...@cworth.org
+ * Authors: Carl Worth cwo...@cworth.org
+ * Michal Sojka sojk...@fel.cvut.cz
  */
 
-#include notmuch-client.h
-
-#include unistd.h
+#define _GNU_SOURCE/* For asprintf() */
+#include notmuch.h
+#include mailstore-private.h
+#include dirent.h
 
 typedef struct _filename_node {
 char *filename;
@@ -32,38 +35,10 @@ typedef struct _filename_list {
 _filename_node_t **tail;
 } _filename_list_t;
 
-typedef struct {
-int output_is_a_tty;
-int verbose;
-
-int total_files;
-int processed_files;
-int added_messages;
-struct timeval tv_start;
-
+typedef struct _indexing_context_priv {
 _filename_list_t *removed_files;
 _filename_list_t *removed_directories;
-} add_files_state_t;
-
-static volatile sig_atomic_t do_add_files_print_progress = 0;
-
-static void
-handle_sigalrm (unused (int signal))
-{
-do_add_files_print_progress = 1;
-}
-
-static volatile sig_atomic_t interrupted;
-
-static void
-handle_sigint (unused (int sig))
-{
-ssize_t ignored;
-static char msg[] = Stopping... \n;
-
-ignored = write(2, msg, sizeof(msg)-1);
-interrupted = 1;
-}
+} _indexing_context_priv_t;
 
 static _filename_list_t *
 _filename_list_create (const void *ctx)
@@ -100,34 +75,6 @@ tag_inbox_and_unread (notmuch_message_t *message)
 notmuch_message_add_tag (message, unread);
 }
 
-static void
-add_files_print_progress (add_files_state_t *state)
-{
-struct timeval tv_now;
-double elapsed_overall, rate_overall;
-
-gettimeofday (tv_now, NULL);
-
-elapsed_overall = notmuch_time_elapsed (state-tv_start, tv_now);
-rate_overall = (state-processed_files) / elapsed_overall;
-
-printf (Processed %d, state-processed_files);
-
-if (state-total_files) {
-   double time_remaining;
-
-   time_remaining = ((state-total_files - state-processed_files) /
- rate_overall);
-   printf ( of %d files (, state-total_files);
-   notmuch_time_print_formatted_seconds (time_remaining);
-   printf ( remaining).  \r);
-} else {
-   printf ( files (%d files/sec.)\r, (int) rate_overall);
-}
-
-fflush (stdout);
-}
-
 static int
 dirent_sort_inode (const struct dirent **a, const struct dirent **b)
 {
@@ -169,6 +116,7 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
 return 0;
 }
 
+
 /* Examine 'path' recursively as follows:
  *
  *   o Ask the filesystem for the mtime of 'path' (fs_mtime)
@@ -205,9 +153,9 @@ _entries_resemble_maildir (struct dirent **entries, int 
count)
  *   o Tell the database to update its time of 'path' to 'fs_mtime'
  */
 static notmuch_status_t
-add_files_recursive (notmuch_database_t *notmuch,
+add_files_recursive (notmuch_mailstore_t *mailstore,
 const char *path,
-add_files_state_t *state)
+notmuch_indexing_context_t *state)
 {
 DIR *dir = NULL;
 struct dirent *entry = NULL;
@@ -222,6 +170,8 @@ add_files_recursive (notmuch_database_t *notmuch,
 notmuch_filenames_t *db_subdirs = NULL;
 struct stat st;
 notmuch_bool_t is_maildir, new_directory;
+

Re: Plans for the 0.2 release (this week)

[Dirk Hohndel]

On Thu, 08 Apr 2010 09:13:20 +0100, David Edmondson d...@dme.org wrote:
 On Wed, 07 Apr 2010 15:12:44 -0700, Carl Worth cwo...@cworth.org wrote:
* The big batch of emacs-client improvements from David E.'s
  repository. David, do you have particular things to recommend here?
 
 It's necessary for me to merge with your latest batch of changes. That
 won't happen until next week, when I can set aside a few hours. (The
 merge is somewhat painful, as I have modified versions of some of the
 recently applied patches in my tree.)

I hope that this will be a less frequent thing to happen as Carl is
quicker in picking up patches. I'm very eager to see the emacs client
improvements, though.
 
 I worry about committing the JSON based Emacs UI and then immediately
 producing a release - it would be useful to have more people test it
 from git HEAD before dropping it on an unsuspecting public.

Not sure how much of an unsuspecting public we have at this
point... but in general this is of course valid. The question is what
are releases. Or what does it mean to release 0.2. I don't think we
have the notion of a stable branch and a development cycle,
yet. Everything is development (correct me if I'm wrong, Carl)

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[Dirk Hohndel]

On Thu, 08 Apr 2010 09:52:21 -0400, Jameson Rollins 
jroll...@finestructure.net wrote:
  So keep the patches coming, and the pointers to patches that you want me
  to look at.
 
 I would really like to see the patch in spaetz/issue15-handle-fcc-bcc
 applied soon.  This is the lingering issue of bcc'ing the primary email
 address in notmuch replies, which I think really needs to be removed.

+1

The FCC solution is much more sane. Especially with the From address
based path selection that I proposed (but haven't been able to implement
for lack of Lisp skills)

/D

-- 
Dirk Hohndel
Intel Open Source Technology Center
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] apply all the other maildir flag-tag conversions

[Mike Kelly]

---
 notmuch-new.c |   10 ++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/notmuch-new.c b/notmuch-new.c
index 511347d..dc33d69 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -99,6 +99,16 @@ tag_inbox_and_unread (notmuch_message_t *message)
 notmuch_message_add_tag (message, inbox);
 if (! notmuch_message_md_flag(message, 'S'))
 notmuch_message_add_tag (message, unread);
+if (notmuch_message_md_flag(message, 'T'))
+notmuch_message_add_tag (message, deleted);
+if (notmuch_message_md_flag(message, 'D'))
+notmuch_message_add_tag (message, draft);
+if (notmuch_message_md_flag(message, 'F'))
+notmuch_message_add_tag (message, flagged);
+if (notmuch_message_md_flag(message, 'P'))
+notmuch_message_add_tag (message, passed);
+if (notmuch_message_md_flag(message, 'R'))
+notmuch_message_add_tag (message, replied);
 }
 
 static void
-- 
1.7.0.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/3] document new `notmuch new` behavior

[Mike Kelly]

---
 notmuch.1 |8 +---
 notmuch.c |7 ---
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/notmuch.1 b/notmuch.1
index 0e6a2ed..8b4021e 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -93,9 +93,11 @@ The
 .B new
 command scans all sub-directories of the database, performing
 full-text indexing on new messages that are found. Each new message
-will automatically be tagged with both the
-.BR inbox  and  unread
-tags.
+will automatically be tagged with the
+.BR inbox
+tag, and, unless it was already seen by another client, the
+.BR unread
+tag.
 
 You should run
 .B notmuch new
diff --git a/notmuch.c b/notmuch.c
index f5669fc..9002e7d 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -125,9 +125,10 @@ command_t commands[] = {
 { new, notmuch_new_command,
   [--verbose],
   Find and import new messages to the notmuch database.,
-  \tScans all sub-directories of the mail directory, performing\n
-  \tfull-text indexing on new messages that are found. Each new\n
-  \tmessage will be tagged as both \inbox\ and \unread\.\n
+  \t\tScans all sub-directories of the mail directory, performing\n
+  \t\tfull-text indexing on new messages that are found. Each new\n
+  \t\tmessage will be tagged as \inbox\ and, unless it is\n
+  \t\tmarked as \seen\, \unread\.\n
   \n
   \tYou should run \notmuch new\ once after first running\n
   \t\notmuch setup\ to create the initial database. The first\n
-- 
1.7.0.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[Carl Worth]

On Thu, 08 Apr 2010 07:58:47 -0700, Dirk Hohndel hohn...@infradead.org wrote:
 On Thu, 08 Apr 2010 10:03:15 -0400, Jameson Rollins 
 jroll...@finestructure.net wrote:
  Also, fwiw, the folder: indexing is probably the new feature that I'm
  most eagerly awaiting.  I've got all these ideas for ways to handle sent
  mail and drafts if we can get this working.
 
 Maybe I am missing some context here - what exactly does 'indexing' mean
 here?

Indexing here means shoving information into the database to make it
searchable.

I was a bit vague in my description of the features I want to enable
here. So let me start making it a bit more clear. I'd like to add
support for the following:

  1. A folder: prefix in the search syntax that will match components
 of the directory in which mail files are stored. So, if you've got
 mail filed into directories, (from using a folder-based email
 program, or from doing an imap sync from gmail that turns tags into
 directories), then you could search for things like folder:spam,
 folder:important, folder:arbitrary/subset/of/some/path, etc.

  2. A list: prefix to match content from headers that identify
 mailing list. My perception is that there are likely a handful of
 different headers that have been used, and they should all be
 indexed so that list: will search any of them.

  3. Some prefix that can be used to match typical headers added by
 spam-filtering software, (maybe this would be a general
 prefix---see below).

I think the above are probably the three I can think of that pretty much
everybody has asked for, so that should be indexed by default.

Beyond that, I'd like to be able to provide support for arbitrary
headers in the email. I had envisioned allowing the user to configure
specific headers to index. Alternately, I had imagined having a
blacklist of uninteresting headers and indexing everything
else. (Though, now Dirk is very interested in Received and its perhaps
provides the most content of any of the headers I originally thought
would be uninteresting.)

One experiment I should do is to measure how much my database would grow
if I were to index all of the header content. I don't really know what
kind of a percentage we're talking about.

Finally, we'd need a syntax for searching all of these headers. Rather
than trying to map each header name to a custom prefix, I think what
might be best would be a general thing that could look like:

header:X-Spam-Flag: YES

We don't currently have the ability to tie a search to the beginning of
a line, but it occurs to me that we could do something fancy like index
each header with a specific term to indicate line beginnings. Then, with
a custom query parser we could use the common symbol of '^' to map to
that. That would enable something like:

header:^X-Spam-Flag: YES

That's a lot of work though, (and perhaps not as important as it's
probably uncommon for a header name to appear within the value of some
other header).

 Hmm - haven't even thought about drafts, yet. How would the UI deal with
 those? 

I would imagine another folder alongside the others that would list
all drafts, and selecting any such message would bring up the message in
a mode to edit it.

Sup also displayed draft messages in their proper location in the
threads containing the message being replied to, (and highlighted such
threads in the search view).

All of that sounds quite easy to do by simply saving the draft within
the mailstore and giving it a draft tag.

Finally, many mail interfaces prompt from the compose new message
command whether an existing draft should be continued. That's useful to
help the user avoid forgetting to complete and send a draft that was
forgotten.

-Carl




pgpLITQWyc41K.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[Anthony Towns]

On Fri, Apr 9, 2010 at 00:03, Jameson Rollins
jroll...@finestructure.net wrote:
 Presumably others must be annoyed about having to manually read and
 archive all their sent mail, unless there's some other way that people
 having been dealing with this that I'm not aware of.

I haven't switched over to notmuch properly yet, but in my initial
import of my mail I made a couple of patches to notmuch new that let
me get the tags right first. One was to make:

notmuch new --initial-tags=list,unread

set the initial tags for newly found messages to list and unread
instead of the default inbox and unread.

The other was to make:

find Mail/.Saved/ -type f | notmuch new --initial-tags=archived --file-list

only process the mail files listed on stdin (via find), and give them
the explicit tags I specify. That way I could import all my existing
archived mail without it appearing in my inbox or as unread.

Unfortunately the second one ended up complicated and a bit slow (I
think because I'm doing a talloc() for every line on stdin, and
talloc_free() by context every time the base directory changes; that
sort of behaviour was necessary in order to do duplicate checking in a
sane way)

But anyway, that would let you do:

find Mail/.Drafts/ -type f | notmuch new --initial-tags=draft --file-list
notmuch new

to get drafts correctly tagged.

(I don't have the patches handy at the moment; but can certainly dig
them up if there's interest)

Cheers,
aj

-- 
Anthony Towns a...@erisian.com.au
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Plans for the 0.2 release (this week)

[Michal Sojka]

On Wed, 07 Apr 2010, Carl Worth wrote:
   * Anything else that people want, (especially things that already
 exist and that you're already using). Support for maildir flags on
 import would be great here. I'm still waiting to see a complete
 solution I think.
 
 So keep the patches coming, and the pointers to patches that you want me
 to look at.

Decode headers in reply
(id:1267602656-24940-1-git-send-email-sojk...@fel.cvut.cz) is another
patch, which is quite essential to me. Am I the only one here, who needs
to reply to messages with non-ASCII characters?

-Michal
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch