[RFC PATCH 00/14] modular mail stores based on URIs

Hi guys, Sorry for dropping off the mailing list after I sent my last patch series (http://notmuchmail.org/pipermail/notmuch/2012/009470.html). I haven't had the time or a stable enough email address to really follow notmuch development :) I signed onto #notmuch a week or two ago and asked

[RFC PATCH 01/14] All access to mail files goes through the mailstore module

not be one file. Signed-off-by: Ethan Glasser-Camp --- lib/Makefile.local|1 + lib/database.cc |2 +- lib/index.cc |2 +- lib/mailstore.c | 34 lib/message-file.c|6 ++--- lib/notmuch-private.h |3 +++ lib/notmuch.h

[RFC PATCH 02/14] Introduce uriparser

Seeing as there is no glib-standard way to parse URIs, an external library is needed. This commit introduces another program in compat/ and a stanza in ./configure to test if uriparser is there. Signed-off-by: Ethan Glasser-Camp --- Makefile.local |2 +- compat/have_uriparser.c

[RFC PATCH 03/14] mailstore can read from maildir: URLs

No code uses this yet. Signed-off-by: Ethan Glasser-Camp --- lib/mailstore.c | 37 - 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/lib/mailstore.c b/lib/mailstore.c index 48acd47..ae02c12 100644 --- a/lib/mailstore.c +++ b/lib/mailstore.c

[RFC PATCH 04/14] Not all filenames need to be converted to absolute paths

_notmuch_message_ensure_filename_list converts "relative" paths, such as those stored in Xapian until now, to "absolute" paths. However, URLs are already absolute, and prepending the database path will just confuse matters. Signed-off-by: Ethan Glasser-Camp --- l

[RFC PATCH 05/14] new: use new URL-based filenames for messages

This commit breaks a bunch of tests; fixes follow. Signed-off-by: Ethan Glasser-Camp --- notmuch-new.c | 27 +++ 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/notmuch-new.c b/notmuch-new.c index 938ae29..1f11b2c 100644 --- a/notmuch-new.c +++ b/notmuch

[RFC PATCH 06/14] maildir URIs can be used in tags_to_maildir_flags

A better fix would probably be based on scheme. Signed-off-by: Ethan Glasser-Camp --- lib/message.cc | 51 ++- 1 file changed, 46 insertions(+), 5 deletions(-) diff --git a/lib/message.cc b/lib/message.cc index c9857f5..8ecec71 100644 --- a/lib

[RFC PATCH 07/14] Update tests that need to see filenames to use URIs

This fixes all tests except atomicity, which should be next. Signed-off-by: Ethan Glasser-Camp --- test/emacs |2 +- test/json|4 ++-- test/maildir-sync|7 --- test/multipart |4 ++-- test/new

[RFC PATCH 08/14] Don't cache corpus.mail

corpus.mail has already been processed by notmuch-new, so it seems like a good target to cache, but since filenames are no longer being stored relative to the database, it isn't. Recopy on each test, or else filenames from other tests will show up. Signed-off-by: Ethan Glasser-Camp --- test

[RFC PATCH 09/14] Fix atomicity test to work without relocatable mailstores

Instead of assuming that the mailstore doesn't store its absolute filenames, we use a symlink that can change back and forth. As long as filenames contain this symlink, they can work in either the real database, or the current snapshot. Signed-off-by: Ethan Glasser-Camp --- test/atomicity

[RFC PATCH 10/14] new: add "scan" option

This is just a quick hack to get started on adding an mbox backend. The fact that the default maildir is scanned "automagically" is a little weird, but it doesn't do any harm unless you decide to put mail there that you really don't want indexed. Signed-off-by: Ethan Glasser-Camp --

[RFC PATCH 11/14] notmuch-new: pull out useful bits of add_files_recursive

_report_before_adding_file and _report_added_file, as well as _add_message, which actually does the message adding. Signed-off-by: Ethan Glasser-Camp --- notmuch-new.c | 192 +++-- 1 file changed, 119 insertions(+), 73 deletions(-) diff --git

[RFC PATCH 12/14] mailstore: support for mbox:// URIs

Signed-off-by: Ethan Glasser-Camp --- lib/mailstore.c | 85 +++ 1 file changed, 85 insertions(+) diff --git a/lib/mailstore.c b/lib/mailstore.c index ae02c12..e8d9bc1 100644 --- a/lib/mailstore.c +++ b/lib/mailstore.c @@ -19,6 +19,7

[RFC PATCH 13/14] Tests for mbox support

These need to be improved, rather than hard-coding byte offsets. Signed-off-by: Ethan Glasser-Camp --- test/mbox | 59 + test/notmuch-test |1 + 2 files changed, 60 insertions(+) create mode 100755 test/mbox diff --git a/test

[RFC PATCH 14/14] new: Add scan support for mbox:// URIs

A lot of code is duplicated from maildir, I don't think I handled all those errors correctly, and I didn't report any progress. Signed-off-by: Ethan Glasser-Camp --- notmuch-new.c | 299 +++-- 1 file changed, 289 insertions(+), 10 deletions

[PATCH v2 0/8] URI-based modular mail stores

Hi guys, Based on the feedback from the last patch series (thanks a lot to Mark Walters and Jani Nikula for taking the time to look at it!) I revised this patch series to not change the handling of maildir filenames at all. This meant not having to give up relocatability, and also simplified the

[PATCH v2 1/8] All access to mail files goes through the mailstore module

notmuch_sha1_of_file is only used on messages, we convert it to using notmuch_mailstore_open and notmuch_mailstore_close, and rename it notmuch_sha1_of_message. While we are there, we also replace a numeric constant with its symbolic name BLOCK_SIZE. Signed-off-by: Ethan Glasser-Camp --- This patch

[PATCH v2 2/8] Introduce uriparser

Seeing as there is no glib-standard way to parse URIs, an external library is needed. This commit introduces another program in compat/ and a stanza in ./configure to test if uriparser is there. Signed-off-by: Ethan Glasser-Camp --- Makefile.local |2 +- compat/have_uriparser.c

[PATCH v2 3/8] Not all filenames need to be converted to absolute paths

_notmuch_message_ensure_filename_list converts "relative" paths, such as those stored in Xapian until now, to "absolute" paths. However, URLs are already absolute, and prepending the database path will just confuse matters. Signed-off-by: Ethan Glasser-Camp --- l

[PATCH v2 4/8] new: add "scan" option

The new.scan option is a list of URLs that can be "scanned". The fact that the database maildir is scanned "automagically" is a little weird, but it doesn't do any harm unless you decide to put mail there that you really don't want indexed. Signed-off-by: Ethan Glasser-Ca

[PATCH v2 5/8] notmuch-new: pull out useful bits of add_files_recursive

This patch pulls some bits out of add_files_recursive which will be useful for other backends: two reporting functions _report_before_adding_file and _report_added_file, as well as _add_message, which actually does the message adding. No functional changes. Signed-off-by: Ethan Glasser-Camp

[PATCH v2 6/8] mailstore: support for mbox:// URIs

No code uses this yet. Signed-off-by: Ethan Glasser-Camp --- lib/mailstore.c | 117 ++- 1 file changed, 116 insertions(+), 1 deletion(-) diff --git a/lib/mailstore.c b/lib/mailstore.c index 48acd47..a29d734 100644 --- a/lib/mailstore.c +++ b

[PATCH v2 7/8] Tests for mbox support

These are simple tests of one single mbox. Signed-off-by: Ethan Glasser-Camp --- The test uses bash arrays, which have a slightly odd syntax for appending. test/mbox | 63 + test/notmuch-test |1 + 2 files changed, 64 insertions

[PATCH v2 8/8] new: Add scan support for mbox:// URIs

This fixes the broken tests introduced by the last commit. Signed-off-by: Ethan Glasser-Camp --- More text was added to clarify how mbox scanning works. notmuch-config.c |4 + notmuch-new.c| 304 +- 2 files changed, 303 insertions

<    1   2   3