[PATCH 1/4] Mailstore abstraction interface
On Tue, 13 Apr 2010, Carl Worth wrote:
> On Thu, 8 Apr 2010 16:42:43 +0200, Michal Sojka
> wrote:
> > The goal of mailstore abstraction is to allow notmuch to store tags
> > together with email messages. The abstract interface is needed because
> > people want to use different ways of storing their emails. Currently,
> > there exists implementation for two types of mailstore - plain files
> > and maildir. It is expected that additional git-based mailstore will
> > be developed later.
>
> I don't agree with the approach being taken here.
>
> I don't think that the expectation of future need is a good basis for
> adding a level of abstraction now. This gives us current costs without
> benefit.
Thanks for the review, Carl. Since I'm interested in further development
of mailstore abstraction until it is really useful, I'd like to make the
patch series as small as possible to reduce maintenance burden. I think
I can extract some "real features" such as cat subcommand from my
patches and send them separately, perhaps in 0.3 merge window.
> Meanwhile, the two different mail stores that you are currently support
> ("plain" and "maildir") aren't really different types. We do already
> have code within notmuch to detect whether any particular directory is a
> maildir, (with the heuristic of looking for child directories named
> "cur", "new", and "tmp"). So I think we can and should support both
> maildir and non-maildir mail storage just fine.
>
> The question of "once you have maildir storage, should you synchronize
> that tags" is quite separate. The answer there might be "yes", "no", or
> "let the user decide". But if it is configurable, then the configuration
> should be something like
>
> [maildir]
> synchronize_flags=yes
>
> rather than
>
> [mailstore]
> type=maildir
Yes, these two mail stores share a lot of code and there is only a small
difference between them. I agree that it is cleaner for users to see
them as a single mailstore type and use configuration options to change
the behavior.
-Michal
[PATCH 1/4] Mailstore abstraction interface
On Tue, 13 Apr 2010 10:53:12 -0700, Carl Worth wrote: > This series is looking like one of the most complete approaches to > maildir-flag synchronization, (and I like some of the motivation that > leads to "notmuch cat"). But I think the mailstore abstraction is > largely a distraction from the real features here. For my case (of wanting to have backup of my mailstore complete in reasonable time, preferably using less disk space) of wanting mail in git packs, 'notmuch cat' being used everywhere removes a lot of the issues of doing this. (pluggin in an alternative to readdir is fairly simple... but the emacs UI needs to read from it too :) -- Stewart Smith
[PATCH 1/4] Mailstore abstraction interface
On Thu, 8 Apr 2010 16:42:43 +0200, Michal Sojka wrote:
> The goal of mailstore abstraction is to allow notmuch to store tags
> together with email messages. The abstract interface is needed because
> people want to use different ways of storing their emails. Currently,
> there exists implementation for two types of mailstore - plain files
> and maildir. It is expected that additional git-based mailstore will
> be developed later.
I don't agree with the approach being taken here.
I don't think that the expectation of future need is a good basis for
adding a level of abstraction now. This gives us current costs without
benefit.
Meanwhile, the two different mail stores that you are currently support
("plain" and "maildir") aren't really different types. We do already
have code within notmuch to detect whether any particular directory is a
maildir, (with the heuristic of looking for child directories named
"cur", "new", and "tmp"). So I think we can and should support both
maildir and non-maildir mail storage just fine.
The question of "once you have maildir storage, should you synchronize
that tags" is quite separate. The answer there might be "yes", "no", or
"let the user decide". But if it is configurable, then the configuration
should be something like
[maildir]
synchronize_flags=yes
rather than
[mailstore]
type=maildir
This series is looking like one of the most complete approaches to
maildir-flag synchronization, (and I like some of the motivation that
leads to "notmuch cat"). But I think the mailstore abstraction is
largely a distraction from the real features here.
-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
Re: [PATCH 1/4] Mailstore abstraction interface
On Tue, 13 Apr 2010 10:53:12 -0700, Carl Worth cwo...@cworth.org wrote: This series is looking like one of the most complete approaches to maildir-flag synchronization, (and I like some of the motivation that leads to notmuch cat). But I think the mailstore abstraction is largely a distraction from the real features here. For my case (of wanting to have backup of my mailstore complete in reasonable time, preferably using less disk space) of wanting mail in git packs, 'notmuch cat' being used everywhere removes a lot of the issues of doing this. (pluggin in an alternative to readdir is fairly simple... but the emacs UI needs to read from it too :) -- Stewart Smith ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 1/4] Mailstore abstraction interface
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. Currently,
there exists implementation for two types of mailstore - plain files
and maildir. It is expected that additional git-based mailstore will
be developed later.
This patch contains only the interface changes. No functionality is
added, removed or changed.
A new configuration group [mailstore] is defined. Currently, there is
only one key called 'type' whose value determines the used mailstore.
The default value of this option (files) is the plain-file mailstore
currently implemented in notmuch.
This patch changes libnotmuch API (and ABI). The functions
notmuch_database_create() and notmuch_database_open() need additional
parameter to identify the type of mailstore, which is used to access
the messages.
If we want to preserve the API, it would be necessary to encode the
mailstore type into the 'path' parameter. For example the value
# (or ://?) would mean use the mailstore of
given 'type' located at 'path'. If we cannot find the type we would
assuse the default mailstore.
Signed-off-by: Michal Sojka
---
lib/Makefile.local |1 +
lib/database-private.h |1 +
lib/database.cc | 15 ++--
lib/mailstore-private.h | 59
lib/mailstore.c | 85 +++
lib/message.cc | 13 +++
lib/notmuch.h | 82
notmuch-client.h|7
notmuch-config.c| 34 +++
notmuch-count.c |3 +-
notmuch-dump.c |3 +-
notmuch-new.c |6 ++-
notmuch-reply.c |3 +-
notmuch-restore.c |3 +-
notmuch-search-tags.c |3 +-
notmuch-search.c|3 +-
notmuch-show.c |6 ++-
notmuch-tag.c |3 +-
18 files changed, 307 insertions(+), 23 deletions(-)
create mode 100644 lib/mailstore-private.h
create mode 100644 lib/mailstore.c
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 0e3a4d1..6243af1 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -31,6 +31,7 @@ extra_cflags += -I$(dir) -fPIC
libnotmuch_c_srcs =\
$(dir)/libsha1.c\
+ $(dir)/mailstore.c \
$(dir)/message-file.c \
$(dir)/messages.c \
$(dir)/sha1.c \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+notmuch_mailstore_t *mailstore;
};
/* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
*/
#include "database-private.h"
+#include "mailstore-private.h"
#include
@@ -438,7 +439,7 @@ parse_references (void *ctx,
}
notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
}
notmuch = notmuch_database_open (path,
-NOTMUCH_DATABASE_MODE_READ_WRITE);
+NOTMUCH_DATABASE_MODE_READ_WRITE,
+mailstore);
notmuch_database_upgrade (notmuch, NULL, NULL);
DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode)
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = _PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+ notmuch->mailstore = mailstore;
+ mailstore->notmuch = notmuch;
} catch (const Xapian::Error ) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
DONE:
if (message) {
- if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+ if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy
[notmuch] [PATCH 1/4] Mailstore abstraction interface
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. This
patchseries implements two types of mailstore - plain files and
maildir. It is expected that additional git-based mailstore will
follow.
This patch contains only the interface changes. No functionality is
added, removed or changed.
A new configuration group [mailstore] is defined. Currently, there is
only one key 'type' whose value determines the used mailstore. The
default value of this option is the plain-file mailstore currently
implemented in notmuch.
Signed-off-by: Michal Sojka
---
lib/Makefile.local |2 +
lib/database-private.h |1 +
lib/database.cc | 15 ++--
lib/mailstore-files.c | 43
lib/mailstore-private.h | 59
lib/mailstore.c | 77 ++
lib/message.cc | 13 +++
lib/notmuch.h | 85 --
notmuch-client.h|7
notmuch-config.c| 34 +++
notmuch-count.c |3 +-
notmuch-dump.c |3 +-
notmuch-new.c |6 ++-
notmuch-reply.c |3 +-
notmuch-restore.c |3 +-
notmuch-search-tags.c |3 +-
notmuch-search.c|3 +-
notmuch-show.c |3 +-
notmuch-tag.c |3 +-
19 files changed, 348 insertions(+), 18 deletions(-)
create mode 100644 lib/mailstore-files.c
create mode 100644 lib/mailstore-private.h
create mode 100644 lib/mailstore.c
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 495b27e..fc8883d 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -3,6 +3,8 @@ extra_cflags += -I$(dir)
libnotmuch_c_srcs =\
$(dir)/libsha1.c\
+ $(dir)/mailstore.c \
+ $(dir)/mailstore-files.c\
$(dir)/message-file.c \
$(dir)/messages.c \
$(dir)/sha1.c \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+notmuch_mailstore_t *mailstore;
};
/* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
*/
#include "database-private.h"
+#include "mailstore-private.h"
#include
@@ -438,7 +439,7 @@ parse_references (void *ctx,
}
notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
}
notmuch = notmuch_database_open (path,
-NOTMUCH_DATABASE_MODE_READ_WRITE);
+NOTMUCH_DATABASE_MODE_READ_WRITE,
+mailstore);
notmuch_database_upgrade (notmuch, NULL, NULL);
DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode)
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = _PREFIX[i];
notmuch->query_parser->add_prefix (prefix->name, prefix->prefix);
}
+
+ notmuch->mailstore = mailstore;
+ mailstore->notmuch = notmuch;
} catch (const Xapian::Error ) {
fprintf (stderr, "A Xapian exception occurred opening database: %s\n",
error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
DONE:
if (message) {
- if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
+ if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
new file mode 100644
index 000..92d7f5d
--- /dev/null
+++ b/lib/mailstore-files.c
@@ -0,0 +1,43 @@
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
+ *
+ * Copyright ?? 2009 Carl Worth
+ *
+ * This program is free software: you can redistribute it and/or
[notmuch] [PATCH 1/4] Mailstore abstraction interface
The goal of mailstore abstraction is to allow notmuch to store tags
together with email messages. The abstract interface is needed because
people want to use different ways of storing their emails. This
patchseries implements two types of mailstore - plain files and
maildir. It is expected that additional git-based mailstore will
follow.
This patch contains only the interface changes. No functionality is
added, removed or changed.
A new configuration group [mailstore] is defined. Currently, there is
only one key 'type' whose value determines the used mailstore. The
default value of this option is the plain-file mailstore currently
implemented in notmuch.
Signed-off-by: Michal Sojka sojk...@fel.cvut.cz
---
lib/Makefile.local |2 +
lib/database-private.h |1 +
lib/database.cc | 15 ++--
lib/mailstore-files.c | 43
lib/mailstore-private.h | 59
lib/mailstore.c | 77 ++
lib/message.cc | 13 +++
lib/notmuch.h | 85 --
notmuch-client.h|7
notmuch-config.c| 34 +++
notmuch-count.c |3 +-
notmuch-dump.c |3 +-
notmuch-new.c |6 ++-
notmuch-reply.c |3 +-
notmuch-restore.c |3 +-
notmuch-search-tags.c |3 +-
notmuch-search.c|3 +-
notmuch-show.c |3 +-
notmuch-tag.c |3 +-
19 files changed, 348 insertions(+), 18 deletions(-)
create mode 100644 lib/mailstore-files.c
create mode 100644 lib/mailstore-private.h
create mode 100644 lib/mailstore.c
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 495b27e..fc8883d 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -3,6 +3,8 @@ extra_cflags += -I$(dir)
libnotmuch_c_srcs =\
$(dir)/libsha1.c\
+ $(dir)/mailstore.c \
+ $(dir)/mailstore-files.c\
$(dir)/message-file.c \
$(dir)/messages.c \
$(dir)/sha1.c \
diff --git a/lib/database-private.h b/lib/database-private.h
index 41918d7..4499b1a 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -49,6 +49,7 @@ struct _notmuch_database {
Xapian::TermGenerator *term_gen;
Xapian::ValueRangeProcessor *value_range_processor;
+notmuch_mailstore_t *mailstore;
};
/* Convert tags from Xapian internal format to notmuch format.
diff --git a/lib/database.cc b/lib/database.cc
index c91e97c..93c8d0f 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,6 +19,7 @@
*/
#include database-private.h
+#include mailstore-private.h
#include iostream
@@ -438,7 +439,7 @@ parse_references (void *ctx,
}
notmuch_database_t *
-notmuch_database_create (const char *path)
+notmuch_database_create (const char *path, notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL;
@@ -474,7 +475,8 @@ notmuch_database_create (const char *path)
}
notmuch = notmuch_database_open (path,
-NOTMUCH_DATABASE_MODE_READ_WRITE);
+NOTMUCH_DATABASE_MODE_READ_WRITE,
+mailstore);
notmuch_database_upgrade (notmuch, NULL, NULL);
DONE:
@@ -497,7 +499,8 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
notmuch_database_t *
notmuch_database_open (const char *path,
- notmuch_database_mode_t mode)
+ notmuch_database_mode_t mode,
+ notmuch_mailstore_t *mailstore)
{
notmuch_database_t *notmuch = NULL;
char *notmuch_path = NULL, *xapian_path = NULL;
@@ -605,6 +608,9 @@ notmuch_database_open (const char *path,
prefix_t *prefix = PROBABILISTIC_PREFIX[i];
notmuch-query_parser-add_prefix (prefix-name, prefix-prefix);
}
+
+ notmuch-mailstore = mailstore;
+ mailstore-notmuch = notmuch;
} catch (const Xapian::Error error) {
fprintf (stderr, A Xapian exception occurred opening database: %s\n,
error.get_msg().c_str());
@@ -1493,7 +1499,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
DONE:
if (message) {
- if (ret == NOTMUCH_STATUS_SUCCESS message_ret)
+ if ((ret == NOTMUCH_STATUS_SUCCESS ||
+ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) message_ret)
*message_ret = message;
else
notmuch_message_destroy (message);
diff --git a/lib/mailstore-files.c b/lib/mailstore-files.c
new file mode 100644
index 000..92d7f5d
--- /dev/null
+++ b/lib/mailstore-files.c
@@ -0,0 +1,43 @@
+/* mailstore-files.c - Original notmuch mail store - a collection of
+ * plain-text email messages (one message per file).
+ *
+ * Copyright © 2009 Carl Worth
+ *
+ * This program is free
