[notmuch] [PATCH] Make the date parser nicer
Currently we have to enter mail dates as timestamps. This approach does 2
things: 1) it requires the prefix 'date:'
2) it allows dates to be specified in some formats. So a notmuch show
date:2005..2006-05-12 will find all mails from 2005-01-01 until 2006-05-12.
The code is probably not in a proper location yet and needs to be shoved around
by someone more knowledgable than me.
My C++ skills are somewhat,... lacking...
Possible time formats: -MM-DD,-MM (in that month) , (in that year)
MM-DD (month-day in current year), DD (day in current month)
Signed-off-by: Sebastian Spaeth
---
lib/database.cc | 90 ++-
1 files changed, 89 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 5b12320..9c2842d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -494,6 +494,94 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
return NOTMUCH_STATUS_SUCCESS;
}
+struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor {
+MaildateValueRangeProcessor() {}
+
+ time_t
+ parsedate(std::string &str, bool early) {
+ // Parse the begin date to time_t
+ // possible time formats:
+ // -MM-DD (size 10)
+ // -MM(size 7)
+ // (size 4)
+ // MM-DD (size 5)
+ // DD (size 2)
+// begin of time unit when 'early', end of when not
+ struct tm *timeinfo;
+ time_t timet;
+ //const char * startptr;
+ int year = 0, month = 0, day = 0;
+
+ if (str.size() == 2) {
+ // parse day, then remove it from the string
+ day = atoi(str.c_str());
+ str.erase(0,2);
+ }
+
+ if (str.size() == 4 or str.size() == 7 or str.size() == 10) {
+ // parse year, then remove it from the string
+ year = atoi(str.c_str());
+ str.erase(0,5);
+ }
+
+ month = atoi(str.c_str());
+ str.erase(0,3);
+
+ // Do we have a day component left?
+ if (str.size())
+ day = atoi(str.c_str());
+
+ if (year == 0 && month == 0 && day == 0)
+ // no expected time format
+ return NULL ;
+
+ timet = time(NULL);
+ timeinfo = gmtime( &timet );
+ timeinfo -> tm_isdst = 0;
+ if (!early && !month) ++year;
+ if (year) timeinfo -> tm_year = year - 1900;
+
+ if (month) timeinfo -> tm_mon = month - 1;
+ //else if (year) timeinfo -> tm_mon = (early ? 0: 12);
+
+ if (day) timeinfo -> tm_mday = (early ? day : ++day);
+ else timeinfo -> tm_mday = 1;
+
+ timeinfo -> tm_hour = 0;
+ timeinfo -> tm_min = 0;
+ timeinfo -> tm_sec = 0;
+ timet = mktime ( timeinfo );
+
+if (!early) --timet;
+ if (timet == -1)
+ return NULL;
+ return timet;
+ }
+
+Xapian::valueno operator()(std::string &begin, std::string &end) {
+ time_t begintime, endtime;
+
+ if (begin.substr(0, 5) != "date:")
+return Xapian::BAD_VALUENO;
+ begin.erase(0, 5);
+
+ begintime = parsedate ( begin, true);
+ if (begintime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ endtime = parsedate ( end, false);
+ if (endtime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ begin.assign(Xapian::sortable_serialise(begintime));
+ end.assign(Xapian::sortable_serialise(endtime));
+
+ return NOTMUCH_VALUE_TIMESTAMP;
+}
+};
+
notmuch_database_t *
notmuch_database_open (const char *path,
notmuch_database_mode_t mode)
@@ -570,7 +658,7 @@ notmuch_database_open (const char *path,
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
- notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP, "date:", true);
+ notmuch->value_range_processor = new MaildateValueRangeProcessor();
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
--
1.6.3.3
[notmuch] [PATCH] Add install target for desktop files.
On Sun, 22 Nov 2009 15:17:11 -0600, "Jeffrey C. Ollie" wrote: > Add an install target that uses desktop-file-install to install the > desktop file in the appropriate location. The location of the install > can be modified by changing the desktop_dir variable. > > Signed-off-by: Jeffrey C. Ollie Thanks Jeff, I've rebased this to master and committed it, (on an airplane, so hopefully I'll remember to push it out when I get back on a network and this mail goes out[*]). -Carl [*] Hmm... would be nice if I had an easy way to queue up a git push similar to the way I'm queuing up this email... -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100124/817f02d7/attachment.pgp>
[notmuch] [PATCH] notmuch.el: quote args in notmuch-show to facilitate remote use
Put single-quotes around the argument of the `show --entire-thread' command
in notmuch-show. This change should have no effect on normal usage.
However, it allows us to use the notmuch.el client with a remote notmuch
binary and database over ssh (by, e.g., setting `notmuch-command' to a
simple shell script). Without the quotes, ssh will not send the command
properly.
One very simple example script is as follows. (Note that it requires
keypair login to the ssh server.)
#!/bin/sh
SSH_BIN="/path/to/local/ssh"
NOTMUCH_HOST="my.remote.server"
NOTMUCH_REMOTE_PATH="/path/to/remote/notmuch"
$SSH_BIN $NOTMUCH_HOST $NOTMUCH_REMOTE_PATH $@
---
notmuch.el |6 --
1 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/notmuch.el b/notmuch.el
index e333f24..e6eef46 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -953,8 +953,10 @@ matching this search term are shown if non-nil. "
(erase-buffer)
(goto-char (point-min))
(save-excursion
- (let* ((basic-args (list notmuch-command nil t nil "show"
"--entire-thread" thread-id))
- (args (if query-context (append basic-args (list "and ("
query-context ")")) basic-args)))
+ (let* ((basic-args (list notmuch-command nil t nil "show"
"--entire-thread" "\'" thread-id))
+ (args (if query-context
+ (append basic-args (list "and (" query-context ")\'"))
+ (append basic-args (list "\'")
(apply 'call-process args)
(when (and (eq (buffer-size) 0) query-context)
(apply 'call-process basic-args)))
--
1.6.5.3
[notmuch] [PATCH] Notmuch shared library
Here is what I believe should be the final set of patches building notmuch as a shared library. Everything seems to work. Let me know if there are any objections. Thanks! - Ben
[notmuch] [PATCH 1/2] Fix typo in message
---
vim/plugin/notmuch.vim |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/vim/plugin/notmuch.vim b/vim/plugin/notmuch.vim
index a226f20..a9754f2 100644
--- a/vim/plugin/notmuch.vim
+++ b/vim/plugin/notmuch.vim
@@ -178,7 +178,7 @@ let g:notmuch_compose_imaps = {
function! s:NM_cmd_folders(words)
if len(a:words)
-throw 'Not exapecting any arguments for folders command.'
+throw 'Not expecting any arguments for folders command.'
endif
let cmd = ['count']
let disp = []
--
1.6.3.3
[notmuch] [PATCH 2/2] Build and link against notmuch shared library
--- Makefile |1 + Makefile.local |6 -- lib/Makefile.local | 10 ++ 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 64b9d4a..6f296bb 100644 --- a/Makefile +++ b/Makefile @@ -1,3 +1,4 @@ +SONAME = libnotmuch.so.1 WARN_CXXFLAGS=-Wall -Wextra -Wwrite-strings -Wswitch-enum WARN_CFLAGS=$(WARN_CXXFLAGS) -Wmissing-declarations diff --git a/Makefile.local b/Makefile.local index d579242..a61eb67 100644 --- a/Makefile.local +++ b/Makefile.local @@ -21,8 +21,8 @@ notmuch_client_srcs = \ show-message.c notmuch_client_modules = $(notmuch_client_srcs:.c=.o) -notmuch: $(notmuch_client_modules) lib/notmuch.a - $(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -o $@ +notmuch: $(notmuch_client_modules) lib/libnotmuch.so + $(call quiet,CC,$(LDFLAGS)) -lnotmuch $(filter-out lib/libnotmuch.so,$^) $(FINAL_LDFLAGS) -o $@ notmuch.1.gz: notmuch.1 $(call quiet,gzip) --stdout $^ > $@ @@ -33,6 +33,8 @@ install: all notmuch.1.gz install -d $$d ; \ done ; install notmuch $(DESTDIR)$(prefix)/bin/ + install lib/$(SONAME) $(DESTDIR)$(prefix)/lib/ + ln -sf $(DESTDIR)$(prefix)/lib/$(SONAME) $(DESTDIR)$(prefix)/lib/libnotmuch.so install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/ install-emacs: install emacs diff --git a/lib/Makefile.local b/lib/Makefile.local index 70489e1..cfefc9b 100644 --- a/lib/Makefile.local +++ b/lib/Makefile.local @@ -1,5 +1,5 @@ dir=lib -extra_cflags += -I$(dir) +extra_cflags += -I$(dir) -fPIC libnotmuch_c_srcs =\ $(dir)/libsha1.c\ @@ -18,8 +18,10 @@ libnotmuch_cxx_srcs =\ $(dir)/thread.cc libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o) -$(dir)/notmuch.a: $(libnotmuch_modules) - $(call quiet,AR) rcs $@ $^ +$(dir)/$(SONAME): $(libnotmuch_modules) + $(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -Wl,-soname=$(SONAME) -shared -o $@ +$(dir)/libnotmuch.so: $(dir)/$(SONAME) + ln -sf $(dir)/$(SONAME) $(dir)/libnotmuch.so SRCS := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs) -CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a +CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/libnotmuch.so $(dir)/$(SONAME) -- 1.6.3.3
[notmuch] [PATCH] Build and link against notmuch shared library
Excerpts from bgamari.foss's message of Sat Jan 23 12:58:42 -0600 2010: > True, but I don't think that this means that we need to link the > executable with a C++ compiler. I've tried linking with CC and it seems > to succeed, so I don't think there should be a problem changing it. > Are you using "cc" or "gcc"? Even when cc is an alias to gcc, there are different semantics. And "cc" vs. "cpp" can be even more different on non-GCC compilers. I wouldn't mess with it. :-)
[notmuch] [PATCH] Make the date parser nicer
Currently we have to enter mail dates as timestamps. This approach does 2
things: 1) it requires the prefix 'date:'
2) it allows dates to be specified in some formats. So a notmuch show
date:2005..2006-05-12 will find all mails from 2005-01-01 until 2006-05-12.
The code is probably not in a proper location yet and needs to be shoved around
by someone more knowledgable than me.
My C++ skills are somewhat,... lacking...
Possible time formats: -MM-DD,-MM (in that month) , (in that year)
MM-DD (month-day in current year), DD (day in current month)
Signed-off-by: Sebastian Spaeth
---
lib/database.cc | 90 ++-
1 files changed, 89 insertions(+), 1 deletions(-)
diff --git a/lib/database.cc b/lib/database.cc
index 5b12320..9c2842d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -494,6 +494,94 @@ _notmuch_database_ensure_writable (notmuch_database_t
*notmuch)
return NOTMUCH_STATUS_SUCCESS;
}
+struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor {
+MaildateValueRangeProcessor() {}
+
+ time_t
+ parsedate(std::string &str, bool early) {
+ // Parse the begin date to time_t
+ // possible time formats:
+ // -MM-DD (size 10)
+ // -MM(size 7)
+ // (size 4)
+ // MM-DD (size 5)
+ // DD (size 2)
+// begin of time unit when 'early', end of when not
+ struct tm *timeinfo;
+ time_t timet;
+ //const char * startptr;
+ int year = 0, month = 0, day = 0;
+
+ if (str.size() == 2) {
+ // parse day, then remove it from the string
+ day = atoi(str.c_str());
+ str.erase(0,2);
+ }
+
+ if (str.size() == 4 or str.size() == 7 or str.size() == 10) {
+ // parse year, then remove it from the string
+ year = atoi(str.c_str());
+ str.erase(0,5);
+ }
+
+ month = atoi(str.c_str());
+ str.erase(0,3);
+
+ // Do we have a day component left?
+ if (str.size())
+ day = atoi(str.c_str());
+
+ if (year == 0 && month == 0 && day == 0)
+ // no expected time format
+ return NULL ;
+
+ timet = time(NULL);
+ timeinfo = gmtime( &timet );
+ timeinfo -> tm_isdst = 0;
+ if (!early && !month) ++year;
+ if (year) timeinfo -> tm_year = year - 1900;
+
+ if (month) timeinfo -> tm_mon = month - 1;
+ //else if (year) timeinfo -> tm_mon = (early ? 0: 12);
+
+ if (day) timeinfo -> tm_mday = (early ? day : ++day);
+ else timeinfo -> tm_mday = 1;
+
+ timeinfo -> tm_hour = 0;
+ timeinfo -> tm_min = 0;
+ timeinfo -> tm_sec = 0;
+ timet = mktime ( timeinfo );
+
+if (!early) --timet;
+ if (timet == -1)
+ return NULL;
+ return timet;
+ }
+
+Xapian::valueno operator()(std::string &begin, std::string &end) {
+ time_t begintime, endtime;
+
+ if (begin.substr(0, 5) != "date:")
+return Xapian::BAD_VALUENO;
+ begin.erase(0, 5);
+
+ begintime = parsedate ( begin, true);
+ if (begintime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ endtime = parsedate ( end, false);
+ if (endtime == -1)
+ // no valid time format
+ return Xapian::BAD_VALUENO;
+
+ begin.assign(Xapian::sortable_serialise(begintime));
+ end.assign(Xapian::sortable_serialise(endtime));
+
+ return NOTMUCH_VALUE_TIMESTAMP;
+}
+};
+
notmuch_database_t *
notmuch_database_open (const char *path,
notmuch_database_mode_t mode)
@@ -570,7 +658,7 @@ notmuch_database_open (const char *path,
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
- notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor
(NOTMUCH_VALUE_TIMESTAMP, "date:", true);
+ notmuch->value_range_processor = new MaildateValueRangeProcessor();
notmuch->query_parser->set_default_op (Xapian::Query::OP_AND);
notmuch->query_parser->set_database (*notmuch->xapian_db);
--
1.6.3.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Add install target for desktop files.
On Sun, 22 Nov 2009 15:17:11 -0600, "Jeffrey C. Ollie" wrote: > Add an install target that uses desktop-file-install to install the > desktop file in the appropriate location. The location of the install > can be modified by changing the desktop_dir variable. > > Signed-off-by: Jeffrey C. Ollie Thanks Jeff, I've rebased this to master and committed it, (on an airplane, so hopefully I'll remember to push it out when I get back on a network and this mail goes out[*]). -Carl [*] Hmm... would be nice if I had an easy way to queue up a git push similar to the way I'm queuing up this email... pgpQYhZfFp5Da.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] notmuch.el: quote args in notmuch-show to facilitate remote use
Put single-quotes around the argument of the `show --entire-thread' command
in notmuch-show. This change should have no effect on normal usage.
However, it allows us to use the notmuch.el client with a remote notmuch
binary and database over ssh (by, e.g., setting `notmuch-command' to a
simple shell script). Without the quotes, ssh will not send the command
properly.
One very simple example script is as follows. (Note that it requires
keypair login to the ssh server.)
#!/bin/sh
SSH_BIN="/path/to/local/ssh"
NOTMUCH_HOST="my.remote.server"
NOTMUCH_REMOTE_PATH="/path/to/remote/notmuch"
$SSH_BIN $NOTMUCH_HOST $NOTMUCH_REMOTE_PATH $@
---
notmuch.el |6 --
1 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/notmuch.el b/notmuch.el
index e333f24..e6eef46 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -953,8 +953,10 @@ matching this search term are shown if non-nil. "
(erase-buffer)
(goto-char (point-min))
(save-excursion
- (let* ((basic-args (list notmuch-command nil t nil "show"
"--entire-thread" thread-id))
- (args (if query-context (append basic-args (list "and ("
query-context ")")) basic-args)))
+ (let* ((basic-args (list notmuch-command nil t nil "show"
"--entire-thread" "\'" thread-id))
+ (args (if query-context
+ (append basic-args (list "and (" query-context ")\'"))
+ (append basic-args (list "\'")
(apply 'call-process args)
(when (and (eq (buffer-size) 0) query-context)
(apply 'call-process basic-args)))
--
1.6.5.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Notmuch shared library
Here is what I believe should be the final set of patches building notmuch as a shared library. Everything seems to work. Let me know if there are any objections. Thanks! - Ben ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH 1/2] Fix typo in message
---
vim/plugin/notmuch.vim |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/vim/plugin/notmuch.vim b/vim/plugin/notmuch.vim
index a226f20..a9754f2 100644
--- a/vim/plugin/notmuch.vim
+++ b/vim/plugin/notmuch.vim
@@ -178,7 +178,7 @@ let g:notmuch_compose_imaps = {
function! s:NM_cmd_folders(words)
if len(a:words)
-throw 'Not exapecting any arguments for folders command.'
+throw 'Not expecting any arguments for folders command.'
endif
let cmd = ['count']
let disp = []
--
1.6.3.3
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH 2/2] Build and link against notmuch shared library
--- Makefile |1 + Makefile.local |6 -- lib/Makefile.local | 10 ++ 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/Makefile b/Makefile index 64b9d4a..6f296bb 100644 --- a/Makefile +++ b/Makefile @@ -1,3 +1,4 @@ +SONAME = libnotmuch.so.1 WARN_CXXFLAGS=-Wall -Wextra -Wwrite-strings -Wswitch-enum WARN_CFLAGS=$(WARN_CXXFLAGS) -Wmissing-declarations diff --git a/Makefile.local b/Makefile.local index d579242..a61eb67 100644 --- a/Makefile.local +++ b/Makefile.local @@ -21,8 +21,8 @@ notmuch_client_srcs = \ show-message.c notmuch_client_modules = $(notmuch_client_srcs:.c=.o) -notmuch: $(notmuch_client_modules) lib/notmuch.a - $(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -o $@ +notmuch: $(notmuch_client_modules) lib/libnotmuch.so + $(call quiet,CC,$(LDFLAGS)) -lnotmuch $(filter-out lib/libnotmuch.so,$^) $(FINAL_LDFLAGS) -o $@ notmuch.1.gz: notmuch.1 $(call quiet,gzip) --stdout $^ > $@ @@ -33,6 +33,8 @@ install: all notmuch.1.gz install -d $$d ; \ done ; install notmuch $(DESTDIR)$(prefix)/bin/ + install lib/$(SONAME) $(DESTDIR)$(prefix)/lib/ + ln -sf $(DESTDIR)$(prefix)/lib/$(SONAME) $(DESTDIR)$(prefix)/lib/libnotmuch.so install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/ install-emacs: install emacs diff --git a/lib/Makefile.local b/lib/Makefile.local index 70489e1..cfefc9b 100644 --- a/lib/Makefile.local +++ b/lib/Makefile.local @@ -1,5 +1,5 @@ dir=lib -extra_cflags += -I$(dir) +extra_cflags += -I$(dir) -fPIC libnotmuch_c_srcs =\ $(dir)/libsha1.c\ @@ -18,8 +18,10 @@ libnotmuch_cxx_srcs =\ $(dir)/thread.cc libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o) -$(dir)/notmuch.a: $(libnotmuch_modules) - $(call quiet,AR) rcs $@ $^ +$(dir)/$(SONAME): $(libnotmuch_modules) + $(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -Wl,-soname=$(SONAME) -shared -o $@ +$(dir)/libnotmuch.so: $(dir)/$(SONAME) + ln -sf $(dir)/$(SONAME) $(dir)/libnotmuch.so SRCS := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs) -CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a +CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/libnotmuch.so $(dir)/$(SONAME) -- 1.6.3.3 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
also sprach Asheesh Laroia [2010.01.21.1928 +1300]:
> >I suppose that I never actually considered merges on the IMAP
> >server side, but obviously the IMAP server has to work off a clone,
> >and that means it needs to merge.
>
> It's not "merge" that's unsafe; that just builds a tree in the git
> index (assuming no conflicts). It's the ensuing process of git
> writing a tree to the filesystem that is problematic.
There is no way to make that atomic, I am afraid. As you say.
> I could probably actually write a wrapper that locks the Maildir
> while git is operating. It would probably be specific to each IMAP
> server.
Ouch! I'd really rather not go there.
> Note that this mean git is fundamentally incompatible with
> Maildir, not just IMAP servers.
We had an idea about using Git to replace IMAP altogether, along
with making notmuch use a bare Git repository as object store. The
idea is that notmuch uses low-level Git commands to access the .git
repository (from which you can still checkout a tree tying the blobs
into a Maildir). The benefit would be compression, lower inode count
(due to packs), and backups using clones/merges.
You could either have the MDA write to a Git repo on the server side
and use git packs to download mail to a local clone, or one could
have e.g. offlineimap grow a Git storage backend. The interface to
notmuch would be the same.
If we used this, all the rename and delete code would be refactored
into Git and could be removed from notmuch. In addition, notmuch
could actually use Git tree objects to represent the results of
searches, and you could checkout these trees. However, deleting
messages from search results would not have any effect on the
message or its existence in other search results, much like what
happens with mairix nowadays.
I think we all kinda agreed that the Maildir flags should not be
used by notmuch and that things like Sebastian's notmuchsync should
be used if people wanted flags represented in Maildir filenames.
Instead of a Maildir checkout, notmuch could provide an interface to
browse the store contents in a way that could make it accessible to
mutt. The argument is that with 'notmuch {ls,cat,rm,…}', a mutt
backend could be trivially written. I am not sure about that, but
it's worth a try.
But there are still good reasons why you'd want to have IMAP
capability too, e.g. Webmail. Given the atomicity problems that come
from Git, maybe an IMAP server reading from the Git store would make
sense.
However, this all sounds like a lot of NIH and reinvention. It's
a bit like the marriage between the hypothetical Maildir2 and Git,
which is definitely worth pursuing. Before we embark on any of this,
however, we'd need to define the way in which Git stores mail.
Stewart, you've worked most on this so far. Would you like to share
your thoughts?
--
martin | http://madduck.net/ | http://two.sentenc.es/
"reife des mannes, das ist es,
den ernst wiedergefunden zu haben, den
man hatte als kind beim spiel."
-- friedrich nietzsche
spamtraps: madduck.bo...@madduck.net
digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Build and link against notmuch shared library
Excerpts from bgamari.foss's message of Sat Jan 23 12:58:42 -0600 2010: > True, but I don't think that this means that we need to link the > executable with a C++ compiler. I've tried linking with CC and it seems > to succeed, so I don't think there should be a problem changing it. > Are you using "cc" or "gcc"? Even when cc is an alias to gcc, there are different semantics. And "cc" vs. "cpp" can be even more different on non-GCC compilers. I wouldn't mess with it. :-) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
On Mon, 25 Jan 2010, martin f krafft wrote:
also sprach Asheesh Laroia [2010.01.21.1928
+1300]:
I suppose that I never actually considered merges on the IMAP server
side, but obviously the IMAP server has to work off a clone, and that
means it needs to merge.
It's not "merge" that's unsafe; that just builds a tree in the git
index (assuming no conflicts). It's the ensuing process of git writing
a tree to the filesystem that is problematic.
There is no way to make that atomic, I am afraid. As you say.
I could probably actually write a wrapper that locks the Maildir while
git is operating. It would probably be specific to each IMAP server.
Ouch! I'd really rather not go there.
You say "Ouch" but you should know Dovecot *already* does this. I don't
mind interoperating with that.
See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues with
the specification", subsection "Locking". I term this the famous readdir()
race. Without this lock, Maildir is fundamentally incompatible with IMAP
-- one Maildir-using process modifying message flags could make a
different Maildir-using process think said message is actually deleted. In
the case of temporary disappearing mails in Mutt locally, that's not the
end of the world. For IMAP, it will make the IMAP daemon (one of the
Maildir-using processes) send a note to IMAP clients saying that the
message has been deleted and expunged.
Note that this mean git is fundamentally incompatible with Maildir, not
just IMAP servers.
We had an idea about using Git to replace IMAP altogether, along with
making notmuch use a bare Git repository as object store. The idea is
that notmuch uses low-level Git commands to access the .git repository
(from which you can still checkout a tree tying the blobs into a
Maildir). The benefit would be compression, lower inode count (due to
packs), and backups using clones/merges.
Sure, that makes sense to me.
You could either have the MDA write to a Git repo on the server side and
use git packs to download mail to a local clone, or one could have e.g.
offlineimap grow a Git storage backend. The interface to notmuch would
be the same.
Yeah, I generally like this.
If we used this, all the rename and delete code would be refactored into
Git and could be removed from notmuch. In addition, notmuch could
actually use Git tree objects to represent the results of searches, and
you could checkout these trees. However, deleting messages from search
results would not have any effect on the message or its existence in
other search results, much like what happens with mairix nowadays.
That's okay with me.
I think we all kinda agreed that the Maildir flags should not be used by
notmuch and that things like Sebastian's notmuchsync should be used if
people wanted flags represented in Maildir filenames.
Aww, I like Maildir flags, but if there's a sync tool, I'm fine with that.
Instead of a Maildir checkout, notmuch could provide an interface to
browse the store contents in a way that could make it accessible to
mutt. The argument is that with 'notmuch {ls,cat,rm,…}', a mutt backend
could be trivially written. I am not sure about that, but it's worth a
try.
Sure.
But there are still good reasons why you'd want to have IMAP capability
too, e.g. Webmail. Given the atomicity problems that come from Git,
maybe an IMAP server reading from the Git store would make sense.
It wouldn't be too hard to write a FUSE filesystem that presented an
interface to a Git repository that didn't allow the contents of files to
be modified. Then Dovecot could think it's interacting with the
filesystem.
However, this all sounds like a lot of NIH and reinvention. It's
a bit like the marriage between the hypothetical Maildir2 and Git,
which is definitely worth pursuing. Before we embark on any of this,
however, we'd need to define the way in which Git stores mail.
Sure. If it were me, I'd just say, "For phase 1 of notmuch, just have git
store Maildir spools." When you need a filesystem interface for e.g.
Dovecot, have a FUSE wrapper.
See how far that can take you, and then see if version 2 is necessary.
(-:
Stewart, you've worked most on this so far. Would you like to share your
thoughts?
I'll listen, too.
Just don't fall into the trap of thinking Maildir is compatible with IMAP.
It's not, because as I understand things, the filesystem doesn't guarantee
that you can actually iterate across a directory's files if another
process is modifying the list of files.
I'm not sure, but maybe it's safe if you refuse to ever modify a
message's flags in the filename.
Anyway, as I see it, further hacks that aren't much worse than Dovecot's
should be considered okay, unless you have a more elegant design up your
sleeve.
If I'm slightly wrong about something, try to give me the benefit of
doubt. It's past midnight. (-:
-- Asheesh.
--
There's no real need to do housework -- after four years it doe
Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
also sprach Asheesh Laroia [2010.01.25.1819 +1300]: > You say "Ouch" but you should know Dovecot *already* does this. I > don't mind interoperating with that. > > See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues > with the specification", subsection "Locking". I term this theQ > famous readdir() race. Yikes. IMAP (including dovecot) just SUCKS. > Without this lock, Maildir is fundamentally incompatible with IMAP > -- one Maildir-using process modifying message flags could make > a different Maildir-using process think said message is actually > deleted. In the case of temporary disappearing mails in Mutt > locally, that's not the end of the world. For IMAP, it will make > the IMAP daemon (one of the Maildir-using processes) send a note > to IMAP clients saying that the message has been deleted and > expunged. […] > Just don't fall into the trap of thinking Maildir is compatible > with IMAP. It's not, because as I understand things, the > filesystem doesn't guarantee that you can actually iterate across > a directory's files if another process is modifying the list of > files. This is all perfect reason to concentrate even more on designing a store that could potentially make IMAP obsolete once and for all! The current idea is to sync Git downstream only, and find a way to keep multiple copies of a tagstore in sync, by way of the "server instance" (where mail is received/delivered). Deleting messages would then be something like setting the notmuch::deleted tag, which clients would honour; on the server, a cleanup process would run regularly to actually delete the blobs associated with deleted messages. This would then propogate the next time one pulls from Git. Whether to store history (commit objects) or just collections (tree objects) needs to be investigated. > >But there are still good reasons why you'd want to have IMAP > >capability too, e.g. Webmail. Given the atomicity problems that > >come from Git, maybe an IMAP server reading from the Git store > >would make sense. > > It wouldn't be too hard to write a FUSE filesystem that presented > an interface to a Git repository that didn't allow the contents of > files to be modified. Then Dovecot could think it's interacting > with the filesystem. Yes, a FUSE layer (which adds a daemon), or a lightweight access API via libnotmuch. Probably the former using the latter. ;) > Aww, I like Maildir flags, but if there's a sync tool, I'm fine > with that. […] > I'm not sure, but maybe it's safe if you refuse to ever modify > a message's flags in the filename. The main point is that there is nothing really in Maildir filenames that you couldn't equally (and possibly better) represent in the notmuch::* tag namespace, and then there is benefit in only having one used primarily (which means notmuchsync can do whatever it wants without affecting or messing with notmuch). -- martin | http://madduck.net/ | http://two.sentenc.es/ "if I can't dance, i don't want to be part of your revolution." - emma goldman spamtraps: madduck.bo...@madduck.net digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
