[notmuch] [PATCH] Add SWIG interface file
--- swig/Makefile| 18 swig/notmuch.py | 222 ++ swig/notmuch_funcs.i |9 ++ 3 files changed, 249 insertions(+), 0 deletions(-) create mode 100644 swig/Makefile create mode 100644 swig/notmuch.py create mode 100644 swig/notmuch_funcs.i diff --git a/swig/Makefile b/swig/Makefile new file mode 100644 index 000..7b10ea7 --- /dev/null +++ b/swig/Makefile @@ -0,0 +1,18 @@ +include ../Makefile.config + +INCLUDES=-I../lib -I/usr/include/python2.6 +CFLAGS=${INCLUDES} + +all : python + +python : _notmuch_funcs.so notmuch.py + +_notmuch_funcs.so : notmuch_funcs_wrap.o + g++ -shared -lnotmuch ${XAPIAN_LDFLAGS} ${GMIME_LDFLAGS} ${TALLOC_LDFLAGS} -o $@ $< + +%_wrap.o : %_wrap.c + gcc ${CFLAGS} -fPIC -c -o $@ $< + +%_wrap.c %.py : %.i + swig -python ${INCLUDES} $< + diff --git a/swig/notmuch.py b/swig/notmuch.py new file mode 100644 index 000..e17f71f --- /dev/null +++ b/swig/notmuch.py @@ -0,0 +1,222 @@ +import notmuch_funcs as nm +from datetime import datetime + +class Exception(Exception): +def get_message(): +errors = { +nm.NOTMUCH_STATUS_OUT_OF_MEMORY: 'out of memory', +nm.NOTMUCH_STATUS_READONLY_DATABASE: 'database opened as read-only', +nm.NOTMUCH_STATUS_XAPIAN_EXCEPTION: 'xapian error', +nm.NOTMUCH_STATUS_FILE_ERROR: 'file error', +nm.NOTMUCH_STATUS_FILE_NOT_EMAIL: 'file not email message', +nm.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: 'duplicate message id', +nm.NOTMUCH_STATUS_NULL_POINTER: 'null pointer', +nm.NOTMUCH_STATUS_TAG_TOO_LONG: 'tag name too long', +nm.NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW: 'unbalanced freeze/thaw', +} +return errors.get(self.status, 'unknown error') + +def __init__(self, status): +self.status = status + +def _handle_status(status): +if (status != nm.NOTMUCH_STATUS_SUCCESS): +raise Exception(status) + +class Database(object): +MODE_READ_ONLY = nm.NOTMUCH_DATABASE_MODE_READ_ONLY +MODE_READ_WRITE = nm.NOTMUCH_DATABASE_MODE_READ_WRITE + +def __init__(self, db): +if not db: +raise Exception("Failed to open database") +self.db = db + +@staticmethod +def create(path): +return Database(nm.notmuch_database_create(path)) + +@staticmethod +def open(path, mode): +return Database(nm.notmuch_database_open(path, mode)) + +def close(self): +nm.notmuch_database_close(self.db) + +def get_path(self): +return nm.notmuch_database_get_path(self.db) + +def set_timestamp(self, key, timestamp): +_handle_status(nm.notmuch_database_set_timestamp(self.db, key, timestamp)) + +def get_timestamp(self, key): +return datetime.fromtimestamp(nm.notmuch_database_get_timestamp(self.db, key)) + +def add_message(self, filename, message): +_handle_status(nm.notmuch_database_add_message(self.db, filename, message.message)) + +def find_message(self, message_id): +return Message(nm.notmuch_database_find_message(self.db, message_id)) + +def get_all_tags(self): +return Tags(nm.notmuch_database_get_all_tags(self.db)) + +def __destroy__(self): +self.close() + +class Query(object): +def __init__(self, db, query_string): +self.query = nm.notmuch_query_create(db, query_string) +if not self.query: # This might not work +raise "Failed to create query" + +def set_sort(self, sort): +nm.notmuch_query_set_sort(self.query, sort) + +def search_threads(self): +return Threads(nm.notmuch_query_search_threads(self.query)) + +def search_messages(self): +return Messages(nm.notmuch_query_search_messages(self.query)) + +def count_message(self): +return nm.notmuch_query_count_messages(self.query) + +def __destroy__(self): +nm.notmuch_query_destroy(self.query) + +class Tags(object): +def __init__(self, tags): +self.tags = tags + +def __iter__(self): +return self + +def next(self): +if not nm.notmuch_tags_has_more(self.tags): +raise StopIteration +else: +h = nm.notmuch_tags_get(self.tags) +nm.notmuch_tags_advance(self.tags) +return h + +def __destroy__(self): +nm.notmuch_messages_destroy(self
[notmuch] [PATCH] SWIG bindings
Hey all, Here's the latest version of my patch adding SWIG interface generation to notmuch. It has been rebased on the shared-library patches I sent over earlier this week, so you'll need those as well. Unfortunately, SWIG has effectively no support for exposing notmuch's C-style object-oriented interface in a reasonable manner. Therefore, I use SWIG to generate a set of low-level bindings which is then wrapped with a light-weight module properly exposing the functions in their intended class structure. This seems like a fairly good solution, given the wide variety of capabilities possessed by modern high-level languages. As you can see, it all works quite nicely, [2252 ben at ben-laptop swig(swig)] $ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from notmuch import Database >>> db=Database.open('/home/ben/.mail', Database.MODE_READ_ONLY) >>> for tag in db.get_all_tags(): print tag ... attachment inbox unread Let me know what you think. Thanks! - Ben
Re: [notmuch] [PATCH] Make the date parser nicer
On Fri, 22 Jan 2010 16:26:11 +0100, Sebastian Spaeth wrote: > Currently we have to enter mail dates as timestamps. This approach > does 2 things: it requires the prefix 'date:' and it allows timestamps > to be specified as , MM or MMDD. So a notmuch show > date:2005..20060512 will find all mails from 2005-01-01 until > 2006-05-12. The code is probably not in a proper location yet and > needs to be shoved around by someone more knowledgable than me. My C++ > skills are somewhat,... lacking... Here's some code which further improves date parsing by allowing lots of date formats, including things like "today", "thisweek", ISO and US date formats and month names. You can separate two dates with .. to make a range, or you can just use the default range ("lastmonth" is everything From the 1st of the previous month to the 1st of the current month). I think this fits nicely with your code. From 432b210a6218a809ebcddbb0e5b94a1ddbd34207 Mon Sep 17 00:00:00 2001 From: Keith Packard Date: Mon, 25 Jan 2010 22:35:30 -0800 Subject: [PATCH] Add date.c --- lib/date.c | 455 1 files changed, 455 insertions(+), 0 deletions(-) create mode 100644 lib/date.c diff --git a/lib/date.c b/lib/date.c new file mode 100644 index 000..611ae15 --- /dev/null +++ b/lib/date.c @@ -0,0 +1,455 @@ +/* + * Copyright © 2009 Keith Packard + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA. + */ + +#include "notmuch.h" +#include +#include +#include +#include + +#define DAY(24 * 60 * 60) + +static void +today(struct tm *result, time_t after) { +time_t t; + +if (after) + t = after; +else + time(&t); +localtime_r(&t, result); +result->tm_sec = result->tm_min = result->tm_hour = 0; +} + +static int parse_today(const char *text, time_t *first, time_t *last, time_t after) { +if (strcasecmp(text, "today") == 0) { + struct tm n; + today(&n, 0); + *first = mktime(&n); + *last = *first + DAY; + return 0; +} +return 1; +} + +static int parse_yesterday(const char *text, time_t *first, time_t *last, time_t after) { +if (strcasecmp(text, "yesterday") == 0) { + struct tm n; + today(&n, 0); + *last = mktime(&n); + *first = *last - DAY; + return 0; +} +return 1; +} + +static int parse_thisweek(const char *text, time_t *first, time_t *last, time_t after) { +if (strcasecmp(text, "thisweek") == 0) { + struct tm n; + today(&n, 0); + *first = mktime(&n) - (n.tm_wday * DAY); + *last = *first + DAY * 7; + return 0; +} +return 1; +} + +static int parse_lastweek(const char *text, time_t *first, time_t *last, time_t after) { +if (strcasecmp(text, "lastweek") == 0) { + struct tm n; + today(&n, 0); + *last = mktime(&n) - (n.tm_wday * DAY); + *first = *last - DAY * 7; + return 0; +} +return 1; +} + +static int parse_thismonth(const char *text, time_t *first, time_t *last, time_t after) { +if (strcasecmp(text, "thismonth") == 0) { + struct tm n; + today(&n, 0); + n.tm_mday = 1; + *first = mktime(&n); + if (n.tm_mon++ == 12) { + n.tm_mon = 0; + n.tm_year++; + } + *last = mktime(&n); + return 0; +} +return 1; +} + +static int parse_lastmonth(const char *text, time_t *first, time_t *last, time_t after) { +if (strcasecmp(text, "lastmonth") == 0) { + struct tm n; + today(&n, 0); + n.tm_mday = 1; + if (n.tm_mon == 0) { + n.tm_year--; + n.tm_mon = 11; + } else + n.tm_mon--; + *first = mktime(&n); + if (n.tm_mon++ == 12) { + n.tm_mon = 0; + n.tm_year++; + } + *last = mktime(&n); + return 0; +} +return 1; +} + +static const char *months[12][2] = { +{ "January", "Jan" }, +{ "February", "Feb" }, +{ "March", "Mar" }, +{ "April", "Apr" }, +{ "May", "May" }, +{ "June", "Jun" }, +{ "July", "Jul" }, +{ "August", "Aug" }, +{ "September", "Sep" }, +{ "October", "Oct" }, +{ "November", "Nov" }, +{ "December", "Dec" }, +}; + +static int year(const char *text, int *y) { +char *end; +
[notmuch] [PATCH] Make the date parser nicer
On Fri, 22 Jan 2010 16:26:11 +0100, Sebastian Spaeth wrote: > Currently we have to enter mail dates as timestamps. This approach > does 2 things: it requires the prefix 'date:' and it allows timestamps > to be specified as , MM or MMDD. So a notmuch show > date:2005..20060512 will find all mails from 2005-01-01 until > 2006-05-12. The code is probably not in a proper location yet and > needs to be shoved around by someone more knowledgable than me. My C++ > skills are somewhat,... lacking... Here's some code which further improves date parsing by allowing lots of date formats, including things like "today", "thisweek", ISO and US date formats and month names. You can separate two dates with .. to make a range, or you can just use the default range ("lastmonth" is everything
[PATCH] Add date.c
ext, time_t *first, time_t *last, time_t after) = { +parse_today, +parse_yesterday, +parse_thisweek, +parse_lastweek, +parse_thismonth, +parse_lastmonth, +parse_month, +parse_iso, +parse_us, +0, +}; + +static notmuch_status_t +notmuch_one_date(const char *text, time_t *first, time_t *last, time_t after) +{ +inti; +for (i = 0; parsers[i]; i++) + if (parsers[i](text, first, last, after) == 0) + return NOTMUCH_STATUS_SUCCESS; +return NOTMUCH_STATUS_INVALID_DATE; +} + +notmuch_status_t +notmuch_date(const char *text, time_t *first, time_t *last) +{ +char *dots; +char first_text[80], last_text[80]; +notmuch_status_t status; +time_t first_first, first_last, last_first, last_last; + +if (strlen(text) > sizeof (first_text)) + return NOTMUCH_STATUS_INVALID_DATE; +dots = strstr(text, ".."); +if (dots) { + strncpy(first_text, text, dots - text); + first_text[dots-text] = '\0'; + status = notmuch_one_date(first_text, &first_first, &first_last, 0); + if (status) + return status; + status = notmuch_one_date(dots + 2, &last_first, &last_last, first_first); + if (status) + return status; + *first = first_first; + *last = last_last; + return 0; +} +return notmuch_one_date(text, first, last, 0); +} + +#if 1 +int +main (int argc, char **argv) +{ +inti; +for (i = 1; i < argc; i++) { + time_t first, last; + + if (notmuch_date(argv[i], &first, &last) == 0) { + charfirst_string[80], last_string[80]; + + ctime_r(&first, first_string); + first_string[strlen(first_string)-1] = '\0'; + ctime_r(&last, last_string); + last_string[strlen(last_string)-1] = '\0'; + printf ("%s: %s - %s\n", argv[i], first_string, last_string); + } +} +} +#endif -- 1.6.6 -- keith.packard at intel.com -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/1fa59cc1/attachment-0001.pgp>
[notmuch] [PATCH] Mail::Notmuch Perl wrapper
Excerpts from Simon Cozens's message of Mon Jan 25 13:17:16 -0500 2010: > (Resent with compressed patch) > > Hi there, > Here's a Perl library, Mail::Notmuch, which wraps the notmuch library. > I've attached it because it's pretty huge. It requires you to > build notmuch as a shared library - there's a patch on the mailing list > earlier which does that. I know practically nothing about writing Perl bindings, but it seems like this might be work better left to a bindings generator. I currently have a patch which enables binding generation through SWIG. It works well, although SWIG unfortunately doesn't have very good support for exposing object-oriented C interfaces like notmuchs' through the target language's type system. Nonetheless, this is easily done by a wrapper library to prettify the binding (which in the case of Python took very little time). I sent the patchset out a few weeks ago, but I'll rebase it on top of my shared-library branch and send it out again tonight. Cheers, - Ben
[notmuch] proposal for more streamlined mail flow in notmuch
Excerpts from Jameson Rollins's message of Sat Jan 16 15:49:55 -0500 2010: > To fascilitate this, two new functions should be implemented at the > notmuch CLI level, so that things don't get fractured by different > mail reader implementations: > > notmuch purge > > delete all 'delete' tagged messages, and/or execute "delete hooks" > > notmuch archive > > remove 'inbox' flags from messages and move messages an archive > maildir, and/or execute "archive hooks" > I'm a little suspicious of this. Again, it seems like there is nothing whatsoever preventing one from implementing this at a higher-level than notmuch, which I believe should be little more than a thin-wrapper around Xapian unless more functionality is absolutely necessary. I don't understand why these sorts of proposals keep coming up when the above, for example, could be easily implemented in a pair of ten line shell scripts. Am I missing something? In my opinion, the "fractured implementations" argument is mislead at best. We currently have exactly two (in the case of vim, somewhat) usable UIs. It's not difficult to see to it that they obey similar conventions. If another UI pops up that fails to obey your favorite convention, feel free to patch it. IMHO, it is not the place of notmuch to force semantics onto the user (in fact, it tries to avoid this wherever possible). - Ben
[notmuch] [PATCH] Build and link against notmuch shared library
Inger in #notmuch brought to light some build issues that will occur when the notmuch binary is being built before libnotmuch is installed. Here is an updated patch that resolves these issues. --- .gitignore |1 + Makefile |1 + Makefile.local |6 -- lib/Makefile.local |9 + 4 files changed, 11 insertions(+), 6 deletions(-) diff --git a/.gitignore b/.gitignore index efa98fb..daf8094 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,7 @@ tags .deps notmuch notmuch.1.gz +libnotmuch.so* *.[ao] *~ .*.swp diff --git a/Makefile b/Makefile index 64b9d4a..6f296bb 100644 --- a/Makefile +++ b/Makefile @@ -1,3 +1,4 @@ +SONAME = libnotmuch.so.1 WARN_CXXFLAGS=-Wall -Wextra -Wwrite-strings -Wswitch-enum WARN_CFLAGS=$(WARN_CXXFLAGS) -Wmissing-declarations diff --git a/Makefile.local b/Makefile.local index 04bac83..5b6af0a 100644 --- a/Makefile.local +++ b/Makefile.local @@ -21,8 +21,8 @@ notmuch_client_srcs = \ show-message.c notmuch_client_modules = $(notmuch_client_srcs:.c=.o) -notmuch: $(notmuch_client_modules) lib/notmuch.a - $(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -o $@ +notmuch: $(notmuch_client_modules) lib/libnotmuch.so + $(call quiet,CC,$(LDFLAGS)) -Llib -lnotmuch $(filter-out lib/libnotmuch.so,$^) $(FINAL_LDFLAGS) -o $@ notmuch.1.gz: notmuch.1 $(call quiet,gzip) --stdout $^ > $@ @@ -33,6 +33,8 @@ install: all notmuch.1.gz install -d $$d ; \ done ; install notmuch $(DESTDIR)$(prefix)/bin/ + install lib/$(SONAME) $(DESTDIR)$(prefix)/lib/ + ln -sf $(DESTDIR)$(prefix)/lib/$(SONAME) $(DESTDIR)$(prefix)/lib/libnotmuch.so install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/ install-emacs: install emacs diff --git a/lib/Makefile.local b/lib/Makefile.local index 70489e1..a6462ae 100644 --- a/lib/Makefile.local +++ b/lib/Makefile.local @@ -1,5 +1,5 @@ dir=lib -extra_cflags += -I$(dir) +extra_cflags += -I$(dir) -fPIC libnotmuch_c_srcs =\ $(dir)/libsha1.c\ @@ -18,8 +18,9 @@ libnotmuch_cxx_srcs = \ $(dir)/thread.cc libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o) -$(dir)/notmuch.a: $(libnotmuch_modules) - $(call quiet,AR) rcs $@ $^ +$(dir)/libnotmuch.so : $(libnotmuch_modules) + $(call quiet,CXX,$(LDFLAGS)) $^ $(FINAL_LDFLAGS) -shared -Wl,-soname=$(SONAME) -o $@ + ln -sf $(SONAME) $@ SRCS := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs) -CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a +CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/libnotmuch.so *.so -- 1.6.3.3 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Add SWIG interface file
For the record, both the swig branch and the shared-library branch are available from, anonymous: git://goldnerlab.physics.umass.edu/notmuch gitweb: http://goldnerlab.physics.umass.edu/git?p=notmuch.git;a=summary Hope this helps, - Ben ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
also sprach Asheesh Laroia [2010.01.25.1819 +1300]: > You say "Ouch" but you should know Dovecot *already* does this. I > don't mind interoperating with that. > > See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues > with the specification", subsection "Locking". I term this theQ > famous readdir() race. Yikes. IMAP (including dovecot) just SUCKS. > Without this lock, Maildir is fundamentally incompatible with IMAP > -- one Maildir-using process modifying message flags could make > a different Maildir-using process think said message is actually > deleted. In the case of temporary disappearing mails in Mutt > locally, that's not the end of the world. For IMAP, it will make > the IMAP daemon (one of the Maildir-using processes) send a note > to IMAP clients saying that the message has been deleted and > expunged. [?] > Just don't fall into the trap of thinking Maildir is compatible > with IMAP. It's not, because as I understand things, the > filesystem doesn't guarantee that you can actually iterate across > a directory's files if another process is modifying the list of > files. This is all perfect reason to concentrate even more on designing a store that could potentially make IMAP obsolete once and for all! The current idea is to sync Git downstream only, and find a way to keep multiple copies of a tagstore in sync, by way of the "server instance" (where mail is received/delivered). Deleting messages would then be something like setting the notmuch::deleted tag, which clients would honour; on the server, a cleanup process would run regularly to actually delete the blobs associated with deleted messages. This would then propogate the next time one pulls from Git. Whether to store history (commit objects) or just collections (tree objects) needs to be investigated. > >But there are still good reasons why you'd want to have IMAP > >capability too, e.g. Webmail. Given the atomicity problems that > >come from Git, maybe an IMAP server reading from the Git store > >would make sense. > > It wouldn't be too hard to write a FUSE filesystem that presented > an interface to a Git repository that didn't allow the contents of > files to be modified. Then Dovecot could think it's interacting > with the filesystem. Yes, a FUSE layer (which adds a daemon), or a lightweight access API via libnotmuch. Probably the former using the latter. ;) > Aww, I like Maildir flags, but if there's a sync tool, I'm fine > with that. [?] > I'm not sure, but maybe it's safe if you refuse to ever modify > a message's flags in the filename. The main point is that there is nothing really in Maildir filenames that you couldn't equally (and possibly better) represent in the notmuch::* tag namespace, and then there is benefit in only having one used primarily (which means notmuchsync can do whatever it wants without affecting or messing with notmuch). -- martin | http://madduck.net/ | http://two.sentenc.es/ "if I can't dance, i don't want to be part of your revolution." - emma goldman spamtraps: madduck.bogus at madduck.net -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature (see http://martin-krafft.net/gpg/) URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/0c16410d/attachment.pgp>
[notmuch] [PATCH] Add SWIG interface file
--- swig/Makefile| 18 swig/notmuch.py | 222 ++ swig/notmuch_funcs.i |9 ++ 3 files changed, 249 insertions(+), 0 deletions(-) create mode 100644 swig/Makefile create mode 100644 swig/notmuch.py create mode 100644 swig/notmuch_funcs.i diff --git a/swig/Makefile b/swig/Makefile new file mode 100644 index 000..7b10ea7 --- /dev/null +++ b/swig/Makefile @@ -0,0 +1,18 @@ +include ../Makefile.config + +INCLUDES=-I../lib -I/usr/include/python2.6 +CFLAGS=${INCLUDES} + +all : python + +python : _notmuch_funcs.so notmuch.py + +_notmuch_funcs.so : notmuch_funcs_wrap.o + g++ -shared -lnotmuch ${XAPIAN_LDFLAGS} ${GMIME_LDFLAGS} ${TALLOC_LDFLAGS} -o $@ $< + +%_wrap.o : %_wrap.c + gcc ${CFLAGS} -fPIC -c -o $@ $< + +%_wrap.c %.py : %.i + swig -python ${INCLUDES} $< + diff --git a/swig/notmuch.py b/swig/notmuch.py new file mode 100644 index 000..e17f71f --- /dev/null +++ b/swig/notmuch.py @@ -0,0 +1,222 @@ +import notmuch_funcs as nm +from datetime import datetime + +class Exception(Exception): +def get_message(): +errors = { +nm.NOTMUCH_STATUS_OUT_OF_MEMORY: 'out of memory', +nm.NOTMUCH_STATUS_READONLY_DATABASE: 'database opened as read-only', +nm.NOTMUCH_STATUS_XAPIAN_EXCEPTION: 'xapian error', +nm.NOTMUCH_STATUS_FILE_ERROR: 'file error', +nm.NOTMUCH_STATUS_FILE_NOT_EMAIL: 'file not email message', +nm.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: 'duplicate message id', +nm.NOTMUCH_STATUS_NULL_POINTER: 'null pointer', +nm.NOTMUCH_STATUS_TAG_TOO_LONG: 'tag name too long', +nm.NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW: 'unbalanced freeze/thaw', +} +return errors.get(self.status, 'unknown error') + +def __init__(self, status): +self.status = status + +def _handle_status(status): +if (status != nm.NOTMUCH_STATUS_SUCCESS): +raise Exception(status) + +class Database(object): +MODE_READ_ONLY = nm.NOTMUCH_DATABASE_MODE_READ_ONLY +MODE_READ_WRITE = nm.NOTMUCH_DATABASE_MODE_READ_WRITE + +def __init__(self, db): +if not db: +raise Exception("Failed to open database") +self.db = db + +@staticmethod +def create(path): +return Database(nm.notmuch_database_create(path)) + +@staticmethod +def open(path, mode): +return Database(nm.notmuch_database_open(path, mode)) + +def close(self): +nm.notmuch_database_close(self.db) + +def get_path(self): +return nm.notmuch_database_get_path(self.db) + +def set_timestamp(self, key, timestamp): +_handle_status(nm.notmuch_database_set_timestamp(self.db, key, timestamp)) + +def get_timestamp(self, key): +return datetime.fromtimestamp(nm.notmuch_database_get_timestamp(self.db, key)) + +def add_message(self, filename, message): +_handle_status(nm.notmuch_database_add_message(self.db, filename, message.message)) + +def find_message(self, message_id): +return Message(nm.notmuch_database_find_message(self.db, message_id)) + +def get_all_tags(self): +return Tags(nm.notmuch_database_get_all_tags(self.db)) + +def __destroy__(self): +self.close() + +class Query(object): +def __init__(self, db, query_string): +self.query = nm.notmuch_query_create(db, query_string) +if not self.query: # This might not work +raise "Failed to create query" + +def set_sort(self, sort): +nm.notmuch_query_set_sort(self.query, sort) + +def search_threads(self): +return Threads(nm.notmuch_query_search_threads(self.query)) + +def search_messages(self): +return Messages(nm.notmuch_query_search_messages(self.query)) + +def count_message(self): +return nm.notmuch_query_count_messages(self.query) + +def __destroy__(self): +nm.notmuch_query_destroy(self.query) + +class Tags(object): +def __init__(self, tags): +self.tags = tags + +def __iter__(self): +return self + +def next(self): +if not nm.notmuch_tags_has_more(self.tags): +raise StopIteration +else: +h = nm.notmuch_tags_get(self.tags) +nm.notmuch_tags_advance(self.tags) +return h + +def __destroy__(self): +nm.notmuch_messages_destroy(self
[notmuch] [PATCH] SWIG bindings
Hey all, Here's the latest version of my patch adding SWIG interface generation to notmuch. It has been rebased on the shared-library patches I sent over earlier this week, so you'll need those as well. Unfortunately, SWIG has effectively no support for exposing notmuch's C-style object-oriented interface in a reasonable manner. Therefore, I use SWIG to generate a set of low-level bindings which is then wrapped with a light-weight module properly exposing the functions in their intended class structure. This seems like a fairly good solution, given the wide variety of capabilities possessed by modern high-level languages. As you can see, it all works quite nicely, [2252 b...@ben-laptop swig(swig)] $ python Python 2.6.4 (r264:75706, Dec 7 2009, 18:43:55) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from notmuch import Database >>> db=Database.open('/home/ben/.mail', Database.MODE_READ_ONLY) >>> for tag in db.get_all_tags(): print tag ... attachment inbox unread Let me know what you think. Thanks! - Ben ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Mail::Notmuch Perl wrapper
Excerpts from Simon Cozens's message of Mon Jan 25 13:17:16 -0500 2010: > (Resent with compressed patch) > > Hi there, > Here's a Perl library, Mail::Notmuch, which wraps the notmuch library. > I've attached it because it's pretty huge. It requires you to > build notmuch as a shared library - there's a patch on the mailing list > earlier which does that. I know practically nothing about writing Perl bindings, but it seems like this might be work better left to a bindings generator. I currently have a patch which enables binding generation through SWIG. It works well, although SWIG unfortunately doesn't have very good support for exposing object-oriented C interfaces like notmuchs' through the target language's type system. Nonetheless, this is easily done by a wrapper library to prettify the binding (which in the case of Python took very little time). I sent the patchset out a few weeks ago, but I'll rebase it on top of my shared-library branch and send it out again tonight. Cheers, - Ben ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] proposal for more streamlined mail flow in notmuch
Excerpts from Jameson Rollins's message of Sat Jan 16 15:49:55 -0500 2010: > To fascilitate this, two new functions should be implemented at the > notmuch CLI level, so that things don't get fractured by different > mail reader implementations: > > notmuch purge > > delete all 'delete' tagged messages, and/or execute "delete hooks" > > notmuch archive > > remove 'inbox' flags from messages and move messages an archive > maildir, and/or execute "archive hooks" > I'm a little suspicious of this. Again, it seems like there is nothing whatsoever preventing one from implementing this at a higher-level than notmuch, which I believe should be little more than a thin-wrapper around Xapian unless more functionality is absolutely necessary. I don't understand why these sorts of proposals keep coming up when the above, for example, could be easily implemented in a pair of ten line shell scripts. Am I missing something? In my opinion, the "fractured implementations" argument is mislead at best. We currently have exactly two (in the case of vim, somewhat) usable UIs. It's not difficult to see to it that they obey similar conventions. If another UI pops up that fails to obey your favorite convention, feel free to patch it. IMHO, it is not the place of notmuch to force semantics onto the user (in fact, it tries to avoid this wherever possible). - Ben ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Mail::Notmuch Perl wrapper
(Resent with compressed patch) Hi there, Here's a Perl library, Mail::Notmuch, which wraps the notmuch library. I've attached it because it's pretty huge. It requires you to build notmuch as a shared library - there's a patch on the mailing list earlier which does that. Notmuch else to say, really. Enjoy. Simon -- next part -- A non-text attachment was scrubbed... Name: Mail-Notmuch-perl.patch.gz Type: application/x-gzip Size: 55832 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/9ee79950/attachment-0001.bin>
[notmuch] tag dir proposal [was: Re: Git as notmuch object store]
On Mon, 25 Jan 2010 11:22:47 -0500 (EST), Mike Kelly wrote: > Similarly, provide a mechanism for correlating the folder name with > some set of tags, and change those tags as messages are moved around. > > For example, I might have: > > ~/.notmuch-config: > > [database] > path=/home/pioto/mail > ... > [tags] > pioto at pioto.org/INBOX.ListMail.notmuch = notmuch > > So, a 'tags' section, where each key is the folder name, relative to the > db path, and the value is one or more tag names I think this idea is a really good one and I would like to pursue it as a tangent thread here. I was going to propose something very similar to this. I think it's a very flexible idea that would help in a lot of ways. For instance, notmuch emacs (and emacs message-mode) is currently not good at handling sent mail. At the moment mail is just Bcc'd to yourself. However, this means that these sent messages end up back in your inbox with 'inbox' and 'unread' tags which then need to be removed so that the sent message is archived. If one could configure notmuch such that only new mail in an inbox directory would be tagged with 'inbox' and 'unread', and manage to coax emacs to fcc directly into an archive, then these sent messages would not have the problematic 'inbox' and 'unread' tags. Even better, then sent mail could be fcc'd to a sent mail directory would could then be configured to automatically get a 'sent' tag. Notmuch emacs also currently does not handle message drafts, which makes it very difficult to resume messages that were postponed from a previous session. If notmuch could be configured to tag messages in the message-mode "message-auto-save-directory" with a 'draft' tag, then it would greatly facilitate finding draft messages. It would also be sweet if this could remove tags as well (maybe be prepending '-' or '+' to the tag specification. For example, I can imagine implementing the above examples like this: [database] path=/home/jrollins/.mail [tags] inbox = +inbox,+unread sent = +sent drafts = +draft archive = -inbox I think we should definitely implement something like this. It would make things a lot more flexible. Notmuch could be configured to not tag any messages by default (which would make a lot of people using notmuch for other backends happier) and then notmuch setup could could provide an example tags stanza that would tag new messages with 'inbox' and 'unread' (maybe with a wildcard that would replicate the current behavior): [tags] * = +inbox,+unread I would love to see this. Hopefully we can rally some more support for this idea. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/4b013ce3/attachment.pgp>
Re: [notmuch] Git feature branch
> I think it would make sense to move the mainline to git.debian.org > for now, or another place where everyone can easily get an account. > As alternatives I propose repo.or.cz. I'd prefer to stay away from > commercial services like Github. Any of those sounds fine to me, really. No preferences. I have little experience with git, so I can't comment on the hierarchy of branches. Cherry-picking all those patches "up" does sound like it needs some dedicated attention though... > Sebastian, would it make sense to migrate your work into a 'pu' > branch? Not sure. My feature branch is basically just the branch I use and I have ecclectically been adding patches that seemed useful to me and ignored others I was not so sure about. I have no clue if Carl would want those features/patches in master, but if yes he is free to pull it into an unstable branch or however we want to name it. > What patch tracking workflow should we adopt? Keep sending patches > to the mailing list and have a few people who integrate them into > master or pu (or even maint/next), as appropriate? Use the Debian > bug tracker? Use Sebastian's Roundup instance? Set up a patch queue > manager on notmuchmail.org? Use patchwork [1]? Personally, I like how patchwork complements mailing lists for collecting patches, so a patchwork-supported mailing list would work well for me. I have less experience with the Debian bug tracker, not sure if a 'downstream' project can simply use it (but you would know that :-)). I wouldn't mind to have my roundup used, but it should probably be on some more officially-looking URL, rather than on a random personal webspace, shouldn't it? All I know is that there is a great momentum behind notmuch at the moment, and it would be great to not lose that. Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] Git feature branch
> I think it would make sense to move the mainline to git.debian.org > for now, or another place where everyone can easily get an account. > As alternatives I propose repo.or.cz. I'd prefer to stay away from > commercial services like Github. Any of those sounds fine to me, really. No preferences. I have little experience with git, so I can't comment on the hierarchy of branches. Cherry-picking all those patches "up" does sound like it needs some dedicated attention though... > Sebastian, would it make sense to migrate your work into a 'pu' > branch? Not sure. My feature branch is basically just the branch I use and I have ecclectically been adding patches that seemed useful to me and ignored others I was not so sure about. I have no clue if Carl would want those features/patches in master, but if yes he is free to pull it into an unstable branch or however we want to name it. > What patch tracking workflow should we adopt? Keep sending patches > to the mailing list and have a few people who integrate them into > master or pu (or even maint/next), as appropriate? Use the Debian > bug tracker? Use Sebastian's Roundup instance? Set up a patch queue > manager on notmuchmail.org? Use patchwork [1]? Personally, I like how patchwork complements mailing lists for collecting patches, so a patchwork-supported mailing list would work well for me. I have less experience with the Debian bug tracker, not sure if a 'downstream' project can simply use it (but you would know that :-)). I wouldn't mind to have my roundup used, but it should probably be on some more officially-looking URL, rather than on a random personal webspace, shouldn't it? All I know is that there is a great momentum behind notmuch at the moment, and it would be great to not lose that. Sebastian
[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft wrote: > I think we all kinda agreed that the Maildir flags should not be > used by notmuch and that things like Sebastian's notmuchsync should > be used if people wanted flags represented in Maildir filenames. While notmuchsync fullfils my needs, it is a kludge. It needs to call "notmuch" for each mail where a MailDir flag has changed (which can be quite often on an initial run, where most mails are likely to be read), this can take a long, long time. It would makes sense IMHO to at least pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1]. Or - at the very least - not to set the "unread" flag by default. Sebastian [1] pioto's noarg-count branch (http://git.pioto.org/gitweb/notmuch.git Announced in mail id:20100121204201.1C82764A0E at aether.pioto.org)
[notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword) (final mail)
Sorry, very last mail from me on this issue. I squashed the patches into 3 distinct ones, and inherited a tree from cworth's master tree for easier pulling (rather than basing it off my all-features branch). This branch is here: http://github.com/spaetz/notmuch-all-feature/commits/dateparser2 (git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser2')
[notmuch] [PATCH] Mail::Notmuch Perl wrapper
On Mon, 25 Jan 2010 18:17:16 +, Simon Cozens wrote: > (Resent with compressed patch) > > Hi there, > Here's a Perl library, Mail::Notmuch, which wraps the notmuch library. > I've attached it because it's pretty huge. It requires you to > build notmuch as a shared library - there's a patch on the mailing list > earlier which does that. > Notmuch else to say, really. Enjoy. Hey, I was working on that! :p But, thanks. I'll probably be giving you more thorough feedback soon. -- Mike Kelly
[notmuch] tag dir proposal [was: Re: Git as notmuch object store]
On Mon, 25 Jan 2010 11:22:47 -0500 (EST), Mike Kelly wrote: > Similarly, provide a mechanism for correlating the folder name with > some set of tags, and change those tags as messages are moved around. > > For example, I might have: > > ~/.notmuch-config: > > [database] > path=/home/pioto/mail > ... > [tags] > pi...@pioto.org/INBOX.ListMail.notmuch = notmuch > > So, a 'tags' section, where each key is the folder name, relative to the > db path, and the value is one or more tag names I think this idea is a really good one and I would like to pursue it as a tangent thread here. I was going to propose something very similar to this. I think it's a very flexible idea that would help in a lot of ways. For instance, notmuch emacs (and emacs message-mode) is currently not good at handling sent mail. At the moment mail is just Bcc'd to yourself. However, this means that these sent messages end up back in your inbox with 'inbox' and 'unread' tags which then need to be removed so that the sent message is archived. If one could configure notmuch such that only new mail in an inbox directory would be tagged with 'inbox' and 'unread', and manage to coax emacs to fcc directly into an archive, then these sent messages would not have the problematic 'inbox' and 'unread' tags. Even better, then sent mail could be fcc'd to a sent mail directory would could then be configured to automatically get a 'sent' tag. Notmuch emacs also currently does not handle message drafts, which makes it very difficult to resume messages that were postponed from a previous session. If notmuch could be configured to tag messages in the message-mode "message-auto-save-directory" with a 'draft' tag, then it would greatly facilitate finding draft messages. It would also be sweet if this could remove tags as well (maybe be prepending '-' or '+' to the tag specification. For example, I can imagine implementing the above examples like this: [database] path=/home/jrollins/.mail [tags] inbox = +inbox,+unread sent = +sent drafts = +draft archive = -inbox I think we should definitely implement something like this. It would make things a lot more flexible. Notmuch could be configured to not tag any messages by default (which would make a lot of people using notmuch for other backends happier) and then notmuch setup could could provide an example tags stanza that would tag new messages with 'inbox' and 'unread' (maybe with a wildcard that would replicate the current behavior): [tags] * = +inbox,+unread I would love to see this. Hopefully we can rally some more support for this idea. jamie. pgpbXdzAXxgFn.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
also sprach Asheesh Laroia [2010.01.21.1928 +1300]: > >I suppose that I never actually considered merges on the IMAP > >server side, but obviously the IMAP server has to work off a clone, > >and that means it needs to merge. > > It's not "merge" that's unsafe; that just builds a tree in the git > index (assuming no conflicts). It's the ensuing process of git > writing a tree to the filesystem that is problematic. There is no way to make that atomic, I am afraid. As you say. > I could probably actually write a wrapper that locks the Maildir > while git is operating. It would probably be specific to each IMAP > server. Ouch! I'd really rather not go there. > Note that this mean git is fundamentally incompatible with > Maildir, not just IMAP servers. We had an idea about using Git to replace IMAP altogether, along with making notmuch use a bare Git repository as object store. The idea is that notmuch uses low-level Git commands to access the .git repository (from which you can still checkout a tree tying the blobs into a Maildir). The benefit would be compression, lower inode count (due to packs), and backups using clones/merges. You could either have the MDA write to a Git repo on the server side and use git packs to download mail to a local clone, or one could have e.g. offlineimap grow a Git storage backend. The interface to notmuch would be the same. If we used this, all the rename and delete code would be refactored into Git and could be removed from notmuch. In addition, notmuch could actually use Git tree objects to represent the results of searches, and you could checkout these trees. However, deleting messages from search results would not have any effect on the message or its existence in other search results, much like what happens with mairix nowadays. I think we all kinda agreed that the Maildir flags should not be used by notmuch and that things like Sebastian's notmuchsync should be used if people wanted flags represented in Maildir filenames. Instead of a Maildir checkout, notmuch could provide an interface to browse the store contents in a way that could make it accessible to mutt. The argument is that with 'notmuch {ls,cat,rm,?}', a mutt backend could be trivially written. I am not sure about that, but it's worth a try. But there are still good reasons why you'd want to have IMAP capability too, e.g. Webmail. Given the atomicity problems that come from Git, maybe an IMAP server reading from the Git store would make sense. However, this all sounds like a lot of NIH and reinvention. It's a bit like the marriage between the hypothetical Maildir2 and Git, which is definitely worth pursuing. Before we embark on any of this, however, we'd need to define the way in which Git stores mail. Stewart, you've worked most on this so far. Would you like to share your thoughts? -- martin | http://madduck.net/ | http://two.sentenc.es/ "reife des mannes, das ist es, den ernst wiedergefunden zu haben, den man hatte als kind beim spiel." -- friedrich nietzsche spamtraps: madduck.bogus at madduck.net -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature (see http://martin-krafft.net/gpg/) URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/6a70e84f/attachment.pgp>
Re: [notmuch] Git feature branch
also sprach Carl Worth [2010.01.23.1010 +1300]: > Anyway, I'll be on vacation for the next few days, so will likely > not have much, (likely have not much?), time for patch merging. > > But I *am* anxious to get back to the backlog. And in the > meantime, I really appreciate others merging and sharing patches. I discussed this with Carl at LCA a bit and ideally we should come up with a way to relieve Carl of the bottleneck burden (obviously without stealing away his dictator hat ;) I think it would make sense to move the mainline to git.debian.org for now, or another place where everyone can easily get an account. As alternatives I propose repo.or.cz. I'd prefer to stay away from commercial services like Github. Once we're there, how about copying the branch structure from Git development[0]: maint/— the stable release master/ — the stablising head next/ — testing branch pu/ — patch integration branch (proposed updates) Each of the four branches is usually a direct descendant of the one above it. Conceptually, the feature enters at an unstable branch (usually 'next' or 'pu'), and "graduates" to 'master' for the next release once it is considered stable enough. 0. http://repo.or.cz/w/git.git/blob/HEAD:/Documentation/gitworkflows.txt Sebastian, would it make sense to migrate your work into a 'pu' branch? What patch tracking workflow should we adopt? Keep sending patches to the mailing list and have a few people who integrate them into master or pu (or even maint/next), as appropriate? Use the Debian bug tracker? Use Sebastian's Roundup instance? Set up a patch queue manager on notmuchmail.org? Use patchwork [1]? Cheers, -- martin | http://madduck.net/ | http://two.sentenc.es/ "in all unimportant matters, style, not sincerity, is the essential. in all important matters, style, not sincerity, is the essential." -- oscar wilde spamtraps: madduck.bo...@madduck.net digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword)
The patch previously sent can be considered my final attempt to a nicer date parser. In addition to some testing, I have updated the documentation to reflect the new syntax, and I have also added the keyword 'now' as a possibility. So 'date:2002..now' (all mails from 2002 until now) or 'date:now..31' (all mails that McFly sent to you between now and the 31st of this month) should all work. Note that this is not a sincere: 'since' replacement, as 2001..now won't find mails with timestamps in the future. The relevant 4 patches are in my git tree, git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser': 59e9b56 remove superfluous debug statements from date parsing 4bdb0b0 allow 'now' as keyword for now cf6c500 Adapt documentation to new date: syntax d8d3d0b Make the date parser nicer. This is v3 and considered to be final (but the documentation). Thanks, Sebastian
[notmuch] debs of xapian upstream
Hey, folks. I'm being killed by the slowness of {,un}tagging in notmuch on Debian, which, as I understand it, comes from a xapian bug that has been fixed upstream (all of which has been discussed quite a bit on this list). A request for an upload of a new xapian verion that fixes the bug has been made [0], but no new upload has happened yet. I was wondering if anyone on this list has managed to make a working debian package out of any xapian version that fixes this bug. Upstream included a debian dir, and I have been trying to build from that but have had no success (my attempts are also outlined in [0]). If anyone out there has managed to build a deb of xapian-core I would *really* appreciate some pointers as to how you got it working. In fact, if any one actually has an x86 deb I would gladly just take that as well. jamie. [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=564830 -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100125/71216e90/attachment-0001.pgp>
Re: [notmuch] proposal for more streamlined mail flow in notmuch
also sprach Jameson Rollins [2010.01.17.0949 +1300]: > I would like to put forth here a proposal for a couple of changes > to notmuch that I believe will considerably streamline message > handling and new message flow through notmuch. Notmuch is still > new and I believe it hasn't quite figured out the best way to > handle message flow in the new mail handling paradigm that it is > working. This is a proposal to fix that. > > I believe that most people are syncing their mail from remote IMAP or > POP servers to local maildirs (via offlineimap for instance), and that > most people's IMAP servers have limited storage capacity. This > practically means that people can not keep all of their mail in a > single directory. However, notmuch currently basically assumes that > this is what is happening. In order to keep synced maildirs from > growing out of hand, messages need to be either deleted or moved out > of the "inbox" where they initially show up. We should keep this in mind when designing an object store, whether or not that's based on Git. If we use Git, then a local branch is all we really need to support this. \o/ -- martin | http://madduck.net/ | http://two.sentenc.es/ above all, we should not wish to divest our existence of its rich ambiguity. --friedrich nietzsche spamtraps: madduck.bo...@madduck.net digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Make the date parser nicer. This is v3 and considered to be final (but the documentation).
Currently we have to enter mail dates as timestamps. This approach does 2 things: 1) it requires the prefix 'date:' 2) it allows dates to be specified in a flexible way. So a notmuch show date:2005..2006-05-12 will find all mails from 2005-01-01 until 2006-05-12. Possible time formats: -MM-DD, -MM (from/through that month) , (from/through that year), MM-DD (month-day in current year), DD (day in current month). Signed-off-by: Sebastian Spaeth --- lib/database.cc | 80 ++- 1 files changed, 79 insertions(+), 1 deletions(-) diff --git a/lib/database.cc b/lib/database.cc index 5b12320..da2fda8 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -494,6 +494,84 @@ _notmuch_database_ensure_writable (notmuch_database_t *notmuch) return NOTMUCH_STATUS_SUCCESS; } +struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor { +MaildateValueRangeProcessor() {} + + time_t + parsedate(std::string &str, bool early) { +/* Parse the date to a 'time_t', return NULL on error*/ +/* possible time formats: -MM-DD, -MM, , */ +/* MM-DD (current month), DD (day in current month). */ +/* Uses start of time unit when 'early', end otherwise, e.g. */ +/* 2001:=2001-01-01:00:00:00 when 'early' or 2001-12-31:23:59:59 */ +struct tm *timeinfo; +time_t timet; +int year = 0, month = 0, day = 0; + +if (str.size() == 2) { + /* We got just current day in month, parse & remove it */ + day = atoi(str.c_str()); + str.erase(0,2); +} + +if (str.size() == 4 or str.size() > 5) { + /* expect a year, parse & remove it */ + year = atoi(str.c_str()); + str.erase(0,5); +} + +/* parse & remove month if there is sth left in the string */ +month = atoi(str.c_str()); +str.erase(0,3); + +/* Parse day if we have one left */ +if (str.size()) + day = atoi(str.c_str()); + +if (year == 0 && month == 0 && day == 0) + // no expected time format + return -1 ; + +timet = time(NULL);/* init timeinfo with current time */ +timeinfo = gmtime(&timet); +/* add timeunit if !early (1 second too much, which we deduct later */ +if (!early) { + if (year && !month)++year; /* only year given */ + if (year && month && !day) ++month; /* year & month given */ +} +if (year) timeinfo -> tm_year = year - 1900; +if (month) timeinfo -> tm_mon = month - 1; +if (day) timeinfo -> tm_mday = (early ? day : ++day); +else timeinfo -> tm_mday = 1; + +timeinfo -> tm_hour = 0; +timeinfo -> tm_min = 0; +timeinfo -> tm_sec = (early ? 0 : -1); /* -1 sec if !early */ +timet = mktime(timeinfo); + +return timet; + } + +Xapian::valueno operator()(std::string &begin, std::string &end) { + time_t begintime, endtime; + + if (begin.substr(0, 5) != "date:") +return Xapian::BAD_VALUENO; + begin.erase(0, 5); + + begintime = parsedate(begin, true); + endtime = parsedate(end, false); + if ((begintime == -1) || (endtime == -1)) + // parsedate failed, no valid time format + return Xapian::BAD_VALUENO; + + begin.assign(Xapian::sortable_serialise(begintime)); + end.assign(Xapian::sortable_serialise(endtime)); + + return NOTMUCH_VALUE_TIMESTAMP; +} +}; + notmuch_database_t * notmuch_database_open (const char *path, notmuch_database_mode_t mode) @@ -570,8 +648,7 @@ notmuch_database_open (const char *path, notmuch->query_parser = new Xapian::QueryParser; notmuch->term_gen = new Xapian::TermGenerator; notmuch->term_gen->set_stemmer (Xapian::Stem ("english")); - notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_TIMESTAMP); + notmuch->value_range_processor = new MaildateValueRangeProcessor(); notmuch->query_parser->set_default_op (Xapian::Query::OP_AND); notmuch->query_parser->set_database (*notmuch->xapian_db); -- 1.6.3.3
Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
also sprach Sebastian Spaeth [2010.01.26.0249 +1300]: > While notmuchsync fullfils my needs, it is a kludge. It needs to > call "notmuch" for each mail where a MailDir flag has changed > (which can be quite often on an initial run, where most mails are > likely to be read), this can take a long, long time. It would > makes sense IMHO to at least pick pioto's "don't set unread if 'S' > flag is set" on notmuch new[1]. I am sure this could be implemented with libnotmuch if it proves to be useful. -- martin | http://madduck.net/ | http://two.sentenc.es/ "it isn't pollution that's harming the environment. it's the impurities in our air and water that are doing it." - dan quayle spamtraps: madduck.bo...@madduck.net digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
On Mon, 25 Jan 2010 14:49:00 +0100, "Sebastian Spaeth" wrote: > > On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft > wrote: > > I think we all kinda agreed that the Maildir flags should not be > > used by notmuch and that things like Sebastian's notmuchsync should > > be used if people wanted flags represented in Maildir filenames. > > While notmuchsync fullfils my needs, it is a kludge. It needs to call > "notmuch" for each mail where a MailDir flag has changed (which can be > quite often on an initial run, where most mails are likely to be read), > this can take a long, long time. It would makes sense IMHO to at least > pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1]. notmuchsync, as currently implemented, suffers from major performance issues, in my opinion. It's a useful short term workaround, but not a good long term solution. But, I personally will always be using both notmuch and some other IMAP client (my phone). I want the two to remain in sync easily enough. notmuch is already much more robust with respect to that than sup, I think (in terms of handling renames without barfing, etc). At the very least, I want `notmuch new` to be able to: If it sees a rename that involves changing maildir flags, alter the related tags as necessary. Similarly, provide a mechanism for correlating the folder name with some set of tags, and change those tags as messages are moved around. For example, I might have: ~/.notmuch-config: [database] path=/home/pioto/mail ... [tags] pioto at pioto.org/INBOX.ListMail.notmuch = notmuch So, a 'tags' section, where each key is the folder name, relative to the db path, and the value is one or more tag names This means that I could relabel a message in gmail, for example, and have the changes apply to notmuch at my next offlineimap run. And, it means that my existing procmail rules will still be useful both to notmuch, and to my phone, for the purpose of categorizing things. I agree that all this should be optional. But, since it is likely the behavior most people would expect, I think it should be the default. PS. You mean the 'new-unread' branch, not the 'noarg-count' branch, from my repo. -- Mike Kelly
Re: [notmuch] [PATCH] Mail::Notmuch Perl wrapper
On Mon, 25 Jan 2010 18:17:16 +, Simon Cozens wrote: > (Resent with compressed patch) > > Hi there, > Here's a Perl library, Mail::Notmuch, which wraps the notmuch library. > I've attached it because it's pretty huge. It requires you to > build notmuch as a shared library - there's a patch on the mailing list > earlier which does that. > Notmuch else to say, really. Enjoy. Hey, I was working on that! :p But, thanks. I'll probably be giving you more thorough feedback soon. -- Mike Kelly ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] debs of xapian upstream
Hey, folks. I'm being killed by the slowness of {,un}tagging in notmuch on Debian, which, as I understand it, comes from a xapian bug that has been fixed upstream (all of which has been discussed quite a bit on this list). A request for an upload of a new xapian verion that fixes the bug has been made [0], but no new upload has happened yet. I was wondering if anyone on this list has managed to make a working debian package out of any xapian version that fixes this bug. Upstream included a debian dir, and I have been trying to build from that but have had no success (my attempts are also outlined in [0]). If anyone out there has managed to build a deb of xapian-core I would *really* appreciate some pointers as to how you got it working. In fact, if any one actually has an x86 deb I would gladly just take that as well. jamie. [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=564830 pgp3ah5PiqqYD.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
On Mon, 25 Jan 2010 14:49:00 +0100, "Sebastian Spaeth" wrote: > > On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft > wrote: > > I think we all kinda agreed that the Maildir flags should not be > > used by notmuch and that things like Sebastian's notmuchsync should > > be used if people wanted flags represented in Maildir filenames. > > While notmuchsync fullfils my needs, it is a kludge. It needs to call > "notmuch" for each mail where a MailDir flag has changed (which can be > quite often on an initial run, where most mails are likely to be read), > this can take a long, long time. It would makes sense IMHO to at least > pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1]. notmuchsync, as currently implemented, suffers from major performance issues, in my opinion. It's a useful short term workaround, but not a good long term solution. But, I personally will always be using both notmuch and some other IMAP client (my phone). I want the two to remain in sync easily enough. notmuch is already much more robust with respect to that than sup, I think (in terms of handling renames without barfing, etc). At the very least, I want `notmuch new` to be able to: If it sees a rename that involves changing maildir flags, alter the related tags as necessary. Similarly, provide a mechanism for correlating the folder name with some set of tags, and change those tags as messages are moved around. For example, I might have: ~/.notmuch-config: [database] path=/home/pioto/mail ... [tags] pi...@pioto.org/INBOX.ListMail.notmuch = notmuch So, a 'tags' section, where each key is the folder name, relative to the db path, and the value is one or more tag names This means that I could relabel a message in gmail, for example, and have the changes apply to notmuch at my next offlineimap run. And, it means that my existing procmail rules will still be useful both to notmuch, and to my phone, for the purpose of categorizing things. I agree that all this should be optional. But, since it is likely the behavior most people would expect, I think it should be the default. PS. You mean the 'new-unread' branch, not the 'noarg-count' branch, from my repo. -- Mike Kelly ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
On Mon, 25 Jan 2010 13:46:59 +1300, martin f krafft wrote: > I think we all kinda agreed that the Maildir flags should not be > used by notmuch and that things like Sebastian's notmuchsync should > be used if people wanted flags represented in Maildir filenames. While notmuchsync fullfils my needs, it is a kludge. It needs to call "notmuch" for each mail where a MailDir flag has changed (which can be quite often on an initial run, where most mails are likely to be read), this can take a long, long time. It would makes sense IMHO to at least pick pioto's "don't set unread if 'S' flag is set" on notmuch new[1]. Or - at the very least - not to set the "unread" flag by default. Sebastian [1] pioto's noarg-count branch (http://git.pioto.org/gitweb/notmuch.git Announced in mail id:20100121204201.1c82764...@aether.pioto.org) ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword) (final mail)
Sorry, very last mail from me on this issue. I squashed the patches into 3 distinct ones, and inherited a tree from cworth's master tree for easier pulling (rather than basing it off my all-features branch). This branch is here: http://github.com/spaetz/notmuch-all-feature/commits/dateparser2 (git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser2') ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] [PATCH] Make the date parser nicer (v3 + 'now' keyword)
The patch previously sent can be considered my final attempt to a nicer date parser. In addition to some testing, I have updated the documentation to reflect the new syntax, and I have also added the keyword 'now' as a possibility. So 'date:2002..now' (all mails from 2002 until now) or 'date:now..31' (all mails that McFly sent to you between now and the 31st of this month) should all work. Note that this is not a sincere: 'since' replacement, as 2001..now won't find mails with timestamps in the future. The relevant 4 patches are in my git tree, git://github.com/spaetz/notmuch-all-feature.git then switch to branch 'dateparser': 59e9b56 remove superfluous debug statements from date parsing 4bdb0b0 allow 'now' as keyword for now cf6c500 Adapt documentation to new date: syntax d8d3d0b Make the date parser nicer. This is v3 and considered to be final (but the documentation). Thanks, Sebastian ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] [PATCH] Make the date parser nicer. This is v3 and considered to be final (but the documentation).
Currently we have to enter mail dates as timestamps. This approach does 2 things: 1) it requires the prefix 'date:' 2) it allows dates to be specified in a flexible way. So a notmuch show date:2005..2006-05-12 will find all mails from 2005-01-01 until 2006-05-12. Possible time formats: -MM-DD, -MM (from/through that month) , (from/through that year), MM-DD (month-day in current year), DD (day in current month). Signed-off-by: Sebastian Spaeth --- lib/database.cc | 80 ++- 1 files changed, 79 insertions(+), 1 deletions(-) diff --git a/lib/database.cc b/lib/database.cc index 5b12320..da2fda8 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -494,6 +494,84 @@ _notmuch_database_ensure_writable (notmuch_database_t *notmuch) return NOTMUCH_STATUS_SUCCESS; } +struct MaildateValueRangeProcessor : public Xapian::ValueRangeProcessor { +MaildateValueRangeProcessor() {} + + time_t + parsedate(std::string &str, bool early) { +/* Parse the date to a 'time_t', return NULL on error*/ +/* possible time formats: -MM-DD, -MM, , */ +/* MM-DD (current month), DD (day in current month). */ +/* Uses start of time unit when 'early', end otherwise, e.g. */ +/* 2001:=2001-01-01:00:00:00 when 'early' or 2001-12-31:23:59:59 */ +struct tm *timeinfo; +time_t timet; +int year = 0, month = 0, day = 0; + +if (str.size() == 2) { + /* We got just current day in month, parse & remove it */ + day = atoi(str.c_str()); + str.erase(0,2); +} + +if (str.size() == 4 or str.size() > 5) { + /* expect a year, parse & remove it */ + year = atoi(str.c_str()); + str.erase(0,5); +} + +/* parse & remove month if there is sth left in the string */ +month = atoi(str.c_str()); +str.erase(0,3); + +/* Parse day if we have one left */ +if (str.size()) + day = atoi(str.c_str()); + +if (year == 0 && month == 0 && day == 0) + // no expected time format + return -1 ; + +timet = time(NULL);/* init timeinfo with current time */ +timeinfo = gmtime(&timet); +/* add timeunit if !early (1 second too much, which we deduct later */ +if (!early) { + if (year && !month)++year; /* only year given */ + if (year && month && !day) ++month; /* year & month given */ +} +if (year) timeinfo -> tm_year = year - 1900; +if (month) timeinfo -> tm_mon = month - 1; +if (day) timeinfo -> tm_mday = (early ? day : ++day); +else timeinfo -> tm_mday = 1; + +timeinfo -> tm_hour = 0; +timeinfo -> tm_min = 0; +timeinfo -> tm_sec = (early ? 0 : -1); /* -1 sec if !early */ +timet = mktime(timeinfo); + +return timet; + } + +Xapian::valueno operator()(std::string &begin, std::string &end) { + time_t begintime, endtime; + + if (begin.substr(0, 5) != "date:") +return Xapian::BAD_VALUENO; + begin.erase(0, 5); + + begintime = parsedate(begin, true); + endtime = parsedate(end, false); + if ((begintime == -1) || (endtime == -1)) + // parsedate failed, no valid time format + return Xapian::BAD_VALUENO; + + begin.assign(Xapian::sortable_serialise(begintime)); + end.assign(Xapian::sortable_serialise(endtime)); + + return NOTMUCH_VALUE_TIMESTAMP; +} +}; + notmuch_database_t * notmuch_database_open (const char *path, notmuch_database_mode_t mode) @@ -570,8 +648,7 @@ notmuch_database_open (const char *path, notmuch->query_parser = new Xapian::QueryParser; notmuch->term_gen = new Xapian::TermGenerator; notmuch->term_gen->set_stemmer (Xapian::Stem ("english")); - notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_TIMESTAMP); + notmuch->value_range_processor = new MaildateValueRangeProcessor(); notmuch->query_parser->set_default_op (Xapian::Query::OP_AND); notmuch->query_parser->set_database (*notmuch->xapian_db); -- 1.6.3.3 ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[notmuch] Git as notmuch object store (was: Potential problem using Git for mail)
On Mon, 25 Jan 2010, martin f krafft wrote: > also sprach Asheesh Laroia [2010.01.21.1928 > +1300]: >>> I suppose that I never actually considered merges on the IMAP server >>> side, but obviously the IMAP server has to work off a clone, and that >>> means it needs to merge. >> >> It's not "merge" that's unsafe; that just builds a tree in the git >> index (assuming no conflicts). It's the ensuing process of git writing >> a tree to the filesystem that is problematic. > > There is no way to make that atomic, I am afraid. As you say. > >> I could probably actually write a wrapper that locks the Maildir while >> git is operating. It would probably be specific to each IMAP server. > > Ouch! I'd really rather not go there. You say "Ouch" but you should know Dovecot *already* does this. I don't mind interoperating with that. See http://wiki.dovecot.org/MailboxFormat/Maildir, section "Issues with the specification", subsection "Locking". I term this the famous readdir() race. Without this lock, Maildir is fundamentally incompatible with IMAP -- one Maildir-using process modifying message flags could make a different Maildir-using process think said message is actually deleted. In the case of temporary disappearing mails in Mutt locally, that's not the end of the world. For IMAP, it will make the IMAP daemon (one of the Maildir-using processes) send a note to IMAP clients saying that the message has been deleted and expunged. >> Note that this mean git is fundamentally incompatible with Maildir, not >> just IMAP servers. > > We had an idea about using Git to replace IMAP altogether, along with > making notmuch use a bare Git repository as object store. The idea is > that notmuch uses low-level Git commands to access the .git repository > (from which you can still checkout a tree tying the blobs into a > Maildir). The benefit would be compression, lower inode count (due to > packs), and backups using clones/merges. Sure, that makes sense to me. > You could either have the MDA write to a Git repo on the server side and > use git packs to download mail to a local clone, or one could have e.g. > offlineimap grow a Git storage backend. The interface to notmuch would > be the same. Yeah, I generally like this. > If we used this, all the rename and delete code would be refactored into > Git and could be removed from notmuch. In addition, notmuch could > actually use Git tree objects to represent the results of searches, and > you could checkout these trees. However, deleting messages from search > results would not have any effect on the message or its existence in > other search results, much like what happens with mairix nowadays. That's okay with me. > I think we all kinda agreed that the Maildir flags should not be used by > notmuch and that things like Sebastian's notmuchsync should be used if > people wanted flags represented in Maildir filenames. Aww, I like Maildir flags, but if there's a sync tool, I'm fine with that. > Instead of a Maildir checkout, notmuch could provide an interface to > browse the store contents in a way that could make it accessible to > mutt. The argument is that with 'notmuch {ls,cat,rm,?}', a mutt backend > could be trivially written. I am not sure about that, but it's worth a > try. Sure. > But there are still good reasons why you'd want to have IMAP capability > too, e.g. Webmail. Given the atomicity problems that come from Git, > maybe an IMAP server reading from the Git store would make sense. It wouldn't be too hard to write a FUSE filesystem that presented an interface to a Git repository that didn't allow the contents of files to be modified. Then Dovecot could think it's interacting with the filesystem. > However, this all sounds like a lot of NIH and reinvention. It's > a bit like the marriage between the hypothetical Maildir2 and Git, > which is definitely worth pursuing. Before we embark on any of this, > however, we'd need to define the way in which Git stores mail. Sure. If it were me, I'd just say, "For phase 1 of notmuch, just have git store Maildir spools." When you need a filesystem interface for e.g. Dovecot, have a FUSE wrapper. See how far that can take you, and then see if version 2 is necessary. (-: > Stewart, you've worked most on this so far. Would you like to share your > thoughts? I'll listen, too. Just don't fall into the trap of thinking Maildir is compatible with IMAP. It's not, because as I understand things, the filesystem doesn't guarantee that you can actually iterate across a directory's files if another process is modifying the list of files. I'm not sure, but maybe it's safe if you refuse to ever modify a message's flags in the filename. Anyway, as I see it, further hacks that aren't much worse than Dovecot's should be considered okay, unless you have a more elegant design up your sleeve. If I'm slightly wrong about something, try to give me the benefit of doubt. It's past midni