[announce] Bower 0.4
On Tue, 14 Aug 2012 07:25:00 +0200, David Froger wrote: > > So I think I understand my problem. The reason is that in put in my > .config/bower/bower.conf: > > notmuch_deliver=/bin/true > > To solve the problem of "duplicated sent email" with Gmail: > http://support.google.com/mail/bin/answer.py?hl=en=78892 > > But I didn't realize that notmuch_deliver was also used for postponed > messages. > > What is the good config? (I see that you're using gmail too :-) ...)? You can disable it by setting `post_sendmail' to empty. I don't notice duplicated emails because both gmail and notmuch suppress them, and it's more important that I see what I've sent out immediately. Peter
Alternative (raw) message store (i.e. instead of maildir)
On Tue, Aug 14, 2012 at 8:11 PM, Christophe-Marie Duquesne wrote: > one could complete this work with an > interface to couchdb for offlineimap *I meant for notmuch
Alternative (raw) message store (i.e. instead of maildir)
On Tue, Aug 14, 2012 at 7:05 PM, Ciprian Dorin Craciun wrote: > I proposed -- better said queried if possible or at least wanted > -- to have an internal interface (SPI) that any mail store would have > to implement in order to be indexed and used by notmuch. I guess the > interface would be quite lightweight, and would need just the > following: > * open store; > * create a cursor iterating through all the emails, yielding only the > keys; > * read the envelope (as a byte blob) of a particular key; (used > only for displaying thread lists, etc.;) > * read the body (as a byte blob) of a particular key; > * maybe create a cursor iterating over all those emails that have > changed since a particular timestamp; Someone wrote a fork of offlineimap to store mail in couchdb [1]. The same couchdb can be mounted with fuse as a maildir [2] for mutt. According to the author [3], the fuse interface is read only. Assuming your proposal was implemented, one could complete this work with an interface to couchdb for offlineimap and get all the features previously requested. [1]: https://github.com/theodoreb/offlineimap [2]: https://github.com/theodoreb/couchdb-maildir-fuse [3]: http://theodoreb.net/resume
Re: ‘class Xapian::Database’ has no member named ‘close’
Quoting Jani Nikula (2012-08-14 14:57:13) > > Is close() really required though? The comment justifies it saying > > that: > > > > Many Xapian objects hold references to the database, so merely > > deleting the database may not suffice to close it. Thus, we > > explicitly close it here. > > > > but wouldn't the database get closed when the last reference gets > > deleted anyway? Yes, it is. > IIRC this was related to how closing is handled in the python > bindings. Well this issue isn't in anyway specific to the python bindings, but in c you can determine exactly whether a database has been closed because this only happens if you explicitly destroy the database object. In any garbage collected language you have to wait for the object to be collected and this is usually not easy to detect. So we decided to explicitly close the database and to split ..database_destroy in ..database_close and ..database_destroy later on. Justus
‘class Xapian::Database’ has no member named ‘close’
On Tue, 14 Aug 2012, Michal Nazarewicz wrote: > Yeah, it seems that INSTALL claims ?Notmuch will work best with Xapian > 1.0.18 (or later) or Xapian 1.1.4 (or later)?, which doesn't seem to be > the case. > > On top of that, configure does not seem to even check the version > number. Someone(tm) should fix both of the above. > Is close() really required though? The comment justifies it saying > that: > > Many Xapian objects hold references to the database, so merely > deleting the database may not suffice to close it. Thus, we > explicitly close it here. > > but wouldn't the database get closed when the last reference gets > deleted anyway? IIRC this was related to how closing is handled in the python bindings. The close was introduced in the commit below, not too long ago. I just reverted that one when I had to use notmuch with ancient xapian (but the usual disclaimers apply). BR, Jani. commit cfc5f1059aa16753cba610c41601cacc97260e08 Author: Justus Winter <4winter at informatik.uni-hamburg.de> Date: Fri Mar 2 15:58:39 2012 +0100 Actually close the xapian database in notmuch_database_close Formerly the xapian database object was deleted and closed in its destructor once the object was garbage collected. Explicitly call close() so that the database and the associated lock is released immediately. The comment is a courtesy of Austin Clements. Signed-off-by: Justus Winter <4winter at informatik.uni-hamburg.de>
Alternative (raw) message store (i.e. instead of maildir)
Vladimir Marek writes: > Hi, > > I have objections against maildir too, but I tried to tackle it from > different perspective. Store the maildir in zip file and use fuse-zip to > manage it. It works sort of but it has two major disadvantages: huh... this is fairly interesting one of the downsides of a million odd files for mail is that filesystem dump and restore takes a *LOT* longer than if it's just giant files on disk. Combined with afuse (fuse automounter) this could be a pretty elegant solution to the problem of storing archival Maildirs. One large archival maildir here went from 6.5GB (du -sh on XFS) to a 2.3GB ZIP archive that will never, ever change. Think about the performance difference between creating 560,000 files for backup/restore versus copying a single 2.3GB file. > - fuse zip stores all changes in memory until unmounted > - fuse zip (and libzip for that matter) creates new temporary file when >updating archive, which takes considerable time when the archive is >very big. This isn't much of a hastle if you have maildir per time period and archive off. Maybe if you sync flags it may be... > Of course this solution would have some disadvantages too, but for me > the advantages would win. At the moment I'm not sure if I want to > continue working on that. Maybe if there would be more interested guys I'm *really* tempted to investigate making this work for archived mail. Of course, the list of mounted file systems could get insane depending on granularity I guess... -- Stewart Smith -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20120814/cf027a1b/attachment.pgp>
‘class Xapian::Database’ has no member named ‘close’
> Michal Nazarewicz writes: >> $ dpkg -l |grep xapian >> ii apt-xapian-index 0.25ubuntu2 maintenance tools for a Xapian index >> of Debi >> ii libxapian-dev 1.0.18-1 Development files for Xapian search >> engine l >> ii libxapian15 1.0.18-1 Search engine library David Bremner writes: > This might be a bug with the install documentation; it could be we > really need a later xapian to build. Yeah, it seems that INSTALL claims ?Notmuch will work best with Xapian 1.0.18 (or later) or Xapian 1.1.4 (or later)?, which doesn't seem to be the case. On top of that, configure does not seem to even check the version number. > From memory, maybe 1.0.20 has the "close" method. 1.0.18 is pretty > old; Debian squeeze has 1.2.3, and that must be almost 3 years old > now. Is close() really required though? The comment justifies it saying that: Many Xapian objects hold references to the database, so merely deleting the database may not suffice to close it. Thus, we explicitly close it here. but wouldn't the database get closed when the last reference gets deleted anyway? -- Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Micha? ?mina86? Nazarewicz(o o) ooo +--ooO--(_)--Ooo-- -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20120814/5bd3fd0d/attachment.pgp>
‘class Xapian::Database’ has no member named ‘close’
Michal Nazarewicz writes: > > $ dpkg -l |grep xapian > ii apt-xapian-index 0.25ubuntu2 maintenance tools for a Xapian index > of Debi > ii libxapian-dev 1.0.18-1 Development files for Xapian search > engine l > ii libxapian15 1.0.18-1 Search engine library Hi Michal; This might be a bug with the install documentation; it could be we really need a later xapian to build. From memory, maybe 1.0.20 has the "close" method. 1.0.18 is pretty old; Debian squeeze has 1.2.3, and that must be almost 3 years old now. d
[announce] Bower 0.4
> This is the command to save drafts: > > notmuch-deliver Drafts --tag=draft \ > --remove-tag=inbox --remove-tag=unread < message_file > > Does it work if you run it manually? Just make a dummy message_file > with Date:, From:, To:, Subject: headers and a body. The manual command works! So I think I understand my problem. The reason is that in put in my .config/bower/bower.conf: notmuch_deliver=/bin/true To solve the problem of "duplicated sent email" with Gmail: http://support.google.com/mail/bin/answer.py?hl=en=78892 But I didn't realize that notmuch_deliver was also used for postponed messages. What is the good config? (I see that you're using gmail too :-) ...)? thanks, David
[announce] Bower 0.4
On Sun, 12 Aug 2012 21:33:39 +0100, Patrick Totzke wrote: > Quoting David Froger (2012-08-12 16:53:43) > > > > 3. Would it be possible to have the feature 'kill a thread' like Sup > > > > have? (see > > > > http://sup.rubyforge.org/README.txt). > > > > > > I haven't needed such a feature so I don't have a clear idea how it > > > should work. There are some patches/ideas about muting threads using > > > notmuch generally around here. I guess the simplest thing to do is hide > > > a thread if it has a certain tag, e.g. "muted" as suggested by notmuch > > > TODO file, unless you search for it explicitly. > > I like this solution, I will use it (adding 'and not tag:mutted' at the > > end of my search). > > When I'm not interessed with a long discussion on a mailling list, I > > like to be able to 'kill' > > the thread so it does not reappear with every new mail. > > I'm not sure if this actually solves your problem, because > notmuch's search is message, not thread based: > If you search for threads that match "is:inbox and not is:muted", you will > see threads > that contain messages which are tagged with "inbox" and not with "muted". > This means if you just tag one (or all) msg in a thread "muted" and there is > a new response, > this thread will contain such a matching message again and show up in your > result list. Thanks for the clarification, it's very helpfull! > > To make this work, you need to set up your tagging script (that you run after > 'notmuch new') > so that it updates those new messages accordingly. If a msg belongs to a > thread that has > muted messages, also tag the new one muted. > I can pretty much recommend Justus' tagging script afew: > https://github.com/teythoon/afew > One of the filters that it supports out of the box is exactly the "kill > threads" function > you want. Thanks, this script looks very cool and I think I will replace mine with it (I should have visited (http://notmuchmail.org/initial_tagging/ ...). You're right, the filter I'm interessed is in in aftew/filters/KillThreadsFilter.py, and do this: for all email M that does not have tag 'killed': I = thread Id of the email M if there are emails that have thread id I and the 'killed' tag: apply 'killed' tag on email E Thanks again for all the help.
[Patch v2 1/6] hex-escape: (en|de)code strings to/from restricted character set
The character set is chosen to be suitable for pathnames, and the same
as that used by contrib/nmbug
[With additions by Jani Nikula]
---
util/Makefile.local |2 +-
util/hex-escape.c | 168 +++
util/hex-escape.h | 41 +
3 files changed, 210 insertions(+), 1 deletion(-)
create mode 100644 util/hex-escape.c
create mode 100644 util/hex-escape.h
diff --git a/util/Makefile.local b/util/Makefile.local
index c7cae61..3ca623e 100644
--- a/util/Makefile.local
+++ b/util/Makefile.local
@@ -3,7 +3,7 @@
dir := util
extra_cflags += -I$(srcdir)/$(dir)
-libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c
+libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c $(dir)/hex-escape.c
libutil_modules := $(libutil_c_srcs:.c=.o)
diff --git a/util/hex-escape.c b/util/hex-escape.c
new file mode 100644
index 000..d8905d0
--- /dev/null
+++ b/util/hex-escape.c
@@ -0,0 +1,168 @@
+/* hex-escape.c - Manage encoding and decoding of byte strings into path names
+ *
+ * Copyright (c) 2011 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner da...@tethera.net
+ */
+
+#include assert.h
+#include string.h
+#include talloc.h
+#include ctype.h
+#include error_util.h
+#include hex-escape.h
+
+static const size_t default_buf_size = 1024;
+
+static const char *output_charset =
+ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-_@=.:,;
+
+static const char escape_char = '%';
+
+static int
+is_output (char c)
+{
+return (strchr (output_charset, c) != NULL);
+}
+
+static int
+maybe_realloc (void *ctx, size_t needed, char **out, size_t *out_size)
+{
+if (*out_size needed) {
+
+ if (*out == NULL)
+ *out = talloc_size (ctx, needed);
+ else
+ *out = talloc_realloc (ctx, *out, char, needed);
+
+ if (*out == NULL)
+ return 0;
+
+ *out_size = needed;
+}
+return 1;
+}
+
+hex_status_t
+hex_encode (void *ctx, const char *in, char **out, size_t *out_size)
+{
+
+const unsigned char *p;
+char *q;
+
+size_t escape_count = 0;
+size_t len = 0;
+size_t needed;
+
+assert (ctx); assert (in); assert (out); assert (out_size);
+
+for (p = (unsigned char *) in; *p; p++) {
+ escape_count += (!is_output (*p));
+ len++;
+}
+
+needed = len + escape_count * 2 + 1;
+
+if (*out == NULL)
+ *out_size = 0;
+
+if (!maybe_realloc (ctx, needed, out, out_size))
+ return HEX_OUT_OF_MEMORY;
+
+q = *out;
+p = (unsigned char *) in;
+
+while (*p) {
+ if (is_output (*p)) {
+ *q++ = *p++;
+ } else {
+ sprintf (q, %%%02x, *p++);
+ q += 3;
+ }
+}
+
+*q = '\0';
+return HEX_SUCCESS;
+}
+
+/* Hex decode 'in' to 'out'.
+ *
+ * This must succeed for in == out to support hex_decode_inplace().
+ */
+static hex_status_t
+hex_decode_internal (const char *in, unsigned char *out)
+{
+char buf[3];
+
+while (*in) {
+ if (*in == escape_char) {
+ char *endp;
+
+ /* This also handles unexpected end-of-string. */
+ if (!isxdigit ((unsigned char) in[1]) ||
+ !isxdigit ((unsigned char) in[2]))
+ return HEX_SYNTAX_ERROR;
+
+ buf[0] = in[1];
+ buf[1] = in[2];
+ buf[2] = '\0';
+
+ *out = strtoul (buf, endp, 16);
+
+ if (endp != buf + 2)
+ return HEX_SYNTAX_ERROR;
+
+ in += 3;
+ out++;
+ } else {
+ *out++ = *in++;
+ }
+}
+
+*out = '\0';
+
+return HEX_SUCCESS;
+}
+
+hex_status_t
+hex_decode_inplace (char *s)
+{
+/* A decoded string is never longer than the encoded one, so it is
+ * safe to decode a string onto itself. */
+return hex_decode_internal (s, (unsigned char *) s);
+}
+
+hex_status_t
+hex_decode (void *ctx, const char *in, char **out, size_t * out_size)
+{
+const char *p;
+size_t escape_count = 0;
+size_t needed = 0;
+
+assert (ctx); assert (in); assert (out); assert (out_size);
+
+size_t len = strlen (in);
+
+for (p = in; *p; p++)
+ escape_count += (*p == escape_char);
+
+needed = len - escape_count * 2 + 1;
+
+if (!maybe_realloc (ctx, needed, out, out_size))
+ return HEX_OUT_OF_MEMORY;
+
+return
Random message corpus, v2
This obsoletes the series id:1344190393-22497-1-git-send-email-da...@tethera.net [Patch v2 1/6] hex-escape: (en|de)code strings to/from restricted [Patch v2 2/6] test/hex-xcode: new test binary [Patch v2 3/6] test/hex-escaping: new test for hex escaping routines In order to avoid the problem of trashing people's terminals, the new version hex escapes the files being compared. This brings in another three patches from the previous dump-restore series, but we need them anyway for the eventual new dump-restore format. [Patch v2 4/6] test: add database routines for testing. No changes since v1. [Patch v2 5/6] test: add generator for random stub messages - Simplified to have only two character classes. - Fix bug with stop - start as opposed to start - stop See id:87393x3jz1@qmul.ac.uk [Patch v2 6/6] test: add broken roundtrip test - use hexdump to encode non-ascii octets. See id:87zk6524f0@qmul.ac.uk - Both Mark and Jamie remarked on the use of 'test_expect_success' Basically there are two styles of running tests, git style as written here, or notmuch style using test_begin_subtest. The git style seems to be more robust against failure of intermediate commands: for example, if the hex-xcode binary is missing, the corresponding notmuch style test shows (wrongly) as fixed (see also id:1317317811-29540-1-git-send-email-tho...@schwinge.name for Thomas converting some notmuch style tests to git style). Obviously the notmuch style is more pleasant to edit. Maybe there is a way to make them more robust; something like set -e. Note that some failures still do not abort the test, because of a missing set -o pipefile. Setting this locally in a test seems definitely wrong, but maybe it should be set globally in test-lib.sh ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[Patch v2 5/6] test: add generator for random stub messages
Initial use case is testing dump and restore, so we only have
message-ids and tags.
The message ID's are nothing like RFC compliant, but it doesn't seem
any harder to roundtrip random UTF-8 strings than RFC-compliant ones.
Tags are UTF-8, even though notmuch is in principle more generous than
that.
---
test/.gitignore |1 +
test/Makefile.local |8 ++
test/basic |2 +-
test/random-corpus.c | 203 ++
4 files changed, 213 insertions(+), 1 deletion(-)
create mode 100644 test/random-corpus.c
diff --git a/test/.gitignore b/test/.gitignore
index be7ab5e..1eff7ce 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -4,4 +4,5 @@ smtp-dummy
symbol-test
arg-test
hex-xcode
+random-corpus
tmp.*
diff --git a/test/Makefile.local b/test/Makefile.local
index ab13814..501aa4a 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -16,6 +16,13 @@ $(dir)/arg-test: $(dir)/arg-test.o command-line-arguments.o
util/libutil.a
$(dir)/hex-xcode: $(dir)/hex-xcode.o command-line-arguments.o util/libutil.a
$(call quiet,CC) -I. $^ -o $@ -ltalloc
+random_corpus_deps = $(dir)/random-corpus.o $(dir)/database-test.o \
+ notmuch-config.o command-line-arguments.o \
+ lib/libnotmuch.a util/libutil.a
+
+$(dir)/random-corpus: $(random_corpus_deps)
+ $(call quiet,CC) $(CFLAGS_FINAL) $^ -o $@ $(CONFIGURE_LDFLAGS)
+
$(dir)/smtp-dummy: $(smtp_dummy_modules)
$(call quiet,CC) $^ -o $@
@@ -26,6 +33,7 @@ $(dir)/symbol-test: $(dir)/symbol-test.o
TEST_BINARIES=$(dir)/arg-test \
$(dir)/hex-xcode \
+ $(dir)/random-corpus \
$(dir)/smtp-dummy \
$(dir)/symbol-test
diff --git a/test/basic b/test/basic
index 3eebb37..d923d5a 100755
--- a/test/basic
+++ b/test/basic
@@ -54,7 +54,7 @@ test_begin_subtest 'Ensure that all available tests will be
run by notmuch-test'
eval $(sed -n -e '/^TESTS=$/,/^$/p' $TEST_DIRECTORY/notmuch-test)
tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
available=$(find $TEST_DIRECTORY -maxdepth 1 -type f -executable -printf
'%f\n' | \
-sed -r -e
/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|hex-escape)$/d
| \
+sed -r -e
/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|hex-escape|random-corpus)$/d
| \
sort)
test_expect_equal $tests_in_suite $available
diff --git a/test/random-corpus.c b/test/random-corpus.c
new file mode 100644
index 000..a318267
--- /dev/null
+++ b/test/random-corpus.c
@@ -0,0 +1,203 @@
+/*
+ * Generate a random corpus of stub messages.
+ *
+ * Initial use case is testing dump and restore, so we only have
+ * message-ids and tags.
+ *
+ * Generated message-id's and tags are intentionally nasty.
+ *
+ * Copyright (c) 2012 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner da...@tethera.net
+ */
+
+#include stdlib.h
+#include assert.h
+#include talloc.h
+#include string.h
+#include glib.h
+#include math.h
+
+#include notmuch-client.h
+#include command-line-arguments.h
+#include database-test.h
+
+/* Current largest UTF-32 value defined. Note that most of these will
+ * be printed as boxes in most fonts.
+ */
+
+#define GLYPH_MAX 0x10FFFE
+#define GLYPH_CLASSES 4
+
+static gunichar
+random_unichar ()
+{
+int start=1, stop=GLYPH_MAX;
+int class = random() % 2;
+
+/*
+ * Choose about half ascii as test characters, as ascii
+ * punctation and whitespace is the main cause of problems for
+ * the (old) restore parser
+*/
+switch (class) {
+case 0:
+ /* ascii */
+ start=0x01;
+ stop=0x7f;
+ break;
+case 1:
+ /* the rest of unicode */
+ start=0x80;
+ stop=GLYPH_MAX;
+}
+
+if (start == stop)
+ return start;
+else
+ return start + (random() % (stop - start + 1));
+}
+
+static char *
+random_utf8_string (void *ctx, size_t char_count)
+{
+
+gchar *buf = NULL;
+size_t buf_size = 0;
+
+size_t offset = 0;
+
+size_t i;
+
+buf = talloc_realloc (ctx, NULL, gchar, char_count);
+buf_size = char_count;
+
+for (i = 0; i char_count; i++) {
+ gunichar randomchar;
+ size_t written;
+
+ /* 6 for one glyph, one for null */
+
[Patch v2 4/6] test: add database routines for testing.
Initially, provide a way to create stub messages in the notmuch
database without corresponding files. This is essentially cut and
paste from lib/database.cc. This is a seperate file since we don't
want to export these symbols from libnotmuch or bloat the library with
non-exported code.
---
test/database-test.c | 72 ++
test/database-test.h | 21 +++
2 files changed, 93 insertions(+)
create mode 100644 test/database-test.c
create mode 100644 test/database-test.h
diff --git a/test/database-test.c b/test/database-test.c
new file mode 100644
index 000..f0f1c8e
--- /dev/null
+++ b/test/database-test.c
@@ -0,0 +1,72 @@
+/*
+ * Database routines intended only for testing, not exported from
+ * library.
+ *
+ * Copyright (c) 2012 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner da...@tethera.net
+ */
+
+#include notmuch-private.h
+#include database-test.h
+
+notmuch_status_t
+notmuch_database_add_stub_message (notmuch_database_t *notmuch,
+ const char *message_id,
+ const char **tags)
+{
+const char **tag;
+notmuch_status_t ret;
+notmuch_private_status_t private_status;
+notmuch_message_t *message;
+
+ret = _notmuch_database_ensure_writable (notmuch);
+if (ret)
+ return ret;
+
+message = _notmuch_message_create_for_message_id (notmuch,
+ message_id,
+ private_status);
+if (message == NULL) {
+ return COERCE_STATUS (private_status,
+ Unexpected status value from
_notmuch_message_create_for_message_id);
+
+}
+
+if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
+ _notmuch_message_add_term (message, type, mail);
+} else {
+ return NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
+}
+
+if (tags) {
+ ret = notmuch_message_freeze (message);
+ if (ret)
+ return ret;
+
+ for (tag = tags; *tag; tag++) {
+ ret = notmuch_message_add_tag (message, *tag);
+ if (ret)
+ return ret;
+ }
+}
+
+ret = notmuch_message_thaw (message);
+if (ret)
+ return ret;
+
+return NOTMUCH_STATUS_SUCCESS;
+}
diff --git a/test/database-test.h b/test/database-test.h
new file mode 100644
index 000..84f7988
--- /dev/null
+++ b/test/database-test.h
@@ -0,0 +1,21 @@
+#ifndef _DATABASE_TEST_H
+#define _DATABASE_TEST_H
+/* Add a new stub message to the given notmuch database.
+ *
+ * At least the following return values are possible:
+ *
+ * NOTMUCH_STATUS_SUCCESS: Message successfully added to database.
+ *
+ * NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: Message has the same message
+ * ID as another message already in the database.
+ *
+ * NOTMUCH_STATUS_READ_ONLY_DATABASE: Database was opened in read-only
+ * mode so no message can be added.
+ */
+
+notmuch_status_t
+notmuch_database_add_stub_message (notmuch_database_t *database,
+ const char *message_id,
+ const char **tag_list);
+
+#endif
--
1.7.10.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[Patch v2 2/6] test/hex-xcode: new test binary
This program is used both as a test-bed/unit-tester for
../util/hex-escape.c, and also as a utility in future tests of dump
and restore.
---
test/.gitignore |1 +
test/Makefile.local | 10 -
test/basic |2 +-
test/hex-xcode.c| 103 +++
4 files changed, 114 insertions(+), 2 deletions(-)
create mode 100644 test/hex-xcode.c
diff --git a/test/.gitignore b/test/.gitignore
index e63c689..be7ab5e 100644
--- a/test/.gitignore
+++ b/test/.gitignore
@@ -3,4 +3,5 @@ corpus.mail
smtp-dummy
symbol-test
arg-test
+hex-xcode
tmp.*
diff --git a/test/Makefile.local b/test/Makefile.local
index c7f1435..ab13814 100644
--- a/test/Makefile.local
+++ b/test/Makefile.local
@@ -13,6 +13,9 @@ smtp_dummy_modules = $(smtp_dummy_srcs:.c=.o)
$(dir)/arg-test: $(dir)/arg-test.o command-line-arguments.o util/libutil.a
$(call quiet,CC) -I. $^ -o $@
+$(dir)/hex-xcode: $(dir)/hex-xcode.o command-line-arguments.o util/libutil.a
+ $(call quiet,CC) -I. $^ -o $@ -ltalloc
+
$(dir)/smtp-dummy: $(smtp_dummy_modules)
$(call quiet,CC) $^ -o $@
@@ -21,7 +24,12 @@ $(dir)/symbol-test: $(dir)/symbol-test.o
.PHONY: test check
-test-binaries: $(dir)/arg-test $(dir)/smtp-dummy $(dir)/symbol-test
+TEST_BINARIES=$(dir)/arg-test \
+ $(dir)/hex-xcode \
+ $(dir)/smtp-dummy \
+ $(dir)/symbol-test
+
+test-binaries: $(TEST_BINARIES)
test: all test-binaries
@${dir}/notmuch-test $(OPTIONS)
diff --git a/test/basic b/test/basic
index d6aed24..3eebb37 100755
--- a/test/basic
+++ b/test/basic
@@ -54,7 +54,7 @@ test_begin_subtest 'Ensure that all available tests will be
run by notmuch-test'
eval $(sed -n -e '/^TESTS=$/,/^$/p' $TEST_DIRECTORY/notmuch-test)
tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
available=$(find $TEST_DIRECTORY -maxdepth 1 -type f -executable -printf
'%f\n' | \
-sed -r -e
/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test)$/d
| \
+sed -r -e
/^(aggregate-results.sh|notmuch-test|smtp-dummy|test-verbose|symbol-test|arg-test|hex-escape)$/d
| \
sort)
test_expect_equal $tests_in_suite $available
diff --git a/test/hex-xcode.c b/test/hex-xcode.c
new file mode 100644
index 000..eec6541
--- /dev/null
+++ b/test/hex-xcode.c
@@ -0,0 +1,103 @@
+/* No, nothing to to with IDE from Apple Inc.
+ testbed for ../util/hex-escape.c.
+
+ usage:
+ hex-xcode [--direction=(encode|decode)] [--omit-newline] file
+ hex-xcode [--direction=(encode|decode)] [--omit-newline] arg1 arg2 arg3 ...
+
+ */
+
+#include notmuch-client.h
+#include hex-escape.h
+#include assert.h
+
+
+enum direction {
+ENCODE,
+DECODE
+};
+
+static int
+xcode (void *ctx, enum direction dir, char *in, char **buf_p, size_t *size_p)
+{
+hex_status_t status;
+
+if (dir == ENCODE)
+ status = hex_encode (ctx, in, buf_p, size_p);
+else
+ status = hex_decode (ctx, in, buf_p, size_p);
+
+if (status == HEX_SUCCESS)
+ fputs (*buf_p, stdout);
+
+return status;
+}
+
+
+int
+main (int argc, char **argv)
+{
+
+
+enum direction dir = DECODE;
+int omit_newline = FALSE;
+
+notmuch_opt_desc_t options[] = {
+ { NOTMUCH_OPT_KEYWORD, dir, direction, 'd',
+ (notmuch_keyword_t []){ { encode, ENCODE },
+ { decode, DECODE },
+ { 0, 0 } } },
+ { NOTMUCH_OPT_BOOLEAN, omit_newline, omit-newline, 'n', 0 },
+ { 0, 0, 0, 0, 0 }
+};
+
+int opt_index = parse_arguments (argc, argv, options, 1);
+
+if (opt_index 0)
+ exit (1);
+
+void *ctx = talloc_new (NULL);
+
+char *line = NULL;
+size_t line_size;
+ssize_t line_len;
+
+char *buffer = NULL;
+size_t buf_size = 0;
+
+notmuch_bool_t read_stdin = TRUE;
+
+for (; opt_index argc; opt_index++) {
+
+ if (xcode (ctx, dir, argv[opt_index],
+ buffer, buf_size) != HEX_SUCCESS)
+ return 1;
+
+ if (!omit_newline)
+ putchar ('\n');
+
+ read_stdin = FALSE;
+}
+
+if (!read_stdin)
+ return 0;
+
+while ((line_len = getline (line, line_size, stdin)) != -1) {
+
+ chomp_newline (line);
+
+ if (xcode (ctx, dir, line, buffer, buf_size) != HEX_SUCCESS)
+ return 1;
+
+ if (!omit_newline)
+ putchar ('\n');
+
+}
+
+if (line)
+ free (line);
+
+talloc_free (ctx);
+
+return 0;
+}
--
1.7.10.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
[Patch v2 6/6] test: add broken roundtrip test
We demonstrate the current notmuch restore parser being confused by
message-id's and tags containing non alpha numeric characters
(particularly space and parentheses are problematic because they are
not escaped by notmuch dump).
We save the files as hex escaped on disk so that the output from the
failing test will not confuse the terminal emulator of people running
the test.
---
test/dump-restore |9 +
1 file changed, 9 insertions(+)
diff --git a/test/dump-restore b/test/dump-restore
index f25f7cf..6923952 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -78,4 +78,13 @@ test_begin_subtest dump --output=outfile -- from:cworth
notmuch dump --output=dump-outfile-dash-inbox.actual -- from:cworth
test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual
+test_expect_success 'roundtripping random message-ids and tags' \
+'test_subtest_known_broken set -o pipefail
+ ${TEST_DIRECTORY}/random-corpus --num-messages=10
--config-path=${NOTMUCH_CONFIG}
+ notmuch dump | ${TEST_DIRECTORY}/hex-xcode --direction=encode
EXPECTED.$test_count
+ notmuch tag -random-corpus tag:random-corpus
+ ${TEST_DIRECTORY}/hex-xcode --direction=decode EXPECTED.$test_count |
notmuch restore 2/dev/null
+ notmuch dump | ${TEST_DIRECTORY}/hex-xcode --direction=encode
OUTPUT.$test_count
+ test_cmp EXPECTED.$test_count OUTPUT.$test_count'
+
test_done
--
1.7.10.4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: Alternative (raw) message store (i.e. instead of maildir)
On Tue, Aug 14, 2012 at 7:05 PM, Ciprian Dorin Craciun ciprian.crac...@gmail.com wrote: I proposed -- better said queried if possible or at least wanted -- to have an internal interface (SPI) that any mail store would have to implement in order to be indexed and used by notmuch. I guess the interface would be quite lightweight, and would need just the following: * open store; * create a cursor iterating through all the emails, yielding only the keys; * read the envelope (as a byte blob) of a particular key; (used only for displaying thread lists, etc.;) * read the body (as a byte blob) of a particular key; * maybe create a cursor iterating over all those emails that have changed since a particular timestamp; Someone wrote a fork of offlineimap to store mail in couchdb [1]. The same couchdb can be mounted with fuse as a maildir [2] for mutt. According to the author [3], the fuse interface is read only. Assuming your proposal was implemented, one could complete this work with an interface to couchdb for offlineimap and get all the features previously requested. [1]: https://github.com/theodoreb/offlineimap [2]: https://github.com/theodoreb/couchdb-maildir-fuse [3]: http://theodoreb.net/resume ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: Alternative (raw) message store (i.e. instead of maildir)
On Tue, Aug 14, 2012 at 8:11 PM, Christophe-Marie Duquesne c...@chmd.fr wrote: one could complete this work with an interface to couchdb for offlineimap *I meant for notmuch ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: Alternative (raw) message store (i.e. instead of maildir)
Vladimir Marek vladimir.ma...@oracle.com writes: Well, if your granularity will be one archive per year of mail, it should not be that bad ... Except for someone like Keith, who has all his email since sometime in the 80s or something insane like that :) -- Stewart Smith pgpqbDWUxd3Kw.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
