[WIP2 2/4] lib: config list iterators

[David Bremner]

Since xapian provides the ability to restrict the iterator to a given
prefix, we expose this ability to the user. Otherwise we mimic the other
iterator interfances in notmuch (e.g. tags.c).
---
 lib/config.cc  | 104 +
 lib/notmuch.h  |  44 +
 test/T590-libconfig.sh |  60 
 3 files changed, 208 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index af00d6f..4cd9314 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -24,6 +24,19 @@
 
 static const std::string CONFIG_PREFIX="C";
 
+struct _notmuch_config_list {
+notmuch_database_t *notmuch;
+Xapian::TermIterator *iterator;
+char *current_key;
+char *current_val;
+};
+
+static int
+_notmuch_config_list_destroy (notmuch_config_list_t *list) {
+delete list->iterator;
+return 0;
+}
+
 notmuch_status_t
 notmuch_database_set_config (notmuch_database_t *notmuch,
 const char *key,
@@ -88,3 +101,94 @@ notmuch_database_get_config (notmuch_database_t *notmuch,
 
 return NOTMUCH_STATUS_SUCCESS;
 }
+
+notmuch_status_t
+notmuch_database_get_config_list (notmuch_database_t *notmuch,
+const char *prefix,
+notmuch_config_list_t **out)
+{
+notmuch_config_list_t *list = NULL;
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+
+list = talloc (notmuch, notmuch_config_list_t);
+if (!list) {
+   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
+   goto DONE;
+}
+
+talloc_set_destructor(list, _notmuch_config_list_destroy);
+list->iterator = new Xapian::TermIterator;
+list->notmuch = notmuch;
+list->current_key = NULL;
+list->current_val = NULL;
+
+try {
+
+   *list->iterator = notmuch->xapian_db->metadata_keys_begin(CONFIG_PREFIX 
+ (prefix ? prefix : NULL));
+
+} catch (const Xapian::Error ) {
+   _notmuch_database_log (notmuch, "A Xapian exception occurred getting 
metadata iterator: %s.\n",
+  error.get_msg().c_str());
+   notmuch->exception_reported = TRUE;
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
+}
+
+*out = list;
+
+ DONE:
+if (status && list)
+   talloc_free (list);
+
+return status;
+}
+
+notmuch_bool_t
+notmuch_config_list_valid (notmuch_config_list_t *metadata)
+{
+if (*(metadata->iterator) == 
metadata->notmuch->xapian_db->metadata_keys_end())
+   return FALSE;
+
+return TRUE;
+}
+
+const char *
+notmuch_config_list_key (notmuch_config_list_t *list)
+{
+if (list->current_key)
+   talloc_free (list->current_key);
+
+list->current_key = talloc_strdup (list, (**(list->iterator)).c_str () + 
CONFIG_PREFIX.length ());
+
+return  list->current_key;
+}
+
+const char *
+notmuch_config_list_value (notmuch_config_list_t *list)
+{
+std::string strval;
+notmuch_status_t status;
+const char *key = notmuch_config_list_key (list);
+
+/* TODO: better error reporting?? */
+status = _metadata_value (list->notmuch, key, strval);
+if (status)
+   return NULL;
+
+if (list->current_val)
+   talloc_free(list->current_val);
+
+list->current_val = talloc_strdup(list, strval.c_str ());
+return list->current_val;
+}
+
+void
+notmuch_config_list_move_to_next (notmuch_config_list_t *list)
+{
+(*(list->iterator))++;
+}
+
+void
+notmuch_config_list_destroy (notmuch_config_list_t *list)
+{
+talloc_free (list);
+}
diff --git a/lib/notmuch.h b/lib/notmuch.h
index c62223b..b439c88 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -197,6 +197,7 @@ typedef struct _notmuch_message notmuch_message_t;
 typedef struct _notmuch_tags notmuch_tags_t;
 typedef struct _notmuch_directory notmuch_directory_t;
 typedef struct _notmuch_filenames notmuch_filenames_t;
+typedef struct _notmuch_config_list notmuch_config_list_t;
 #endif /* __DOXYGEN__ */
 
 /**
@@ -1849,6 +1850,49 @@ notmuch_database_set_config (notmuch_database_t *db, 
const char *key, const char
 notmuch_status_t
 notmuch_database_get_config (notmuch_database_t *db, const char *key, char 
**value);
 
+/**
+ * Create an iterator for all config items with keys matching a given prefix
+ */
+notmuch_status_t
+notmuch_database_get_config_list (notmuch_database_t *db, const char *prefix, 
notmuch_config_list_t **out);
+
+/**
+ * Is 'config_list' iterator valid (i.e. _key, _value, _move_to_next can be 
called).
+ */
+notmuch_bool_t
+notmuch_config_list_valid (notmuch_config_list_t *config_list);
+
+/**
+ * return key for current config pair
+ *
+ * return value is owned by the iterator, and will be destroyed by the
+ * next call to notmuch_config_list_key or notmuch_config_list_destroy.
+ */
+const char *
+notmuch_config_list_key (notmuch_config_list_t *config_list);
+
+/**
+ * return 'value' for current config pair
+ *
+ * return value is owned by the iterator, and will be destroyed by the
+ * next call to notmuch_config_list_value 

second round of library config API and dump

[David Bremner]

Based on feedback on IRC from Austin and Tomi, I've simplified the API
and no longer expose the database level prefix to the caller.

This series does fix at least one memory ownership bug. Not a
particularly subtle one, I apparently only got half way through making
the current 'value' owned by the iterator.

There's also a few more tests, as the API starts to settle down.

Because some global renaming, the interdiff is somewhat noisy, but
hopefully mostly straightforward.

diff --git a/lib/Makefile.local b/lib/Makefile.local
index ccc1e49..eb442d1 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -48,7 +48,7 @@ libnotmuch_cxx_srcs = \
$(dir)/index.cc \
$(dir)/message.cc   \
$(dir)/query.cc \
-   $(dir)/metadata.cc  \
+   $(dir)/config.cc\
$(dir)/thread.cc
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/metadata.cc b/lib/config.cc
similarity index 47%
rename from lib/metadata.cc
rename to lib/config.cc
index 34b46e0..4cd9314 100644
--- a/lib/metadata.cc
+++ b/lib/config.cc
@@ -22,88 +22,36 @@
 #include "notmuch-private.h"
 #include "database-private.h"
 
-struct _notmuch_metadata {
+static const std::string CONFIG_PREFIX="C";
+
+struct _notmuch_config_list {
 notmuch_database_t *notmuch;
 Xapian::TermIterator *iterator;
-notmuch_metadata_class_t mclass;
 char *current_key;
+char *current_val;
 };
 
 static int
-_notmuch_metadata_destroy (notmuch_metadata_t *list) {
+_notmuch_config_list_destroy (notmuch_config_list_t *list) {
 delete list->iterator;
 return 0;
 }
 
-typedef struct prefix {
-notmuch_metadata_class_t mclass;
-const char *prefix;
-} prefix_t;
-
-static prefix_t METADATA_PREFIX[] = {
-{NOTMUCH_METADATA_CONFIG, "C"},
-};
-
-#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
-
-const char *
-notmuch_metadata_prefix_string (notmuch_metadata_class_t mclass)
-{
-unsigned int i;
-
-for (i=0; i < ARRAY_SIZE(METADATA_PREFIX); i++) {
-   if (METADATA_PREFIX[0].mclass == mclass)
-   return METADATA_PREFIX[i].prefix;
-}
-return NULL;
-}
-
-notmuch_status_t _make_key(void *ctx, notmuch_metadata_class_t mclass,
-   const char *in, const char **out)
-{
-const char *term;
-const char *prefix = NULL;
-
-if (!out)
-   return NOTMUCH_STATUS_NULL_POINTER;
-
-prefix = notmuch_metadata_prefix_string(mclass);
-if (!prefix)
-   return NOTMUCH_STATUS_UNSUPPORTED_OPERATION;
-
-term = talloc_asprintf (ctx, "%s%s",
-   prefix, in);
-if (!term)
-   return NOTMUCH_STATUS_OUT_OF_MEMORY;
-
-*out = term;
-return NOTMUCH_STATUS_SUCCESS;
-}
-
 notmuch_status_t
-notmuch_database_set_metadata (notmuch_database_t *notmuch,
-  notmuch_metadata_class_t mclass,
-  const char *key,
-  const char *value)
+notmuch_database_set_config (notmuch_database_t *notmuch,
+const char *key,
+const char *value)
 {
 notmuch_status_t status;
 Xapian::WritableDatabase *db;
-const char *key_term = NULL;
-void *local;
-
-local = talloc_new (NULL);
 
 status = _notmuch_database_ensure_writable (notmuch);
 if (status)
-   goto DONE;
-
-status = _make_key (local, mclass, key, _term);
-if (status)
-   goto DONE;
+   return status;
 
 try {
db = static_cast  (notmuch->xapian_db);
-   db->set_metadata (key_term, value);
+   db->set_metadata (CONFIG_PREFIX+key, value);
 } catch (const Xapian::Error ) {
status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
notmuch->exception_reported = TRUE;
@@ -112,33 +60,18 @@ notmuch_database_set_metadata (notmuch_database_t *notmuch,
   error.get_msg().c_str());
}
 }
- DONE:
-talloc_free (local);
-
-return status;
+return NOTMUCH_STATUS_SUCCESS;
 }
 
 static notmuch_status_t
 _metadata_value (notmuch_database_t *notmuch,
-notmuch_metadata_class_t mclass,
 const char *key,
 std::string )
 {
-notmuch_status_t status;
-
-const char *key_term = NULL;
-void *local;
-
-local = talloc_new (NULL);
-
-status = _make_key (local, mclass, key, _term);
-if (status)
-   goto DONE;
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
 
 try {
-
-   value = notmuch->xapian_db->get_metadata (key_term);
-
+   value = notmuch->xapian_db->get_metadata (CONFIG_PREFIX+key);
 } catch (const Xapian::Error ) {
status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
notmuch->exception_reported = TRUE;
@@ -147,24 +80,20 @@ _metadata_value (notmuch_database_t *notmuch,
   error.get_msg().c_str());
}
 }
-
- DONE:
-talloc_free (local);
  

[WIP2 4/4] CLI: add optional metadata to dump output.

[David Bremner]

This lacks at least documentation. Note that it changes the default dump
output format, but doesn't break existing notmuch-restore. It might
break user scripts though.
---
 notmuch-client.h   |  6 ++
 notmuch-dump.c | 41 ++---
 notmuch-new.c  |  2 +-
 test/T590-libconfig.sh | 10 ++
 4 files changed, 55 insertions(+), 4 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 7c9a1ea..2dca83c 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -443,11 +443,17 @@ typedef enum dump_formats {
 DUMP_FORMAT_SUP
 } dump_format_t;
 
+typedef enum dump_includes {
+DUMP_INCLUDE_TAGS=1,
+DUMP_INCLUDE_CONFIG=2,
+} dump_include_t;
+
 int
 notmuch_database_dump (notmuch_database_t *notmuch,
   const char *output_file_name,
   const char *query_str,
   dump_format_t output_format,
+  dump_include_t include,
   notmuch_bool_t gzip_output);
 
 /* If status is non-zero (i.e. error) print appropriate
diff --git a/notmuch-dump.c b/notmuch-dump.c
index 829781f..4909493 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -23,16 +23,43 @@
 #include "string-util.h"
 #include 
 
+static notmuch_status_t
+database_dump_config(notmuch_database_t *notmuch, gzFile output)
+{
+notmuch_config_list_t *list;
+notmuch_status_t status;
+status = notmuch_database_get_config_list (notmuch, NULL, );
+if (status)
+   return status;
+
+for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next 
(list)) {
+   /* FIXME hexencode key and values */
+   gzprintf(output, "#@ %s %s\n",
+notmuch_config_list_key (list), notmuch_config_list_value 
(list));
+}
+notmuch_config_list_destroy (list);
+
+return NOTMUCH_STATUS_SUCCESS;
+}
 
 static int
 database_dump_file (notmuch_database_t *notmuch, gzFile output,
-   const char *query_str, int output_format)
+   const char *query_str, int output_format, int include)
 {
 notmuch_query_t *query;
 notmuch_messages_t *messages;
 notmuch_message_t *message;
 notmuch_tags_t *tags;
 
+if (include | DUMP_INCLUDE_CONFIG) {
+   if (print_status_database ("notmuch dump", notmuch,
+  database_dump_config(notmuch,output)))
+   return EXIT_FAILURE;
+}
+
+if (! (include & DUMP_INCLUDE_TAGS))
+   return EXIT_SUCCESS;
+
 if (! query_str)
query_str = "";
 
@@ -130,6 +157,7 @@ notmuch_database_dump (notmuch_database_t *notmuch,
   const char *output_file_name,
   const char *query_str,
   dump_format_t output_format,
+  dump_include_t include,
   notmuch_bool_t gzip_output)
 {
 gzFile output = NULL;
@@ -164,7 +192,7 @@ notmuch_database_dump (notmuch_database_t *notmuch,
goto DONE;
 }
 
-ret = database_dump_file (notmuch, output, query_str, output_format);
+ret = database_dump_file (notmuch, output, query_str, output_format, 
include);
 if (ret) goto DONE;
 
 ret = gzflush (output, Z_FINISH);
@@ -226,6 +254,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, 
char *argv[])
 int opt_index;
 
 int output_format = DUMP_FORMAT_BATCH_TAG;
+int include = 0;
 notmuch_bool_t gzip_output = 0;
 
 notmuch_opt_desc_t options[] = {
@@ -233,6 +262,9 @@ notmuch_dump_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "sup", DUMP_FORMAT_SUP },
  { "batch-tag", DUMP_FORMAT_BATCH_TAG },
  { 0, 0 } } },
+   { NOTMUCH_OPT_KEYWORD_FLAGS, , "include", 'i',
+ (notmuch_keyword_t []){ { "metadata", DUMP_INCLUDE_CONFIG },
+ { "tags", DUMP_INCLUDE_TAGS} } },
{ NOTMUCH_OPT_STRING, _file_name, "output", 'o', 0  },
{ NOTMUCH_OPT_BOOLEAN, _output, "gzip", 'z', 0 },
{ NOTMUCH_OPT_INHERIT, (void *) _shared_options, NULL, 0, 0 },
@@ -245,6 +277,9 @@ notmuch_dump_command (notmuch_config_t *config, int argc, 
char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
+if (include == 0)
+   include = DUMP_INCLUDE_CONFIG | DUMP_INCLUDE_TAGS;
+
 if (opt_index < argc) {
query_str = query_string_from_args (notmuch, argc - opt_index, argv + 
opt_index);
if (query_str == NULL) {
@@ -254,7 +289,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, 
char *argv[])
 }
 
 ret = notmuch_database_dump (notmuch, output_file_name, query_str,
-output_format, gzip_output);
+output_format, include, gzip_output);
 
 notmuch_database_destroy (notmuch);
 
diff --git a/notmuch-new.c b/notmuch-new.c
index e503776..fd2ff82 100644
--- a/notmuch-new.c
+++ 

[WIP2 1/4] lib: provide config API

[David Bremner]

This is a thin wrapper around the Xapian metadata API. The job of this
layer is to keep the config key value pairs from colliding with other
metadata by transparently prefixing the keys, along with the usual glue
to provide a C interface.

The split of _get_config into two functions is to allow returning of the
return value with different memory ownership semantics.
---
 lib/Makefile.local |  1 +
 lib/config.cc  | 90 ++
 lib/notmuch.h  | 20 +++
 test/T590-libconfig.sh | 58 
 4 files changed, 169 insertions(+)
 create mode 100644 lib/config.cc
 create mode 100755 test/T590-libconfig.sh

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 3a07090..eb442d1 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -48,6 +48,7 @@ libnotmuch_cxx_srcs = \
$(dir)/index.cc \
$(dir)/message.cc   \
$(dir)/query.cc \
+   $(dir)/config.cc\
$(dir)/thread.cc
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/config.cc b/lib/config.cc
new file mode 100644
index 000..af00d6f
--- /dev/null
+++ b/lib/config.cc
@@ -0,0 +1,90 @@
+/* metadata.cc - API for database metadata
+ *
+ * Copyright © 2015 David Bremner
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: David Bremner 
+ */
+
+#include "notmuch.h"
+#include "notmuch-private.h"
+#include "database-private.h"
+
+static const std::string CONFIG_PREFIX="C";
+
+notmuch_status_t
+notmuch_database_set_config (notmuch_database_t *notmuch,
+const char *key,
+const char *value)
+{
+notmuch_status_t status;
+Xapian::WritableDatabase *db;
+
+status = _notmuch_database_ensure_writable (notmuch);
+if (status)
+   return status;
+
+try {
+   db = static_cast  (notmuch->xapian_db);
+   db->set_metadata (CONFIG_PREFIX+key, value);
+} catch (const Xapian::Error ) {
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
+   notmuch->exception_reported = TRUE;
+   if (! notmuch->exception_reported) {
+   _notmuch_database_log (notmuch, "Error: A Xapian exception occurred 
setting metadata: %s\n",
+  error.get_msg().c_str());
+   }
+}
+return NOTMUCH_STATUS_SUCCESS;
+}
+
+static notmuch_status_t
+_metadata_value (notmuch_database_t *notmuch,
+const char *key,
+std::string )
+{
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+
+try {
+   value = notmuch->xapian_db->get_metadata (CONFIG_PREFIX+key);
+} catch (const Xapian::Error ) {
+   status = NOTMUCH_STATUS_XAPIAN_EXCEPTION;
+   notmuch->exception_reported = TRUE;
+   if (! notmuch->exception_reported) {
+   _notmuch_database_log (notmuch, "Error: A Xapian exception occurred 
getting metadata: %s\n",
+  error.get_msg().c_str());
+   }
+}
+return status;
+}
+
+notmuch_status_t
+notmuch_database_get_config (notmuch_database_t *notmuch,
+const char *key,
+char **value) {
+std::string strval;
+notmuch_status_t status;
+
+if (!value)
+   return NOTMUCH_STATUS_NULL_POINTER;
+
+status = _metadata_value (notmuch, key, strval);
+if (status)
+   return status;
+
+*value = strdup (strval.c_str ());
+
+return NOTMUCH_STATUS_SUCCESS;
+}
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 310a8b8..c62223b 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1829,6 +1829,26 @@ notmuch_filenames_move_to_next (notmuch_filenames_t 
*filenames);
 void
 notmuch_filenames_destroy (notmuch_filenames_t *filenames);
 
+
+/**
+ * set config 'key' to 'value'
+ *
+ */
+notmuch_status_t
+notmuch_database_set_config (notmuch_database_t *db, const char *key, const 
char *value);
+
+/**
+ * retrieve config item 'key', assign to  'value'
+ *
+ * keys which have not been previously set with n_d_set_config will
+ * return an empty string.
+ *
+ * return value is allocated by malloc and should be freed by the
+ * caller.
+ */
+notmuch_status_t
+notmuch_database_get_config (notmuch_database_t *db, const char *key, char 
**value);
+
 /* @} */
 
 NOTMUCH_END_DECLS

[WIP2 3/4] CLI: add print_status_database

[David Bremner]

This could probably be used at quite a few places in the existing code,
but in the immediate future I plan to use in some new code in
notmuch-dump
---
 notmuch-client.h |  5 +
 status.c | 17 +
 2 files changed, 22 insertions(+)

diff --git a/notmuch-client.h b/notmuch-client.h
index 3bd2903..7c9a1ea 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -459,6 +459,11 @@ print_status_query (const char *loc,
const notmuch_query_t *query,
notmuch_status_t status);
 
+notmuch_status_t
+print_status_database (const char *loc,
+  const notmuch_database_t *database,
+  notmuch_status_t status);
+
 #include "command-line-arguments.h"
 
 extern char *notmuch_requested_db_uuid;
diff --git a/status.c b/status.c
index 8fa81cb..45d3fb4 100644
--- a/status.c
+++ b/status.c
@@ -19,3 +19,20 @@ print_status_query (const char *loc,
 }
 return status;
 }
+
+notmuch_status_t
+print_status_database (const char *loc,
+   const notmuch_database_t *notmuch,
+   notmuch_status_t status)
+{
+if (status) {
+   const char *msg;
+
+   fprintf (stderr, "%s: %s\n", loc,
+notmuch_status_to_string (status));
+   msg = notmuch_database_status_string (notmuch);
+   if (msg)
+   fputs (msg, stderr);
+}
+return status;
+}
-- 
2.6.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Crash with Python bindings

[Konrad Hinsen]

Hi everyone,

I have been writing quite a few Python scripts for notmuch before 
running into a strange bug. Here is a minimal script producing it:


--
from notmuch import Query, Database

def foo(bar):
pass

db = Database()
q = Query(db, "*")
db.close()
--

Running this script (Python 3.5, MacOS X) yields:

[1]22478 abort  pydev3 Temp/notmuch_test.py

The crash actually happens *after* db.close(), when the Python 
interpreter exists, and therefore I suspect that no data is lost, but I 
hesitate to use scripts with that behavior in production use.


The strange part is that what causes the crash is the presence of the 
function foo(), even though it is never called. Remove foo and the 
script runs fine. It is also necessary to create a Query object. The 
combination of a function definition (any) and the creation of a Query 
object yields the crash. This looks like a memory management issue, but 
I didn't explore it any further.


Cheers,
  Konrad.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

binding/python/README is hopelessly outdated

[Konrad Hinsen]

Hi everyone,

while looking for the latest source for checking if my recent bug report 
still applies, I noticed that the README for the Python bindings is 
hopelessly outdated. It refers to a bitbucket repository that hasn't 
been updated since 2010. Moreover, it recommends installing from PyPI, 
where I cannot find the notmuch Python bindings any more.


Cheers,
  Konrad.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[Konrad Hinsen]

Hi Justus,

> So I guess what happens is that Python3 changed how the interpreter
> environment is torn down and they actually destroy the 'q' object.  If
> that is so, then your data is indeed safe.

That reminds me of a recent change in object finalization in Python 3:

  https://www.python.org/dev/peps/pep-0442/

I'll look into that, thanks for the gdb exploration!

Cheers,
  Konrad.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[Konrad Hinsen]

David Bremner  writes:

>> from notmuch import Query, Database
>>
>> def foo(bar):
>>  pass
>>
>> db = Database()
>> q = Query(db, "*")
>> db.close()
>
> Do you really call the constructor without a path? Or are you censoring
> the script for some reason?

No path means path=None, which stands for the path from
~/.notmuch-config. That's exactly what I want. Is there some reason not
to rely on this mechanism?

Cheers,
  Konrad.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[David Bremner]

David Bremner  writes:

>> No path means path=None, which stands for the path from
>> ~/.notmuch-config. That's exactly what I want. Is there some reason not
>> to rely on this mechanism?
>
> Oh sorry, I'm (obviously) not that familiar with the python bindings.

Nothing to do with Konrad's crash, but I consider the fact that the
python bindings read ~/.notmuch-config to be a kind of layering
violation, since that file belongs to the CLI, while the bindings are
supposed to provide access to libnotmuch. Whether this is a real problem
or just an aesthetic one, I'm not sure, but I thought I'd mention it
since we are thinking of various config related issues. Obviously the
location of the database is not one of the things it makes sense to
store in the database. I can imagine scenarios where the bindings might
be usable without the CLI, but they seem fairly artificial so far, since
it seems like almost everyone needs notmuch-new / notmuch-insert.

d

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[W. Trevor King]

On Tue, Jan 12, 2016 at 03:23:46PM +0100, Konrad Hinsen wrote:
> Hi Justus,
> 
> > So I guess what happens is that Python3 changed how the
> > interpreter environment is torn down and they actually destroy the
> > 'q' object.  If that is so, then your data is indeed safe.
> 
> That reminds me of a recent change in object finalization in Python
> 3:
> 
>   https://www.python.org/dev/peps/pep-0442/

I'm pretty sure that is what's going on.  The PEP landed in Python
3.4, explaining why I don't see this issue in 3.3 [1].  And it has
[2]:

   In particular, this PEP obsoletes the current guideline that
   "objects with a __del__ method should not be part of a reference
   cycle".

I'm not sure what the best way is to fix __del__ in our Python
bindings, but if you manage them explicitly:

  db = Database()
  q = Query(db, "*")
  del q
  db.close()
  del db

you can avoid the abort (which happens when q.__del__ is called after
db.__del__).  We could make that sort of cleanup easier with context
managers for Query objects (we have them for databases since [3]), and
they look like the only object that keep an internal database
reference:

  with Database() as db:
with Query(db, "*") as q:
  # do something with q
db.close()

Cheers,
Trevor

[1]: id:20160112180813.ga20...@odin.tremily.us
[2]: https://www.python.org/dev/peps/pep-0442/#impact
[3]: 36ce7e3c (python: implement the context manager protocol for
 database objects, 2012-02-15, v0.12)

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[David Bremner]

Konrad Hinsen  writes:

> Hi everyone,
>
> I have been writing quite a few Python scripts for notmuch before 
> running into a strange bug. Here is a minimal script producing it:
>
> --
> from notmuch import Query, Database
>
> def foo(bar):
>  pass
>
> db = Database()
> q = Query(db, "*")
> db.close()

Do you really call the constructor without a path? Or are you censoring
the script for some reason?

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[David Bremner]

Konrad Hinsen  writes:

> David Bremner  writes:
>
>>> from notmuch import Query, Database
>>>
>>> def foo(bar):
>>>  pass
>>>
>>> db = Database()
>>> q = Query(db, "*")
>>> db.close()
>>
>> Do you really call the constructor without a path? Or are you censoring
>> the script for some reason?
>
> No path means path=None, which stands for the path from
> ~/.notmuch-config. That's exactly what I want. Is there some reason not
> to rely on this mechanism?

Oh sorry, I'm (obviously) not that familiar with the python bindings.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Crash with Python bindings

[W. Trevor King]

On Tue, Jan 12, 2016 at 10:41:57AM +0100, Konrad Hinsen wrote:
> --
> from notmuch import Query, Database
> 
> def foo(bar):
>  pass
> 
> db = Database()
> q = Query(db, "*")
> db.close()
> --
> 
> Running this script (Python 3.5, MacOS X) yields:
>
> [1]22478 abort  pydev3 Temp/notmuch_test.py
> …
> The strange part is that what causes the crash is the presence of the 
> function foo(), even though it is never called. Remove foo and the 
> script runs fine. It is also necessary to create a Query object.

Adding some more data-points, I see the same results with Python 3.4.3
on Gentoo, although stderr just gets “Aborted”.  All permutations seem
to work on Python 3.3.5.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Binding access to ~/.notmuch-config (was: Crash with Python bindings)

[W. Trevor King]

On Tue, Jan 12, 2016 at 03:03:15PM -0400, David Bremner wrote:
> Nothing to do with Konrad's crash, but I consider the fact that the
> python bindings read ~/.notmuch-config to be a kind of layering
> violation, since that file belongs to the CLI, while the bindings
> are supposed to provide access to libnotmuch.

I think of ~/.notmuch-config as being shared between all client code,
and in that view it makes sense to have both the CLI and Python
bindings (and other bindings) access it to figure out how to configure
their library access calls.  Having a separate config file for each
client to point at the default database path seems like more trouble
than it's worth, as does adding a library function for “reach into
some local config and return the default database path”.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[Gaudenz Steinlin] Bug#810784: should match email adress case insensitive when sending encrypted mail

[David Bremner]

--- Begin Message ---
Package: notmuch-emacs
Version: 0.21-3
Severity: normal

When sending encrypted mail the key lookup to encrypt to is done case
sensitive on the mail address. As mail addresses are case insensitive
this should be done case insensitive. Otherwise keys for users which for
some reason have uppercase letters in their email address in the key UID
are not found.

Gaudenz

P.S.: I'm not completely sure if this part of the mail sending is done
by notmuch-emacs or some other part of emacs. Feel free to reassign as
appropriate.

-- System Information:
Debian Release: stretch/sid
  APT prefers testing-proposed-updates
  APT policy: (500, 'testing-proposed-updates'), (500, 'testing'), (100, 
'unstable'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.3.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=de_CH.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages notmuch-emacs depends on:
ii  emacs24 24.5+1-3
ii  emacsen-common  2.0.8
ii  notmuch 0.21-3

notmuch-emacs recommends no packages.

notmuch-emacs suggests no packages.

-- no debconf information

--- End Message ---
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch