[PATCH] test: add a known broken test for S/MIME decryption

2019-11-17 Thread David Bremner
This should serve to clarify this feature is not implimented in
notmuch yet.
---
 test/T355-smime.sh | 9 +
 1 file changed, 9 insertions(+)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 336da917..11a4d6cd 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -94,4 +94,13 @@ Verification successful
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Decryption (notmuch CLI)"
+test_subtest_known_broken
+notmuch show --decrypt=true subject:"test encrypted message 001" |\
+grep "^This is a" > OUTPUT
+cat < EXPECTED
+This is a test encrypted message.
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.24.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 5/7] emacs/tree: add notmuch-tree-archive-message-than-next-or-exit

2019-11-17 Thread William Casarin
William Casarin  writes:

> This is the notmuch-tree version of
> notmuch-show-archive-message-than-next-or-exit.
>
> Signed-off-by: William Casarin 
> ---
>  emacs/notmuch-tree.el | 30 --
>  1 file changed, 20 insertions(+), 10 deletions(-)
>
> diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
> index 367b10c4..fd6f9f83 100644
> --- a/emacs/notmuch-tree.el
> +++ b/emacs/notmuch-tree.el
> @@ -563,6 +563,15 @@ message will be \"unarchived\", i.e. the tag changes in
>(notmuch-tree-archive-message unarchive)
>(notmuch-tree-next-matching-message))
>  
> +(defun notmuch-tree-archive-message-then-next-or-exit ()
> +  "Archive current message, then show next open message in current thread.
> +
> +If at the last open message in the current thread, then exit back
> +to search results."
> +  (interactive)
> +  (notmuch-tree-archive-message)
> +  (notmuch-tree-next-matching-message t))
> +
>  (defun notmuch-tree-next-message ()
>"Move to next message."
>(interactive)
> @@ -588,23 +597,24 @@ nil otherwise."
>(forward-line (if prev -1 nil)))
>  (not last)))
>  
> -(defun notmuch-tree-matching-message ( prev)
> +(defun notmuch-tree-matching-message ( prev pop-at-end)
>"Move to the next or previous matching message"
>(interactive "P")
>(forward-line (if prev -1 nil))
> -  (notmuch-tree-goto-matching-message prev)
> -  (when (window-live-p notmuch-tree-message-window)
> -(notmuch-tree-show-message-in)))
> +  (if (and pop-at-end (not (notmuch-tree-goto-matching-message prev)))

bah, this should be

  (if (and (not (notmuch-tree-goto-matching-message prev)) pop-at-end)

since short-circuit evaulation prevents the goto-matching-message from
getting called.

I'll push out a v2 later if this gets enough Concept ACKs.

> +  (notmuch-tree-quit pop-at-end)
> +(when (window-live-p notmuch-tree-message-window)
> +  (notmuch-tree-show-message-in
>  
> -(defun notmuch-tree-prev-matching-message ()
> +(defun notmuch-tree-prev-matching-message ( pop-at-end)
>"Move to previous matching message."
> -  (interactive)
> -  (notmuch-tree-matching-message t))
> +  (interactive "P")
> +  (notmuch-tree-matching-message t pop-at-end))
>  
> -(defun notmuch-tree-next-matching-message ()
> +(defun notmuch-tree-next-matching-message ( pop-at-end)
>"Move to next matching message."
> -  (interactive)
> -  (notmuch-tree-matching-message))
> +  (interactive "P")
> +  (notmuch-tree-matching-message nil pop-at-end))
>  
>  (defun notmuch-tree-refresh-view ()
>"Refresh view."
> -- 
> 2.23.0
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 0/7] Port notmuch-show's x/X bindings to notmuch-tree

2019-11-17 Thread William Casarin
These patches bring notmuch-tree more in line with the user experience
of notmuch-show by adding the x/X bindings.

id:20191113225752.26502-1-j...@jb55.com is also recommended, it adds a
bind for opening a thread in notmuch-tree mode.

William Casarin (7):
  emacs/tree: return true if a thread was found in next-thread
  emacs/tree: add notmuch-tree-goto-matching-message
  emacs/tree: add notmuch-tree-matching-message
  emacs/tree: add kill-both prefix argument to notmuch-tree-quit
  emacs/tree: add notmuch-tree-archive-message-than-next-or-exit
  emacs/tree: add notmuch-tree-archive-thread-then-exit
  emacs/tree: add x/X bindings

 emacs/notmuch-tree.el | 72 +++
 1 file changed, 52 insertions(+), 20 deletions(-)


base-commit: 7ad7cfbff232431377562271901ee00202bf0bd0
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 7/7] emacs/tree: add x/X bindings

2019-11-17 Thread William Casarin
Add x and X binds to notmuch-tree for functionally that we have in
notmuch-show.

The notmuch-tree-quit binding is somewhat redundant, since it is
handled by notmuch-bury-or-kill-this-buffer which is bound to q.

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index b8844b64..d3a54bcb 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -277,7 +277,8 @@ FUNC."
 ;; The main tree view bindings
 (define-key map (kbd "RET") 'notmuch-tree-show-message)
 (define-key map [mouse-1] 'notmuch-tree-show-message)
-(define-key map "x" 'notmuch-tree-quit)
+(define-key map "x" 'notmuch-tree-archive-message-then-next-or-exit)
+(define-key map "X" 'notmuch-tree-archive-thread-then-exit)
 (define-key map "A" 'notmuch-tree-archive-thread)
 (define-key map "a" 'notmuch-tree-archive-message-then-next)
 (define-key map "z" 'notmuch-tree-to-tree)
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 4/7] emacs/tree: add kill-both prefix argument to notmuch-tree-quit

2019-11-17 Thread William Casarin
This allows us to close both windows at the same time.

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index b9173790..367b10c4 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -530,10 +530,10 @@ Shows in split pane or whole window according to value of
   (when (notmuch-tree-scroll-message-window)
 (notmuch-tree-next-matching-message)))
 
-(defun notmuch-tree-quit ()
+(defun notmuch-tree-quit ( kill-both)
   "Close the split view or exit tree."
-  (interactive)
-  (unless (notmuch-tree-close-message-window)
+  (interactive "P")
+  (when (or (not (notmuch-tree-close-message-window)) kill-both)
 (kill-buffer (current-buffer
 
 (defun notmuch-tree-close-message-window ()
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 3/7] emacs/tree: add notmuch-tree-matching-message

2019-11-17 Thread William Casarin
This functions removes some duplicate logic between
notmuch-tree-{next,prev}-matching-message

We do this because we will be adding some additional logic similar to
the notmuch-show-next-open-message function, and it will help if this
logic is all in one place.

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index 9f2d87b5..b9173790 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -588,21 +588,23 @@ nil otherwise."
   (forward-line (if prev -1 nil)))
 (not last)))
 
+(defun notmuch-tree-matching-message ( prev)
+  "Move to the next or previous matching message"
+  (interactive "P")
+  (forward-line (if prev -1 nil))
+  (notmuch-tree-goto-matching-message prev)
+  (when (window-live-p notmuch-tree-message-window)
+(notmuch-tree-show-message-in)))
+
 (defun notmuch-tree-prev-matching-message ()
   "Move to previous matching message."
   (interactive)
-  (forward-line -1)
-  (notmuch-tree-goto-matching-message t)
-  (when (window-live-p notmuch-tree-message-window)
-(notmuch-tree-show-message-in)))
+  (notmuch-tree-matching-message t))
 
 (defun notmuch-tree-next-matching-message ()
   "Move to next matching message."
   (interactive)
-  (forward-line)
-  (notmuch-tree-goto-matching-message)
-  (when (window-live-p notmuch-tree-message-window)
-(notmuch-tree-show-message-in)))
+  (notmuch-tree-matching-message))
 
 (defun notmuch-tree-refresh-view ()
   "Refresh view."
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 5/7] emacs/tree: add notmuch-tree-archive-message-than-next-or-exit

2019-11-17 Thread William Casarin
This is the notmuch-tree version of
notmuch-show-archive-message-than-next-or-exit.

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index 367b10c4..fd6f9f83 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -563,6 +563,15 @@ message will be \"unarchived\", i.e. the tag changes in
   (notmuch-tree-archive-message unarchive)
   (notmuch-tree-next-matching-message))
 
+(defun notmuch-tree-archive-message-then-next-or-exit ()
+  "Archive current message, then show next open message in current thread.
+
+If at the last open message in the current thread, then exit back
+to search results."
+  (interactive)
+  (notmuch-tree-archive-message)
+  (notmuch-tree-next-matching-message t))
+
 (defun notmuch-tree-next-message ()
   "Move to next message."
   (interactive)
@@ -588,23 +597,24 @@ nil otherwise."
   (forward-line (if prev -1 nil)))
 (not last)))
 
-(defun notmuch-tree-matching-message ( prev)
+(defun notmuch-tree-matching-message ( prev pop-at-end)
   "Move to the next or previous matching message"
   (interactive "P")
   (forward-line (if prev -1 nil))
-  (notmuch-tree-goto-matching-message prev)
-  (when (window-live-p notmuch-tree-message-window)
-(notmuch-tree-show-message-in)))
+  (if (and pop-at-end (not (notmuch-tree-goto-matching-message prev)))
+  (notmuch-tree-quit pop-at-end)
+(when (window-live-p notmuch-tree-message-window)
+  (notmuch-tree-show-message-in
 
-(defun notmuch-tree-prev-matching-message ()
+(defun notmuch-tree-prev-matching-message ( pop-at-end)
   "Move to previous matching message."
-  (interactive)
-  (notmuch-tree-matching-message t))
+  (interactive "P")
+  (notmuch-tree-matching-message t pop-at-end))
 
-(defun notmuch-tree-next-matching-message ()
+(defun notmuch-tree-next-matching-message ( pop-at-end)
   "Move to next matching message."
-  (interactive)
-  (notmuch-tree-matching-message))
+  (interactive "P")
+  (notmuch-tree-matching-message nil pop-at-end))
 
 (defun notmuch-tree-refresh-view ()
   "Refresh view."
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/7] emacs/tree: add notmuch-tree-goto-matching-message

2019-11-17 Thread William Casarin
This function captures some common logic when jumping to matching
messages in notmuch-tree mode.

We also add a new return value (t or nil), that indicates if there was
a next matching message in the thread to show.

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index 4bc05160..9f2d87b5 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -577,12 +577,22 @@ message will be \"unarchived\", i.e. the tag changes in
   (when (window-live-p notmuch-tree-message-window)
 (notmuch-tree-show-message-in)))
 
+(defun notmuch-tree-goto-matching-message ( prev)
+  "Move to the next or previous matching message.
+
+Returns t if there was a next matching message in the thread to show,
+nil otherwise."
+  (let (last)
+(while (and (not (setq last (if prev (bobp) (eobp
+(not (notmuch-tree-get-match)))
+  (forward-line (if prev -1 nil)))
+(not last)))
+
 (defun notmuch-tree-prev-matching-message ()
   "Move to previous matching message."
   (interactive)
   (forward-line -1)
-  (while (and (not (bobp)) (not (notmuch-tree-get-match)))
-(forward-line -1))
+  (notmuch-tree-goto-matching-message t)
   (when (window-live-p notmuch-tree-message-window)
 (notmuch-tree-show-message-in)))
 
@@ -590,8 +600,7 @@ message will be \"unarchived\", i.e. the tag changes in
   "Move to next matching message."
   (interactive)
   (forward-line)
-  (while (and (not (eobp)) (not (notmuch-tree-get-match)))
-(forward-line))
+  (notmuch-tree-goto-matching-message)
   (when (window-live-p notmuch-tree-message-window)
 (notmuch-tree-show-message-in)))
 
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/7] emacs/tree: return true if a thread was found in next-thread

2019-11-17 Thread William Casarin
This will allow us to pop back to parent buffers when there are no
more threads to jump to.

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index c00315e8..4bc05160 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -620,10 +620,14 @@ message will be \"unarchived\", i.e. the tag changes in
   (notmuch-tree-thread-top))
 
 (defun notmuch-tree-next-thread ()
+  "Get the next thread in the current tree. Returns t if a thread was
+found or nil if not."
   (interactive)
   (forward-line 1)
-  (while (not (or (notmuch-tree-get-prop :first) (eobp)))
-(forward-line 1)))
+  (let (end)
+(while (not (or (notmuch-tree-get-prop :first) (setq end (eobp
+  (forward-line 1))
+(not end)))
 
 (defun notmuch-tree-thread-mapcar (function)
   "Iterate through all messages in the current thread
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 6/7] emacs/tree: add notmuch-tree-archive-thread-then-exit

2019-11-17 Thread William Casarin
This is the notmuch-tree version of
notmuch-show-archive-thread-then-exit

Signed-off-by: William Casarin 
---
 emacs/notmuch-tree.el | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el
index fd6f9f83..b8844b64 100644
--- a/emacs/notmuch-tree.el
+++ b/emacs/notmuch-tree.el
@@ -563,6 +563,12 @@ message will be \"unarchived\", i.e. the tag changes in
   (notmuch-tree-archive-message unarchive)
   (notmuch-tree-next-matching-message))
 
+(defun notmuch-tree-archive-thread-then-exit ()
+  "Archive all messages in the current buffer, then exit notmuch-tree."
+  (interactive)
+  (notmuch-tree-archive-thread)
+  (notmuch-tree-quit t))
+
 (defun notmuch-tree-archive-message-then-next-or-exit ()
   "Archive current message, then show next open message in current thread.
 
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] configure: fix reference to possibly undefined $PKG_CONFIG_PATH

2019-11-17 Thread Tomi Ollila
In case zlib not found by pkg-config(1) the pkg-config information
is resolved by attempting to print ZLIB_VERSION from from zlib
installation if it exists anyway.

If above done successfully compat/zlib.pc is written for forthcoming
pkg-config execution.

Since `set -u` is in effect (since 124a67e96, 2016-05-06),
expanding unset $PKG_CONFIG_PATH (would have) failed whenever tried.

Now it is changed to set as "$PKG_CONFIG_PATH:compat" if PKG_CONFIG_PATH
is set and is non-empty string, plain "compat" otherwise.
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 2160be37..00a9d03d 100755
--- a/configure
+++ b/configure
@@ -606,7 +606,7 @@ fi
 if ! pkg-config --exists zlib; then
   ${CC} -o compat/gen_zlib_pc "$srcdir"/compat/gen_zlib_pc.c >/dev/null 2>&1 &&
   compat/gen_zlib_pc > compat/zlib.pc &&
-  PKG_CONFIG_PATH="$PKG_CONFIG_PATH":compat &&
+  PKG_CONFIG_PATH=${PKG_CONFIG_PATH:+$PKG_CONFIG_PATH:}compat &&
   export PKG_CONFIG_PATH
   rm -f compat/gen_zlib_pc
 fi
-- 
2.23.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] Move from _add_message to _index_file API

2019-11-17 Thread Floris Bruynooghe
This moves away from the deprecated notmuch_database_add_message API
and instead uses the notmuch_database_index_file API.  This means
instroducing a class to manage the index options and bumping the
library version requirement to 5.1.
---
 bindings/python-cffi/notdb/_build.py| 26 +-
 bindings/python-cffi/notdb/_database.py | 88 -
 bindings/python-cffi/tests/test_database.py | 24 ++
 3 files changed, 131 insertions(+), 7 deletions(-)

diff --git a/bindings/python-cffi/notdb/_build.py 
b/bindings/python-cffi/notdb/_build.py
index 6be7e5b1..642ef77a 100644
--- a/bindings/python-cffi/notdb/_build.py
+++ b/bindings/python-cffi/notdb/_build.py
@@ -12,6 +12,9 @@ ffibuilder.set_source(
 #if LIBNOTMUCH_MAJOR_VERSION < 5
 #error libnotmuch version not supported by notdb
 #endif
+#if LIBNOTMUCH_MINOR_VERSION < 1
+#ERROR libnotmuch  version < 5.1 not supported
+#endif
 """,
 include_dirs=['../../lib'],
 library_dirs=['../../lib'],
@@ -68,6 +71,12 @@ ffibuilder.cdef(
 NOTMUCH_EXCLUDE_FALSE,
 NOTMUCH_EXCLUDE_ALL
 } notmuch_exclude_t;
+typedef enum {
+NOTMUCH_DECRYPT_FALSE,
+NOTMUCH_DECRYPT_TRUE,
+NOTMUCH_DECRYPT_AUTO,
+NOTMUCH_DECRYPT_NOSTASH,
+} notmuch_decryption_policy_t;
 
 // These are fully opaque types for us, we only ever use pointers.
 typedef struct _notmuch_database notmuch_database_t;
@@ -81,6 +90,7 @@ ffibuilder.cdef(
 typedef struct _notmuch_directory notmuch_directory_t;
 typedef struct _notmuch_filenames notmuch_filenames_t;
 typedef struct _notmuch_config_list notmuch_config_list_t;
+typedef struct _notmuch_indexopts notmuch_indexopts_t;
 
 const char *
 notmuch_status_to_string (notmuch_status_t status);
@@ -118,9 +128,10 @@ ffibuilder.cdef(
 notmuch_database_get_revision (notmuch_database_t *notmuch,
const char **uuid);
 notmuch_status_t
-notmuch_database_add_message (notmuch_database_t *database,
-  const char *filename,
-  notmuch_message_t **message);
+notmuch_database_index_file (notmuch_database_t *database,
+ const char *filename,
+ notmuch_indexopts_t *indexopts,
+ notmuch_message_t **message);
 notmuch_status_t
 notmuch_database_remove_message (notmuch_database_t *database,
  const char *filename);
@@ -294,6 +305,15 @@ ffibuilder.cdef(
 notmuch_filenames_move_to_next (notmuch_filenames_t *filenames);
 void
 notmuch_filenames_destroy (notmuch_filenames_t *filenames);
+notmuch_indexopts_t *
+notmuch_database_get_default_indexopts (notmuch_database_t *db);
+notmuch_status_t
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+  notmuch_decryption_policy_t 
decrypt_policy);
+notmuch_decryption_policy_t
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t 
*indexopts);
+void
+notmuch_indexopts_destroy (notmuch_indexopts_t *options);
 """
 )
 
diff --git a/bindings/python-cffi/notdb/_database.py 
b/bindings/python-cffi/notdb/_database.py
index d414082a..8fa631f2 100644
--- a/bindings/python-cffi/notdb/_database.py
+++ b/bindings/python-cffi/notdb/_database.py
@@ -45,6 +45,13 @@ class QueryExclude(enum.Enum):
 ALL = capi.lib.NOTMUCH_EXCLUDE_ALL
 
 
+class DecryptionPolicy(enum.Enum):
+FALSE = capi.lib.NOTMUCH_DECRYPT_FALSE
+TRUE = capi.lib.NOTMUCH_DECRYPT_TRUE
+AUTO = capi.lib.NOTMUCH_DECRYPT_AUTO
+NOSTASH = capi.lib.NOTMUCH_DECRYPT_NOSTASH
+
+
 class Database(base.NotmuchObject):
 """Toplevel access to notmuch.
 
@@ -332,7 +339,17 @@ class Database(base.NotmuchObject):
 def get_directory(self, path):
 raise NotImplementedError
 
-def add(self, filename, *, sync_flags=False):
+def default_indexopts(self):
+"""Returns default index options for the database.
+
+:raises ObjectDestoryedError: if used after destroyed.
+
+:returns: :class:`IndexOptions`.
+"""
+opts = capi.lib.notmuch_database_get_default_indexopts(self._db_p)
+return IndexOptions(self, opts)
+
+def add(self, filename, *, sync_flags=False, indexopts=None):
 """Add a message to the database.
 
 Add a new message to the notmuch database.  The message is
@@ -346,6 +363,11 @@ class Database(base.NotmuchObject):
 :param sync_flags: Whether to sync the known maildir flags to
notmuch tags.  See :meth:`Message.flags_to_tags` for
details.
+:type sync_flags: bool
+:param indexopts: The indexing options, see
+   :meth:`default_indexopts`.  Leave as `None` to use the
+   default options configured in the database.
+:type 

Adressing deprecation warnings

2019-11-17 Thread Floris Bruynooghe
This addresses the deprecation warning pointed out in another
thread on this list.  It's also a nice self-contained example
of adding an API including a new object.

This patch is against the wip/cffi branch, it does not rely
on my other patch against this branch which renames things.
Conflicts with that renaming patch should be trivial if any.


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Python3 cffi bindings

2019-11-17 Thread Gaute Hope
On Sun, Nov 17, 2019 at 6:14 PM Floris Bruynooghe  wrote:
>
> Hi Gaute,
>
> Thanks for trying this out!
>
> On Mon 04 Nov 2019 at 11:27 +0100, Gaute Hope wrote:
> > I just checked out the wip/cffi branch on git.notmuch.org with the
> > purpose of porting Lieer (https://github.com/gauteh/lieer). There
> > seems to be some missing functionality: `Database.get_directory()`
> > specifically.
>
> Yeah, I didn't add that yet because I don't fully understand how it
> should be used.  Specifically I don't know where one might get a
> pathname from to pass to .get_directory() and thus whether the API would
> be cleaner to just return a reasonable directory object from whatever
> location that might be.  Maybe notmuch_database_get_path() is the only
> entrypoint here and you can get further by listing files and directories
> from it?  But maybe people then use the filesystem directly to find a
> directory and create the directories ad-hoc.

If I understand correctly then these are the directories known to
notmuch db, so may not correspond to filesystem. Lieer do not modify
Directory objects directly, but others might.

> I grepped lieer but I think you only use it in one place?  And if I
> understand it correctly you only do this to check if your mailstore/cwd
> is inside the notmuch database.  I.e. this is equivalent to checking if
> your mailstore/cwd has notmuch2.Database.path as prefix which you could
> easily do directly rather than using the FileError exception from
> .get_directory().

Yes, I think that would work here. I need the path of the directory
later (for the path:.. query). Seems that the current python API
removes the leading path of the database for the argument to
notmuch_database_get_directory(..) -- at least if the notmuch API docs
are correct. I had some reported issues with symlinked directories and
absolute paths, but I don't think any of that would be influenced by
changes like these.

Regards, Gaute
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Python3 cffi bindings

2019-11-17 Thread Floris Bruynooghe
Hi Gaute,

Thanks for trying this out!

On Mon 04 Nov 2019 at 11:27 +0100, Gaute Hope wrote:
> I just checked out the wip/cffi branch on git.notmuch.org with the
> purpose of porting Lieer (https://github.com/gauteh/lieer). There
> seems to be some missing functionality: `Database.get_directory()`
> specifically.

Yeah, I didn't add that yet because I don't fully understand how it
should be used.  Specifically I don't know where one might get a
pathname from to pass to .get_directory() and thus whether the API would
be cleaner to just return a reasonable directory object from whatever
location that might be.  Maybe notmuch_database_get_path() is the only
entrypoint here and you can get further by listing files and directories
from it?  But maybe people then use the filesystem directly to find a
directory and create the directories ad-hoc.

I grepped lieer but I think you only use it in one place?  And if I
understand it correctly you only do this to check if your mailstore/cwd
is inside the notmuch database.  I.e. this is equivalent to checking if
your mailstore/cwd has notmuch2.Database.path as prefix which you could
easily do directly rather than using the FileError exception from
.get_directory().

So is anyone else aware of some code which uses db.get_directory() to
give an idea of how and why this is used?

> I also ran into a couple of warning when building
> (included below).

Thanks for pointing these out.  I guess if the bindings are in the main
repo only the latest library version can be supported without any
further concerns.

> By the way, it does not seem that the API is very far from the
> previous python API. If it is close enough, perhaps it is possible to
> get away with a bug version bump in the bindings rather than creating
> a new package. I understand the need for a new package, but it would
> be nice if we could avoid the future confusion of two python binding
> packages (if at all possible).

While I'm glad to hear that you think a migration wouldn't be to painful
for you I am very weary of knowingly breaking APIs.  I'd rather have
people have an easy migration rather than unexpected breakage after an
upgrade.


Cheers,
Floris
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Rename and small tweaks to wip/cffi

2019-11-17 Thread Floris Bruynooghe
Hi,

This is a patch against the wip/cffi branch.  It does the rename
which was discussed before and it adds a small improvment (to me)
to the test runner to show which notmuch version is being tested.

Let me know if this the right or wrong way to contribute to this
branch.

Cheers,
Floris


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2] Rename package to notmuch2

2019-11-17 Thread Floris Bruynooghe
This is based on a previous discussion on the list where this was more
or less seen as the least-bad option.
---
 .../{notdb => notmuch2}/__init__.py| 14 +++---
 .../python-cffi/{notdb => notmuch2}/_base.py   |  4 ++--
 .../python-cffi/{notdb => notmuch2}/_build.py  |  4 ++--
 .../{notdb => notmuch2}/_database.py   | 12 ++--
 .../python-cffi/{notdb => notmuch2}/_errors.py |  2 +-
 .../{notdb => notmuch2}/_message.py|  8 
 .../python-cffi/{notdb => notmuch2}/_query.py  | 10 +-
 .../python-cffi/{notdb => notmuch2}/_tags.py   |  6 +++---
 .../python-cffi/{notdb => notmuch2}/_thread.py | 10 +-
 bindings/python-cffi/setup.py  |  4 ++--
 bindings/python-cffi/tests/test_base.py|  4 ++--
 bindings/python-cffi/tests/test_database.py| 14 +++---
 bindings/python-cffi/tests/test_message.py | 14 +++---
 bindings/python-cffi/tests/test_tags.py|  4 ++--
 bindings/python-cffi/tests/test_thread.py  | 18 +-
 bindings/python-cffi/tox.ini   |  4 ++--
 16 files changed, 66 insertions(+), 66 deletions(-)
 rename bindings/python-cffi/{notdb => notmuch2}/__init__.py (90%)
 rename bindings/python-cffi/{notdb => notmuch2}/_base.py (99%)
 rename bindings/python-cffi/{notdb => notmuch2}/_build.py (99%)
 rename bindings/python-cffi/{notdb => notmuch2}/_database.py (99%)
 rename bindings/python-cffi/{notdb => notmuch2}/_errors.py (99%)
 rename bindings/python-cffi/{notdb => notmuch2}/_message.py (99%)
 rename bindings/python-cffi/{notdb => notmuch2}/_query.py (93%)
 rename bindings/python-cffi/{notdb => notmuch2}/_tags.py (99%)
 rename bindings/python-cffi/{notdb => notmuch2}/_thread.py (97%)

diff --git a/bindings/python-cffi/notdb/__init__.py 
b/bindings/python-cffi/notmuch2/__init__.py
similarity index 90%
rename from bindings/python-cffi/notdb/__init__.py
rename to bindings/python-cffi/notmuch2/__init__.py
index 67051df5..4d76ec15 100644
--- a/bindings/python-cffi/notdb/__init__.py
+++ b/bindings/python-cffi/notmuch2/__init__.py
@@ -42,13 +42,13 @@ usually more and essentially O(n) rather than O(1) as you 
might
 usually expect from Python containers.
 """
 
-from notdb import _capi
-from notdb._base import *
-from notdb._database import *
-from notdb._errors import *
-from notdb._message import *
-from notdb._tags import *
-from notdb._thread import *
+from notmuch2 import _capi
+from notmuch2._base import *
+from notmuch2._database import *
+from notmuch2._errors import *
+from notmuch2._message import *
+from notmuch2._tags import *
+from notmuch2._thread import *
 
 
 NOTMUCH_TAG_MAX = _capi.lib.NOTMUCH_TAG_MAX
diff --git a/bindings/python-cffi/notdb/_base.py 
b/bindings/python-cffi/notmuch2/_base.py
similarity index 99%
rename from bindings/python-cffi/notdb/_base.py
rename to bindings/python-cffi/notmuch2/_base.py
index acb64413..31258149 100644
--- a/bindings/python-cffi/notdb/_base.py
+++ b/bindings/python-cffi/notmuch2/_base.py
@@ -1,8 +1,8 @@
 import abc
 import collections.abc
 
-from notdb import _capi as capi
-from notdb import _errors as errors
+from notmuch2 import _capi as capi
+from notmuch2 import _errors as errors
 
 
 __all__ = ['NotmuchObject', 'BinString']
diff --git a/bindings/python-cffi/notdb/_build.py 
b/bindings/python-cffi/notmuch2/_build.py
similarity index 99%
rename from bindings/python-cffi/notdb/_build.py
rename to bindings/python-cffi/notmuch2/_build.py
index 6be7e5b1..3ba3e558 100644
--- a/bindings/python-cffi/notdb/_build.py
+++ b/bindings/python-cffi/notmuch2/_build.py
@@ -3,14 +3,14 @@ import cffi
 
 ffibuilder = cffi.FFI()
 ffibuilder.set_source(
-'notdb._capi',
+'notmuch2._capi',
 r"""
 #include 
 #include 
 #include 
 
 #if LIBNOTMUCH_MAJOR_VERSION < 5
-#error libnotmuch version not supported by notdb
+#error libnotmuch version not supported by notmuch2 python bindings
 #endif
 """,
 include_dirs=['../../lib'],
diff --git a/bindings/python-cffi/notdb/_database.py 
b/bindings/python-cffi/notmuch2/_database.py
similarity index 99%
rename from bindings/python-cffi/notdb/_database.py
rename to bindings/python-cffi/notmuch2/_database.py
index d414082a..a15c4d03 100644
--- a/bindings/python-cffi/notdb/_database.py
+++ b/bindings/python-cffi/notmuch2/_database.py
@@ -6,12 +6,12 @@ import os
 import pathlib
 import weakref
 
-import notdb._base as base
-import notdb._capi as capi
-import notdb._errors as errors
-import notdb._message as message
-import notdb._query as querymod
-import notdb._tags as tags
+import notmuch2._base as base
+import notmuch2._capi as capi
+import notmuch2._errors as errors
+import notmuch2._message as message
+import notmuch2._query as querymod
+import notmuch2._tags as tags
 
 
 __all__ = ['Database', 'AtomicContext', 'DbRevision']
diff --git a/bindings/python-cffi/notdb/_errors.py 
b/bindings/python-cffi/notmuch2/_errors.py
similarity index 99%
rename 

[PATCH 1/2] Show which notmuch command and version is being used

2019-11-17 Thread Floris Bruynooghe
This add the notmuch version and absolute path of the binary used
in the pytest header.  This is nice when running the tests
interactively as you get confirmation you're testing the version you
thought you were testing.
---
 bindings/python-cffi/tests/conftest.py | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/bindings/python-cffi/tests/conftest.py 
b/bindings/python-cffi/tests/conftest.py
index aa940947..674c7218 100644
--- a/bindings/python-cffi/tests/conftest.py
+++ b/bindings/python-cffi/tests/conftest.py
@@ -10,6 +10,13 @@ import os
 import pytest
 
 
+def pytest_report_header():
+vers = subprocess.run(['notmuch', '--version'], stdout=subprocess.PIPE)
+which = subprocess.run(['which', 'notmuch'], stdout=subprocess.PIPE)
+return ['{} ({})'.format(vers.stdout.decode(errors='replace').strip(),
+ which.stdout.decode(errors='replace').strip())]
+
+
 @pytest.fixture(scope='function')
 def tmppath(tmpdir):
 """The tmpdir fixture wrapped in pathlib.Path."""
-- 
2.24.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: python CFFI bindings integration into notmuch build/test

2019-11-17 Thread David Bremner
Floris Bruynooghe  writes:

>
> I still have a bunch of non-existing messages with weird unicode message
> IDs in them, but no associated filename just an empty string.  AFAIK
> notmuch_database_remove_message() needs a filename though, I've tried
> removing the empty file which does not fail but doesn't clear those
> database entries.  The messages are not ghosts either.  Any idea how I
> might be able to clear those bad messages from the database?

In principle running "notmuch new" should do it, if there are no files
on disk corresponding to those messages. If that doesn't work, you might
have to use "xapian-delve" to figure out what terms are attached to
those message documents.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: python CFFI bindings integration into notmuch build/test

2019-11-17 Thread Floris Bruynooghe
On Sat 16 Nov 2019 at 10:51 -0500, David Bremner wrote:

> Floris Bruynooghe  writes:
>> Anyway, this looks good.  Would you like some changes, e.g. the rename
>> to notmuch2 or so as patches?  What's the next step.
>
> If you could look at the rename that would be great. Patches on top of
> wip/cffi, or a ref for me to pull both sound fine.
>
>> Kind of unrelated, but in my attempt to run the full notmuch test suite
>> (cd tests; make test) I somehow ended up with lots of weird tags in my
>> notmuch database and made the emacs UI pretty horrible - though it seems
>> I haven't lost any email.  I'm sure that was my fault somehow even
>> though I was just trying to follow the readme.  But if anyone has any
>> hints how to recover from this that could save me some time :)
>
> Hmm. I've never encountered that, but maybe it has to do with killing
> the environment? The test harness uses the environment variable
> NOTMUCH_CONFIG to point to a test database, and if that was deleted, it
> would look in ~/.notmuch-config.

Yeah, I probably managed to make it use ~/.notmuch-config somehow.

> Still that would really only be tags from your pytest tests, and you'd
> probably recognize those?

The were a bunch of weird characters that looked like unicode
test-cases, the pytest tests don't try to test notmuch only the bindings
so doesn't really make up weird strings IIRC.  I recovered with
something like "notmuch dump | cleanup_tags.py | notmuch restore", oh
well.

I still have a bunch of non-existing messages with weird unicode message
IDs in them, but no associated filename just an empty string.  AFAIK
notmuch_database_remove_message() needs a filename though, I've tried
removing the empty file which does not fail but doesn't clear those
database entries.  The messages are not ghosts either.  Any idea how I
might be able to clear those bad messages from the database?

Cheers,
Floris
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: notmuch ignoring alot of emails

2019-11-17 Thread David Bremner
David Bremner  writes:

> Eirik Byrkjeflot Anonsen  writes:
>
>>
>> Or, notmuch could just look at the first line of the file. If it starts
>> with "From ", it is an mbox. If it starts with a reasonable mail header,
>> it is not an mbox. If it is neither, fall back to the old heuristics.
>>
>
> FTR, this is what happens now (although iirc the actual check is done by
> GMime). So maybe I'm missing some context here.

Re-reading, I guess your point is maybe that we should ignore successive
unescaped "From "?  We're really trapped a bit by wanting to support
single message mboxes. We tried to remove this support at one point, but
this caused so much problems we put it back.

d

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: can a notmuch query filter the output based on the size of a thread?

2019-11-17 Thread David Bremner
Daniel Kahn Gillmor  writes:

> Just to flesh it out a bit further, do you think that such a thread
> object would have anything attached to it other than message-count? or
> maybe a list of message-ids?  dates?  aggregated senders?  aggregated
> recipients?

I think it would have the union of terms (tags, words, prefixed terms
from e.g. subject) in messages in the thread to allow
"thread:{foo AND bar}" to be done fast. At one point Olly convinced me
that it would be feasible to delete the terms in a sensible way (i.e. to
simulate reference counting) when a message is deleted, but I'm hazy on
the details at this point. dates might be especially tricky.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: notmuch ignoring alot of emails

2019-11-17 Thread David Bremner
Eirik Byrkjeflot Anonsen  writes:

>
> Or, notmuch could just look at the first line of the file. If it starts
> with "From ", it is an mbox. If it starts with a reasonable mail header,
> it is not an mbox. If it is neither, fall back to the old heuristics.
>

FTR, this is what happens now (although iirc the actual check is done by
GMime). So maybe I'm missing some context here.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Tests failing on master

2019-11-17 Thread David Bremner
Johan Parin  writes:

> Johan Parin  writes:
>
>> In order to be able to run some tests and since OS X Mojave only has
>> bash 3.2, I installed a Kali vm, downloaded a fresh notmuch repo,
>> installed the dependencies and did make test. Result:
>>
>> 1123/1128 tests passed.
>> 3 broken tests failed as expected.
>> 2 tests failed.
>>
>
> I tried on another machine running Ubuntu Server 18.04.3, and there all
> 1128 tests minus the broken ones passed.
>
> Could the failures on Kali be related to me running as root?

Yes, being root would break some of the tests, e.g. those that rely on
removing permissions to generate an error. Probably we should just
document that running the test suite as root is not supported? If
someone can explain a good reason to run as root (maybe some odd
auto-builder, the tests in question could probably be modified to do
something more extreme (e.g. delete a file) rather than chmod.

cheers,

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Tests failing on master

2019-11-17 Thread Johan Parin


Johan Parin  writes:

> In order to be able to run some tests and since OS X Mojave only has
> bash 3.2, I installed a Kali vm, downloaded a fresh notmuch repo,
> installed the dependencies and did make test. Result:
>
> 1123/1128 tests passed.
> 3 broken tests failed as expected.
> 2 tests failed.
>

I tried on another machine running Ubuntu Server 18.04.3, and there all
1128 tests minus the broken ones passed.

Could the failures on Kali be related to me running as root?


/Johan
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: can a notmuch query filter the output based on the size of a thread?

2019-11-17 Thread Daniel Kahn Gillmor
On Sat 2019-11-16 10:38:44 -0500, David Bremner wrote:
> Alternatively someone (TM) could add thread objects to the notmuch
> database. This would allow much faster thread operations, but would
> require some fairly delicate maintenance of the thread
> objects. Basically they would need to updated every time a message is
> added or removed from the database, and potentially garbage collected.

This is a pretty interesting idea (though not something i'm likely to
get around to in the near future personally).

Just to flesh it out a bit further, do you think that such a thread
object would have anything attached to it other than message-count? or
maybe a list of message-ids?  dates?  aggregated senders?  aggregated
recipients?

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: unifying and automating draft ("postpone/resume") behavior [was: Re: notmuch "lost" email during power failure]

2019-11-17 Thread Daniel Kahn Gillmor
On Fri 2019-11-15 10:08:02 -0500, Antoine Beaupré wrote:
> The individual calls are not what's prohibitively expensive. The problem
> is latency. Even if it just takes 100ms[1] to save the file to the disk,
> that's a huge delay in terms of human interface, because the entire
> Emacs UI will *freeze* for that amount of time whenever it thinks it
> should do an autosave.

Surely the problem there is that the Emacs UI is freezing and blocking
to do something that has nothing to do with interactive work!

Is it not possible in 2019 to have emacs handle its autosave
functionality in either a background thread, or in a non-blocking
asynchronous interaction with a co-process?

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Tests failing on master

2019-11-17 Thread Johan Parin


In order to be able to run some tests and since OS X Mojave only has
bash 3.2, I installed a Kali vm, downloaded a fresh notmuch repo,
installed the dependencies and did make test. Result:

1123/1128 tests passed.
3 broken tests failed as expected.
2 tests failed.

More details:

T050-new: Testing "notmuch new" in several variations
 FAIL   Xapian exception: read only files
--- T050-new.35.expected2019-11-17 10:05:48.060026443 +
+++ T050-new.35.output  2019-11-17 10:05:48.060026443 +
@@ -1 +1,17 @@
-A Xapian exception occurred opening database

T150-tagging: Testing "notmuch tag"
 FAIL   Xapian exception: read only files
--- T150-tagging.25.expected2019-11-17 10:05:57.536026481 +
+++ T150-tagging.25.output  2019-11-17 10:05:57.536026481 +
@@ -1 +1 @@
-A Xapian exception occurred opening database
+

Can anyone confirm / deny that this should occur on master?

I looked at the tests/README and installed the prerequisite packages,
and also downloaded the test databases.


/Johan
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch