Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

[Daniel Kahn Gillmor]

On Thu 2017-12-07 00:20:12 -0800, Jameson Graef Rollins wrote:
> On Mon, Dec 04 2017, David Bremner  wrote:
>> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
>> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
>> at those.
>
> Yes, this new series is great and definitely addresses all my concerns.
> I'm stoked to see that the first part of it has been pushed, and looking
> forward to the full series!
>
> This is really great progress, Daniel.  Thanks for pushing on this.

Thanks for the review!  I've just pushed v3 of what remains of this
series, which is basically the same as the remaining patches here, with
a couple minor cleanups.

you can find it starting at
id:20171208062404.17269-1-...@fifthhorseman.net

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] crypto: signature verification reports valid User IDs

[Daniel Kahn Gillmor]

On Wed 2017-11-29 23:20:35 -0500, Daniel Kahn Gillmor wrote:
> When i'm trying to understand a message signature, i care that i know
> who it came from (the "validity" of the identity associated with the
> key), *not* whether i'm willing to accept the keyholder's other
> identity assertions (the "trust" associated with the certificate).
>
> We've been reporting User ID information based on the "trust"
> associated with the certificate, because GMime didn't clearly expose
> the validity of the User IDs.
>
> This change relies on fixes made in GMime 3.0.3 and later which
> include https://github.com/jstedfast/gmime/pull/18.

Without this patch, the notmuch test suite currently fails when built
against gmime 3 on debian testing, because gmime 3 on debian testing
now correctly returns key IDs for _get_key_id().

This patch corrects the misbehavior of previous versions of notmuch, and
restores the ability to display correct validity of the keys in
question.

I welcome review for it!

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 12/15] crypto: actually stash session keys when decrypt=true

[Daniel Kahn Gillmor]

If you're going to store the cleartext index of an encrypted message,
in most situations you might just as well store the session key.
Doing this storage has efficiency and recoverability advantages.

Combined with a schedule of regular OpenPGP subkey rotation and
destruction, this can also offer security benefits, like "deletable
e-mail", which is the store-and-forward analog to "forward secrecy".

But wait, i hear you saying, i have a special need to store cleartext
indexes but it's really bad for me to store session keys!  Maybe
(let's imagine) i get lots of e-mails with incriminating photos
attached, and i want to be able to search for them by the text in the
e-mail, but i don't want someone with access to the index to be
actually able to see the photos themselves.

Fret not, the next patch in this series will support your wacky
uncommon use case.
---
 doc/man1/notmuch-config.rst | 10 ++
 doc/man1/notmuch-insert.rst | 13 +++--
 doc/man1/notmuch-new.rst| 21 +++--
 doc/man1/notmuch-reindex.rst| 10 +-
 doc/man7/notmuch-properties.rst |  4 
 lib/index.cc| 18 +-
 test/T357-index-decryption.sh   | 24 +++-
 util/crypto.c   | 16 +++-
 8 files changed, 88 insertions(+), 28 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 1a9e08a3..dabf269f 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -143,10 +143,12 @@ The available configuration items are described below.
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
-index the cleartext.  If ``auto``, it will try to index the
-cleartext if a stashed session key is already known for the message,
-but will not try to access your secret keys.  Use ``false`` to
-avoid decrypting even when a session key is already known.
+index the cleartext, stashing a copy of any discovered session
+keys for the message.  If ``auto``, it will try to index the
+cleartext if a stashed session key is already known for the message
+(e.g. from a previous copy), but will not try to access your
+secret keys.  Use ``false`` to avoid decrypting even when a
+stashed session key is already present.
 
 Be aware that the notmuch index is likely sufficient to
 reconstruct the cleartext of the message itself, so please
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index b22be863..214f261b 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -54,12 +54,13 @@ Supported options for **insert** include
 ``--decrypt=(true|auto|false)``
 
 If ``true`` and the message is encrypted, try to decrypt the
-message while indexing.  If ``auto``, and notmuch already
-knows about a session key for the message, it will try
-decrypting using that session key but will not try to access
-the user's secret keys.  If decryption is successful, index
-the cleartext itself.  Either way, the message is always
-stored to disk in its original form (ciphertext).
+message while indexing, storing any session keys discovered.
+If ``auto``, and notmuch already knows about a session key for
+the message, it will try decrypting using that session key but
+will not try to access the user's secret keys.  If decryption
+is successful, index the cleartext itself.  Either way, the
+message is always stored to disk in its original form
+(ciphertext).
 
 Be aware that the index is likely sufficient to reconstruct
 the cleartext of the message itself, so please ensure that the
diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
index 71df31d7..27676a19 100644
--- a/doc/man1/notmuch-new.rst
+++ b/doc/man1/notmuch-new.rst
@@ -46,16 +46,17 @@ Supported options for **new** include
 ``--decrypt=(true|auto|false)``
 
 If ``true``, when encountering an encrypted message, try to
-decrypt it while indexing.  If decryption is successful, index
-the cleartext itself.  If ``auto``, try to use any session key
-already known to belong to this message, but do not attempt to
-use the user's secret keys.
-
-Be aware that the index is likely
-sufficient to reconstruct the cleartext of the message itself,
-so please ensure that the notmuch message index is adequately
-protected.  DO NOT USE ``--decrypt=true`` without
-considering the security of your index.
+decrypt it while indexing, and store any discovered session
+keys.  If ``auto``, try to use any session key already known
+to belong to this message, but do not attempt to use 

[PATCH v3 10/15] cli/new, insert, reindex: change index.decrypt to "auto" by default

[Daniel Kahn Gillmor]

The new "auto" decryption policy is not only good for "notmuch show"
and "notmuch reindex".  It's also useful for indexing messages --
there's no good reason to not try to go ahead and index the cleartext
of a message that we have a stashed session key for.

This change updates the defaults and tunes the test suite to make sure
that they have taken effect.
---
 doc/man1/notmuch-config.rst   | 2 +-
 lib/indexopts.c   | 8 +---
 test/T357-index-decryption.sh | 4 ++--
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 4835f897..1a9e08a3 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -154,7 +154,7 @@ The available configuration items are described below.
 DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
 
-Default: ``false``.
+Default: ``auto``.
 
 **built_with.**
 
diff --git a/lib/indexopts.c b/lib/indexopts.c
index a04d1c1c..26a31e89 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -26,7 +26,7 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 notmuch_indexopts_t *ret = talloc_zero (db, notmuch_indexopts_t);
 if (!ret)
return ret;
-ret->crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
+ret->crypto.decrypt = NOTMUCH_DECRYPT_AUTO;
 
 char * decrypt_policy;
 notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
_policy);
@@ -38,8 +38,10 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
(!(strcasecmp(decrypt_policy, "yes"))) ||
(!(strcasecmp(decrypt_policy, "1"
notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
-   else if (!strcasecmp(decrypt_policy, "auto"))
-   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_AUTO);
+   else if ((!(strcasecmp(decrypt_policy, "false"))) ||
+(!(strcasecmp(decrypt_policy, "no"))) ||
+(!(strcasecmp(decrypt_policy, "0"
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_FALSE);
 }
 
 free (decrypt_policy);
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index c3730488..bd213415 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -142,7 +142,7 @@ test_expect_equal \
 
 # ensure no session keys are present:
 test_begin_subtest 'reindex using only session keys'
-test_expect_success 'notmuch reindex --decrypt=auto tag:encrypted and 
property:index.decryption=success'
+test_expect_success 'notmuch reindex tag:encrypted and 
property:index.decryption=success'
 test_begin_subtest "reindexed encrypted messages, decrypting only with session 
keys"
 output=$(notmuch search wumpus)
 expected=''
@@ -190,7 +190,7 @@ notmuch restore 

[PATCH v3 08/15] cli/new, insert, reindex: update documentation for --decrypt=auto

[Daniel Kahn Gillmor]

we also include --decrypt=auto in the tab completion.
---
 completion/notmuch-completion.bash |  6 +++---
 doc/man1/notmuch-insert.rst| 16 ++--
 doc/man1/notmuch-new.rst   | 10 +++---
 doc/man1/notmuch-reindex.rst   | 23 ++-
 4 files changed, 34 insertions(+), 21 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index f94dbeed..272131e6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -288,7 +288,7 @@ _notmuch_insert()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
@@ -320,7 +320,7 @@ _notmuch_new()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
@@ -442,7 +442,7 @@ _notmuch_reindex()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index eb9ff11b..b22be863 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -51,14 +51,18 @@ Supported options for **insert** include
 ``--no-hooks``
 Prevent hooks from being run.
 
-``--decrypt=(true|false)``
+``--decrypt=(true|auto|false)``
 
-If true and the message is encrypted, try to decrypt the
-message while indexing.  If decryption is successful, index
+If ``true`` and the message is encrypted, try to decrypt the
+message while indexing.  If ``auto``, and notmuch already
+knows about a session key for the message, it will try
+decrypting using that session key but will not try to access
+the user's secret keys.  If decryption is successful, index
 the cleartext itself.  Either way, the message is always
-stored to disk in its original form (ciphertext).  Be aware
-that the index is likely sufficient to reconstruct the
-cleartext of the message itself, so please ensure that the
+stored to disk in its original form (ciphertext).
+
+Be aware that the index is likely sufficient to reconstruct
+the cleartext of the message itself, so please ensure that the
 notmuch message index is adequately protected. DO NOT USE
 ``--decrypt=true`` without considering the security of
 your index.
diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst
index 1df86f06..71df31d7 100644
--- a/doc/man1/notmuch-new.rst
+++ b/doc/man1/notmuch-new.rst
@@ -43,11 +43,15 @@ Supported options for **new** include
 ``--quiet``
 Do not print progress or results.
 
-``--decrypt=(true|false)``
+``--decrypt=(true|auto|false)``
 
-If true, when encountering an encrypted message, try to
+If ``true``, when encountering an encrypted message, try to
 decrypt it while indexing.  If decryption is successful, index
-the cleartext itself.  Be aware that the index is likely
+the cleartext itself.  If ``auto``, try to use any session key
+already known to belong to this message, but do not attempt to
+use the user's secret keys.
+
+Be aware that the index is likely
 sufficient to reconstruct the cleartext of the message itself,
 so please ensure that the notmuch message index is adequately
 protected.  DO NOT USE ``--decrypt=true`` without
diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst
index 782b0d7b..d87e9d85 100644
--- a/doc/man1/notmuch-reindex.rst
+++ b/doc/man1/notmuch-reindex.rst
@@ -21,15 +21,20 @@ messages using the supplied options.
 
 Supported options for **reindex** include
 
-``--decrypt=(true|false)``
-
-If true, when encountering an encrypted message, try to
-decrypt it while reindexing.  If decryption is successful,
-index the cleartext itself.  Be aware that the index is likely
-sufficient to reconstruct the cleartext of the message itself,
-so please ensure that the notmuch message index is adequately
-protected. DO NOT USE ``--decrypt=true`` without
-considering the security of your index.
+``--decrypt=(true|auto|false)``
+
+If ``true``, when encountering an encrypted message, try to
+decrypt it while reindexing.  If ``auto``, and notmuch already
+knows about a session key for the message, it will try
+decrypting using that session key but will not try to access
+the user's secret keys.  If 

[PATCH v3 14/15] docs: clean up documentation about decryption policies

[Daniel Kahn Gillmor]

Now that the range of sensible decryption policies has come into full
view, we take a bit of space to document the distinctions.

Most people will use either "auto" or "true" -- but we provide "false"
and "nostash" to handle use cases that might reasonably be requested.

Note also that these can be combined in sensible ways.  Like, if your
mail comes in regularly to a service that doesn't have access to your
secret keys, but does have access to your index, and you feel
comfortable adding selected encrypted messages to the index after
you've read them, you could stay in "auto" normally, and then when you
find yourself reading an indexable message (e.g. one you want to be
able to search for in the future, and that you don't mind exposing to
whatever entities have access to your inde), you can do:

notmuch reindex --decrypt=true id:whate...@example.biz

That leaves your default the same (still "auto") but you get the
cleartext index and stashed session key benefits for that particular
message.
---
 doc/man1/notmuch-config.rst | 33 -
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 773fd9da..3ba849b2 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,7 +142,9 @@ The available configuration items are described below.
 
 **[STORED IN DATABASE]**
 
-One of ``false``, ``auto``, ``nostash``, or ``true``.
+Policy for decrypting encrypted messages during indexing.
+Must be one of: ``false``, ``auto``, ``nostash``, or
+``true``.
 
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
@@ -156,6 +158,34 @@ The available configuration items are described below.
 ``nostash`` is the same as ``true`` except that it will not
 stash newly-discovered session keys in the database.
 
+From the command line (i.e. during **notmuch-new(1)**,
+**notmuch-insert(1)**, or **notmuch-reindex(1)**), the user
+can override the database's stored decryption policy with the
+``--decrypt=`` option.
+
+Here is a table that summarizes the functionality of each of
+these policies:
+
+++---+--+-+--+
+|| false | auto | nostash | true |
+++===+==+=+==+
+| Index cleartext using  |   |  X   |X|  X   |
+| stashed session keys   |   |  | |  |
+++---+--+-+--+
+| Index cleartext|   |  |X|  X   |
+| using secret keys  |   |  | |  |
+++---+--+-+--+
+| Stash session keys |   |  | |  X   |
+++---+--+-+--+
+| Delete stashed session |   X   |  | |  |
+| keys on reindex|   |  | |  |
+++---+--+-+--+
+
+Stashed session keys are kept in the database as properties
+associated with the message.  See ``session-key`` in
+**notmuch-properties(7)** for more details about how they can
+be useful.
+
 Be aware that the notmuch index is likely sufficient (and a
 stashed session key is certainly sufficient) to reconstruct
 the cleartext of the message itself, so please ensure that the
@@ -201,5 +231,6 @@ SEE ALSO
 **notmuch-restore(1)**,
 **notmuch-search(1)**,
 **notmuch-search-terms(7)**,
+**notmuch-properties(7)**,
 **notmuch-show(1)**,
 **notmuch-tag(1)**
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 15/15] python: add decrypt_policy argument to Database.index_file()

[Daniel Kahn Gillmor]

We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.
---
 bindings/python/notmuch/database.py | 45 +++--
 bindings/python/notmuch/globals.py  |  5 +
 2 files changed, 48 insertions(+), 2 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 1279804a..2a07e346 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -28,6 +28,7 @@ from .globals import (
 _str,
 NotmuchDatabaseP,
 NotmuchDirectoryP,
+NotmuchIndexoptsP,
 NotmuchMessageP,
 NotmuchTagsP,
 )
@@ -72,6 +73,9 @@ class Database(object):
 MODE = Enum(['READ_ONLY', 'READ_WRITE'])
 """Constants: Mode in which to open the database"""
 
+DECRYPTION_POLICY = Enum(['FALSE', 'TRUE', 'AUTO', 'NOSTASH'])
+"""Constants: policies for decrypting messages during indexing"""
+
 """notmuch_database_get_directory"""
 _get_directory = nmlib.notmuch_database_get_directory
 _get_directory.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchDirectoryP)]
@@ -400,13 +404,25 @@ class Database(object):
 # return the Directory, init it with the absolute path
 return Directory(abs_dirpath, dir_p, self)
 
+_get_default_indexopts = nmlib.notmuch_database_get_default_indexopts
+_get_default_indexopts.argtypes = [NotmuchDatabaseP]
+_get_default_indexopts.restype = NotmuchIndexoptsP
+
+_indexopts_set_decrypt_policy = nmlib.notmuch_indexopts_set_decrypt_policy
+_indexopts_set_decrypt_policy.argtypes = [NotmuchIndexoptsP, c_uint]
+_indexopts_set_decrypt_policy.restype = None
+
+_indexopts_destroy = nmlib.notmuch_indexopts_destroy
+_indexopts_destroy.argtypes = [NotmuchIndexoptsP]
+_indexopts_destroy.restype = None
+
 _index_file = nmlib.notmuch_database_index_file
 _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
  c_void_p,
  POINTER(NotmuchMessageP)]
 _index_file.restype = c_uint
 
-def index_file(self, filename, sync_maildir_flags=False):
+def index_file(self, filename, sync_maildir_flags=False, 
decrypt_policy=None):
 """Adds a new message to the database
 
 :param filename: should be a path relative to the path of the
@@ -427,6 +443,23 @@ class Database(object):
 API. You might want to look into the underlying method
 :meth:`Message.maildir_flags_to_tags`.
 
+:param decrypt_policy: If the message contains any encrypted
+parts, and decrypt_policy is set to
+:attr:`DECRYPTION_POLICY`.TRUE, notmuch will try to
+decrypt the message and index the cleartext, stashing any
+discovered session keys.  If it is set to
+:attr:`DECRYPTION_POLICY`.FALSE, it will never try to
+decrypt during indexing.  If it is set to
+:attr:`DECRYPTION_POLICY`.AUTO, then it will try to use
+any stashed session keys it knows about, but will not try
+to access the user's secret keys.
+:attr:`DECRYPTION_POLICY`.NOSTASH behaves the same as
+:attr:`DECRYPTION_POLICY`.TRUE except that no session keys
+are stashed in the database.  If decrypt_policy is set to
+None (the default), then the database itself will decide
+whether to decrypt, based on the `index.decrypt`
+configuration setting (see notmuch-config(1)).
+
 :returns: On success, we return
 
1) a :class:`Message` object that can be used for things
@@ -457,7 +490,15 @@ class Database(object):
 """
 self._assert_db_is_initialized()
 msg_p = NotmuchMessageP()
-status = self._index_file(self._db, _str(filename), c_void_p(None), 
byref(msg_p))
+indexopts = c_void_p(None)
+if decrypt_policy is not None:
+indexopts = self._get_default_indexopts(self._db)
+self._indexopts_set_decrypt_policy(indexopts, decrypt_policy)
+
+status = self._index_file(self._db, _str(filename), indexopts, 
byref(msg_p))
+
+if indexopts:
+self._indexopts_destroy(indexopts)
 
 if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
 raise NotmuchError(status)
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index b1eec2cf..71426c84 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -88,3 +88,8 @@ NotmuchDirectoryP = POINTER(NotmuchDirectoryS)
 class NotmuchFilenamesS(Structure):
 pass
 NotmuchFilenamesP = POINTER(NotmuchFilenamesS)
+
+
+class NotmuchIndexoptsS(Structure):
+pass
+NotmuchIndexoptsP = POINTER(NotmuchIndexoptsS)
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org

[PATCH v3 06/15] cli/show: use decryption policy "auto" by default.

[Daniel Kahn Gillmor]

When showing a message, if the user doesn't specify --decrypt= at all,
but a stashed session key is known to notmuch, notmuch should just go
ahead and try to decrypt the message with the session key (without
bothering the user for access to their asymmetric secret key).

The user can disable this at the command line with --decrypt=false if
they really don't want to look at the e-mail that they've asked
notmuch to show them.

and of course, "notmuch show --decrypt" still works for accessing the
user's secret keys if necessary.
---
 completion/notmuch-completion.bash |  4 ++--
 doc/man1/notmuch-show.rst  |  4 
 notmuch-show.c | 17 +++--
 test/T357-index-decryption.sh  | 17 +
 4 files changed, 34 insertions(+), 8 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 1cd616b3..f94dbeed 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,7 +517,7 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body)
+   --exclude|--body|--decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -526,7 +526,7 @@ _notmuch_show()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--entire-thread= --format= --exclude= --body= 
--format-version= --part= --verify --decrypt --include-html 
${_notmuch_shared_options}"
+   local options="--entire-thread= --format= --exclude= --body= 
--format-version= --part= --verify --decrypt= --include-html 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 7ba091cf..64caa7a6 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -123,6 +123,10 @@ Supported options for **show** include
 multipart/encrypted part will be replaced by the decrypted
 content.
 
+If a session key is already known for the message, then it
+will be decrypted automatically unless the user explicitly
+sets ``--decrypt=false``.
+
 Decryption expects a functioning **gpg-agent(1)** to provide any
 needed credentials. Without one, the decryption will fail.
 
diff --git a/notmuch-show.c b/notmuch-show.c
index e840a470..591889a9 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1105,13 +1105,14 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
.part = -1,
.omit_excluded = true,
.output_body = true,
-   .crypto = { .decrypt = NOTMUCH_DECRYPT_FALSE },
+   .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
 };
 int format = NOTMUCH_FORMAT_NOT_SPECIFIED;
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
 bool decrypt = false;
+bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -1126,7 +1127,7 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
{ .opt_bool = _thread, .name = "entire-thread",
  .present = _thread_set },
{ .opt_int = , .name = "part" },
-   { .opt_bool = , .name = "decrypt" },
+   { .opt_bool = , .name = "decrypt", .present = _set },
{ .opt_bool = , .name = "verify" },
{ .opt_bool = _body, .name = "body" },
{ .opt_bool = _html, .name = "include-html" },
@@ -1140,10 +1141,14 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
-if (decrypt) {
-   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
-   /* decryption implies verification */
-   params.crypto.verify = true;
+if (decrypt_set) {
+   if (decrypt) {
+   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
+   /* decryption implies verification */
+   params.crypto.verify = true;
+   } else {
+   params.crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
+   }
 }
 
 /* specifying a part implies single message display */
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 31991e22..c3730488 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -210,6 +210,23 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "notmuch show should show cleartext if session key is 
present"
+output=$(notmuch show id:simple-encryp...@crypto.notmuchmail.org | awk 
'/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: 3/{ f=1 }')
+expected='This is a top sekrit message.'
+if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
+test_subtest_known_broken
+fi
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtest 

[PATCH v3 04/15] crypto: new decryption policy "auto"

[Daniel Kahn Gillmor]

This new automatic decryption policy should make it possible to
decrypt messages that we have stashed session keys for, without
incurring a call to the user's asymmetric keys.
---
 doc/man1/notmuch-config.rst   | 11 ---
 lib/index.cc  |  3 ++-
 lib/indexopts.c   | 13 -
 lib/notmuch.h |  1 +
 mime-node.c   |  7 ---
 notmuch-client.h  |  4 +++-
 notmuch.c |  3 ++-
 test/T357-index-decryption.sh | 12 +++-
 util/crypto.c |  9 -
 util/crypto.h |  3 ++-
 10 files changed, 49 insertions(+), 17 deletions(-)

diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index ea3d9754..4835f897 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -142,9 +142,14 @@ The available configuration items are described below.
 
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
-set to true, notmuch will try to decrypt the message and index
-the cleartext.  Be aware that the index is likely sufficient
-to reconstruct the cleartext of the message itself, so please
+set to ``true``, notmuch will try to decrypt the message and
+index the cleartext.  If ``auto``, it will try to index the
+cleartext if a stashed session key is already known for the message,
+but will not try to access your secret keys.  Use ``false`` to
+avoid decrypting even when a session key is already known.
+
+Be aware that the notmuch index is likely sufficient to
+reconstruct the cleartext of the message itself, so please
 ensure that the notmuch message index is adequately protected.
 DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
diff --git a/lib/index.cc b/lib/index.cc
index 905366ae..af999bd3 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,7 +548,8 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (message, crypto_ctx, encrypted_data, 
NULL, );
+clear = _notmuch_crypto_decrypt (notmuch_indexopts_get_decrypt_policy 
(indexopts),
+message, crypto_ctx, encrypted_data, NULL, 
);
 if (err) {
_notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
   err->domain, err->code, err->message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index 78f53391..a04d1c1c 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -33,11 +33,14 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 if (err)
return ret;
 
-if (decrypt_policy &&
-   ((!(strcasecmp(decrypt_policy, "true"))) ||
-(!(strcasecmp(decrypt_policy, "yes"))) ||
-(!(strcasecmp(decrypt_policy, "1")
-   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
+if (decrypt_policy) {
+   if ((!(strcasecmp(decrypt_policy, "true"))) ||
+   (!(strcasecmp(decrypt_policy, "yes"))) ||
+   (!(strcasecmp(decrypt_policy, "1"
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
+   else if (!strcasecmp(decrypt_policy, "auto"))
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_AUTO);
+}
 
 free (decrypt_policy);
 return ret;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 47633496..ff860e06 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2241,6 +2241,7 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
 typedef enum {
 NOTMUCH_DECRYPT_FALSE,
 NOTMUCH_DECRYPT_TRUE,
+NOTMUCH_DECRYPT_AUTO,
 } notmuch_decryption_policy_t;
 
 /**
diff --git a/mime-node.c b/mime-node.c
index c4de708b..49d668fe 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -205,7 +205,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
break;
 
node->decrypt_attempted = true;
-   node->decrypted_child = _notmuch_crypto_decrypt (parent ? 
parent->envelope_file : NULL,
+   node->decrypted_child = _notmuch_crypto_decrypt 
(node->ctx->crypto->decrypt,
+parent ? 
parent->envelope_file : NULL,
 cryptoctx, 
encrypteddata, _result, );
 }
 if (! node->decrypted_child) {
@@ -270,7 +271,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 }
 
 #if (GMIME_MAJOR_VERSION < 3)
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt == 
NOTMUCH_DECRYPT_TRUE))
+if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt != 
NOTMUCH_DECRYPT_FALSE))
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_content_type (part);

session keys followup, version 3

[Daniel Kahn Gillmor]

The first part of the session-keys series has already landed --
thanks to everyone who reviewed it and shepherded it on its way!

This is a respin of the remaining patches in the series, introducing
very minor changes from the previous series: typo corrections, and a
fix to the test suite to mark session-key-dependent subtests as broken
when notmuch is built against an older version of GMime.

Happy hacking,

  --dkg

history: 

v1 was introduced in id:20171025065203.24403-1-...@fifthhorseman.net

v2 was introduced in id:20171130085946.11332-1-...@fifthhorseman.net

the first 6 patches of v2 have already landed, so the numbers in this
series are 6 less than previously ("v3 01/15" corresponds to "v2
07/21", etc)


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 03/15] lib: convert notmuch decryption policy to an enum

[Daniel Kahn Gillmor]

Future patches in this series will introduce new policies; this merely
readies the way for them.

We also convert --try-decrypt to a keyword argument instead of a boolean.
---
 lib/index.cc  |  2 +-
 lib/indexopts.c   | 21 +++--
 lib/notmuch.h | 14 --
 mime-node.c   |  4 ++--
 notmuch-client.h  |  9 +
 notmuch-reply.c   |  6 +-
 notmuch-show.c| 10 +++---
 notmuch.c | 13 -
 test/T357-index-decryption.sh | 12 ++--
 util/crypto.h |  2 +-
 10 files changed, 58 insertions(+), 35 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index ff14e408..905366ae 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -525,7 +525,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 notmuch_database_t * notmuch = NULL;
 GMimeObject *clear = NULL;
 
-if (!indexopts || !notmuch_indexopts_get_decrypt_policy (indexopts))
+if (!indexopts || (notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE))
return;
 
 notmuch = _notmuch_message_database (message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index 0f65b97c..78f53391 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -26,25 +26,26 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
 notmuch_indexopts_t *ret = talloc_zero (db, notmuch_indexopts_t);
 if (!ret)
return ret;
+ret->crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
 
-char * decrypt;
-notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
);
+char * decrypt_policy;
+notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", 
_policy);
 if (err)
return ret;
 
-if (decrypt &&
-   ((!(strcasecmp(decrypt, "true"))) ||
-(!(strcasecmp(decrypt, "yes"))) ||
-(!(strcasecmp(decrypt, "1")
-   notmuch_indexopts_set_decrypt_policy (ret, true);
+if (decrypt_policy &&
+   ((!(strcasecmp(decrypt_policy, "true"))) ||
+(!(strcasecmp(decrypt_policy, "yes"))) ||
+(!(strcasecmp(decrypt_policy, "1")
+   notmuch_indexopts_set_decrypt_policy (ret, NOTMUCH_DECRYPT_TRUE);
 
-free (decrypt);
+free (decrypt_policy);
 return ret;
 }
 
 notmuch_status_t
 notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
- notmuch_bool_t decrypt_policy)
+ notmuch_decryption_policy_t 
decrypt_policy)
 {
 if (!indexopts)
return NOTMUCH_STATUS_NULL_POINTER;
@@ -52,7 +53,7 @@ notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t 
*indexopts,
 return NOTMUCH_STATUS_SUCCESS;
 }
 
-notmuch_bool_t
+notmuch_decryption_policy_t
 notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts)
 {
 if (!indexopts)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index ef463090..47633496 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2233,6 +2233,16 @@ notmuch_config_list_destroy (notmuch_config_list_t 
*config_list);
 notmuch_indexopts_t *
 notmuch_database_get_default_indexopts (notmuch_database_t *db);
 
+/**
+ * Stating a policy about how to decrypt messages.
+ *
+ * See index.decrypt in notmuch-config(1) for more details.
+ */
+typedef enum {
+NOTMUCH_DECRYPT_FALSE,
+NOTMUCH_DECRYPT_TRUE,
+} notmuch_decryption_policy_t;
+
 /**
  * Specify whether to decrypt encrypted parts while indexing.
  *
@@ -2245,7 +2255,7 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
  */
 notmuch_status_t
 notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
- notmuch_bool_t decrypt_policy);
+ notmuch_decryption_policy_t 
decrypt_policy);
 
 /**
  * Return whether to decrypt encrypted parts while indexing.
@@ -2253,7 +2263,7 @@ notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t 
*indexopts,
  *
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
-notmuch_bool_t
+notmuch_decryption_policy_t
 notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts);
 
 /**
diff --git a/mime-node.c b/mime-node.c
index daaae9f8..c4de708b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -270,7 +270,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 }
 
 #if (GMIME_MAJOR_VERSION < 3)
-if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && node->ctx->crypto->decrypt)
+if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt == 
NOTMUCH_DECRYPT_TRUE))
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_content_type (part);
const char *protocol = g_mime_content_type_get_parameter (content_type, 
"protocol");
@@ -286,7 +286,7 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
 #endif
 
 /* Handle PGP/MIME parts 

[PATCH v3 11/15] cli/reindex: destroy stashed session keys when --decrypt=false

[Daniel Kahn Gillmor]

There are some situations where the user wants to get rid of the
cleartext index of a message.  For example, if they're indexing
encrypted messages normally, but suddenly they run across a message
that they really don't want any trace of in their index.

In that case, the natural thing to do is:

   notmuch reindex --decrypt=false id:whate...@example.biz

But of course, clearing the cleartext index without clearing the
stashed session key is just silly.  So we do the expected thing and
also destroy any stashed session keys while we're destroying the index
of the cleartext.

Note that stashed session keys are stored in the xapian database, but
xapian does not currently allow safe deletion (see
https://trac.xapian.org/ticket/742).

As a workaround, after removing session keys and cleartext material
from the database, the user probably should do something like "notmuch
compact" to try to purge whatever recoverable data is left in the
xapian freelist.  This problem really needs to be addressed within
xapian, though, if we want it fixed right.
---
 doc/man1/notmuch-reindex.rst  |  3 +++
 lib/message.cc|  5 +
 test/T357-index-decryption.sh | 17 +
 3 files changed, 25 insertions(+)

diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst
index d87e9d85..e8174f39 100644
--- a/doc/man1/notmuch-reindex.rst
+++ b/doc/man1/notmuch-reindex.rst
@@ -30,6 +30,9 @@ Supported options for **reindex** include
 the user's secret keys.  If decryption is successful, index
 the cleartext itself.
 
+If ``false``, notmuch reindex will also delete any stashed
+session keys for all messages matching the search terms.
+
 Be aware that the index is likely sufficient to reconstruct
 the cleartext of the message itself, so please ensure that the
 notmuch message index is adequately protected. DO NOT USE
diff --git a/lib/message.cc b/lib/message.cc
index 12743460..d5db89b6 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -2002,6 +2002,11 @@ notmuch_message_reindex (notmuch_message_t *message,
 ret = notmuch_message_remove_all_properties_with_prefix (message, 
"index.");
 if (ret)
goto DONE; /* XXX TODO: distinguish from other error returns above? */
+if (indexopts && notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE) {
+   ret = notmuch_message_remove_all_properties (message, "session-key");
+   if (ret)
+   goto DONE;
+}
 
 /* re-add the filenames with the associated indexopts */
 for (; notmuch_filenames_valid (orig_filenames);
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index bd213415..9f46a01b 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -227,6 +227,23 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "purging stashed session keys should lose access to the 
cleartext"
+notmuch reindex --decrypt=false id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
+test_begin_subtest "and cleartext should be unrecoverable now that there are 
no stashed session keys"
+notmuch dump
+notmuch reindex --decrypt=true id:simple-encryp...@crypto.notmuchmail.org
+output=$(notmuch search sekrit)
+expected=''
+test_expect_equal \
+"$output" \
+"$expected"
+
 
 # TODO: test removal of a message from the message store between
 # indexing and reindexing.
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 07/15] cli/show, reply: document use of stashed session keys in notmuch-properties

[Daniel Kahn Gillmor]

The stashed session keys are stored internally as notmuch properties.
So a user or developer who is reading about those properties might
want to understand how they fit into the bigger picture.

Note here that decrypting with a stored session key no longer needs
-decrypt for "notmuch show" and "notmuch reply".
---
 doc/man7/notmuch-properties.rst | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst
index 05444f6c..1a3f690e 100644
--- a/doc/man7/notmuch-properties.rst
+++ b/doc/man7/notmuch-properties.rst
@@ -77,9 +77,12 @@ of its normal activity.
 **session-key**
 
 When **notmuch-show(1)** or **nomtuch-reply** encounters a message
-with an encrypted part and ``--decrypt`` is set, if notmuch finds a
-``session-key`` property associated with the message, it will try
-that stashed session key for decryption.
+with an encrypted part, if notmuch finds a ``session-key``
+property associated with the message, it will try that stashed
+session key for decryption.
+
+If you do not want to use any stashed session keys that might be
+present, you should pass those programs ``--decrypt=false``.
 
 Using a stashed session key with "notmuch show" will speed up
 rendering of long encrypted threads.  It also allows the user to
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 02/15] indexopts: change _try_decrypt to _decrypt_policy

[Daniel Kahn Gillmor]

This terminology makes it clearer what's going on at the API layer,
and paves the way for future changesets that offer more nuanced
decryption policy.
---
 lib/index.cc |  2 +-
 lib/indexopts.c  | 10 +-
 lib/notmuch.h|  8 
 notmuch-client.h |  4 ++--
 notmuch.c| 12 ++--
 5 files changed, 18 insertions(+), 18 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 6eb60f30..ff14e408 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -525,7 +525,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 notmuch_database_t * notmuch = NULL;
 GMimeObject *clear = NULL;
 
-if (!indexopts || !notmuch_indexopts_get_try_decrypt (indexopts))
+if (!indexopts || !notmuch_indexopts_get_decrypt_policy (indexopts))
return;
 
 notmuch = _notmuch_message_database (message);
diff --git a/lib/indexopts.c b/lib/indexopts.c
index ca6bf6c9..0f65b97c 100644
--- a/lib/indexopts.c
+++ b/lib/indexopts.c
@@ -36,24 +36,24 @@ notmuch_database_get_default_indexopts (notmuch_database_t 
*db)
((!(strcasecmp(decrypt, "true"))) ||
 (!(strcasecmp(decrypt, "yes"))) ||
 (!(strcasecmp(decrypt, "1")
-   notmuch_indexopts_set_try_decrypt (ret, true);
+   notmuch_indexopts_set_decrypt_policy (ret, true);
 
 free (decrypt);
 return ret;
 }
 
 notmuch_status_t
-notmuch_indexopts_set_try_decrypt (notmuch_indexopts_t *indexopts,
-  notmuch_bool_t try_decrypt)
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+ notmuch_bool_t decrypt_policy)
 {
 if (!indexopts)
return NOTMUCH_STATUS_NULL_POINTER;
-indexopts->crypto.decrypt = try_decrypt;
+indexopts->crypto.decrypt = decrypt_policy;
 return NOTMUCH_STATUS_SUCCESS;
 }
 
 notmuch_bool_t
-notmuch_indexopts_get_try_decrypt (const notmuch_indexopts_t *indexopts)
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts)
 {
 if (!indexopts)
return false;
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 2c5dcab5..ef463090 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2244,17 +2244,17 @@ notmuch_database_get_default_indexopts 
(notmuch_database_t *db);
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_status_t
-notmuch_indexopts_set_try_decrypt (notmuch_indexopts_t *indexopts,
-  notmuch_bool_t try_decrypt);
+notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts,
+ notmuch_bool_t decrypt_policy);
 
 /**
  * Return whether to decrypt encrypted parts while indexing.
- * see notmuch_indexopts_set_try_decrypt.
+ * see notmuch_indexopts_set_decrypt_policy.
  *
  * @since libnotmuch 5.1 (notmuch 0.26)
  */
 notmuch_bool_t
-notmuch_indexopts_get_try_decrypt (const notmuch_indexopts_t *indexopts);
+notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts);
 
 /**
  * Destroy a notmuch_indexopts_t object.
diff --git a/notmuch-client.h b/notmuch-client.h
index f7524e59..fc981459 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -500,8 +500,8 @@ int notmuch_minimal_options (const char* subcommand_name,
 /* the state chosen by the user invoking one of the notmuch
  * subcommands that does indexing */
 struct _notmuch_client_indexing_cli_choices {
-bool try_decrypt;
-bool try_decrypt_set;
+bool decrypt_policy;
+bool decrypt_policy_set;
 notmuch_indexopts_t * opts;
 };
 extern struct _notmuch_client_indexing_cli_choices indexing_cli_choices;
diff --git a/notmuch.c b/notmuch.c
index 7ee3ad0b..89d5e77b 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -99,8 +99,8 @@ int notmuch_minimal_options (const char *subcommand_name,
 
 struct _notmuch_client_indexing_cli_choices indexing_cli_choices = { };
 const notmuch_opt_desc_t  notmuch_shared_indexing_options [] = {
-{ .opt_bool = _cli_choices.try_decrypt,
-  .present = _cli_choices.try_decrypt_set,
+{ .opt_bool = _cli_choices.decrypt_policy,
+  .present = _cli_choices.decrypt_policy_set,
   .name = "decrypt" },
 { }
 };
@@ -111,21 +111,21 @@ notmuch_process_shared_indexing_options 
(notmuch_database_t *notmuch, g_mime_3_u
 {
 if (indexing_cli_choices.opts == NULL)
indexing_cli_choices.opts = notmuch_database_get_default_indexopts 
(notmuch);
-if (indexing_cli_choices.try_decrypt_set) {
+if (indexing_cli_choices.decrypt_policy_set) {
notmuch_status_t status;
if (indexing_cli_choices.opts == NULL)
return NOTMUCH_STATUS_OUT_OF_MEMORY;
-   status = notmuch_indexopts_set_try_decrypt (indexing_cli_choices.opts, 
indexing_cli_choices.try_decrypt);
+   status = notmuch_indexopts_set_decrypt_policy 
(indexing_cli_choices.opts, indexing_cli_choices.decrypt_policy);
if (status != NOTMUCH_STATUS_SUCCESS) {
fprintf (stderr, "Error: Failed to set index decryption policy to 
%s. (%s)\n",
-

[PATCH v3 09/15] crypto: record whether an actual decryption attempt happened

[Daniel Kahn Gillmor]

In our consolidation of _notmuch_crypto_decrypt, the callers lost
track a little bit of whether any actual decryption was attempted.

Now that we have the more-subtle "auto" policy, it's possible that
_notmuch_crypto_decrypt could be called without having any actual
decryption take place.

This change lets the callers be a little bit smarter about whether or
not any decryption was actually attempted.
---
 lib/index.cc  | 17 -
 mime-node.c   |  4 ++--
 util/crypto.c |  7 ++-
 util/crypto.h |  3 ++-
 4 files changed, 22 insertions(+), 9 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index af999bd3..3914012a 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -548,12 +548,19 @@ _index_encrypted_mime_part (notmuch_message_t *message,
}
 }
 #endif
-clear = _notmuch_crypto_decrypt (notmuch_indexopts_get_decrypt_policy 
(indexopts),
+bool attempted = false;
+clear = _notmuch_crypto_decrypt (, 
notmuch_indexopts_get_decrypt_policy (indexopts),
 message, crypto_ctx, encrypted_data, NULL, 
);
-if (err) {
-   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
-  err->domain, err->code, err->message);
-   g_error_free(err);
+if (!attempted)
+   return;
+if (err || !clear) {
+   if (err) {
+   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(%d:%d) [%s]\n",
+  err->domain, err->code, err->message);
+   g_error_free(err);
+   } else {
+   _notmuch_database_log (notmuch, "Failed to decrypt during indexing. 
(unknown error)\n");
+   }
/* Indicate that we failed to decrypt during indexing */
status = notmuch_message_add_property (message, "index.decryption", 
"failure");
if (status)
diff --git a/mime-node.c b/mime-node.c
index 49d668fe..11df082b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -204,8 +204,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
if (parent->envelope_file)
break;
 
-   node->decrypt_attempted = true;
-   node->decrypted_child = _notmuch_crypto_decrypt 
(node->ctx->crypto->decrypt,
+   node->decrypted_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
+
node->ctx->crypto->decrypt,
 parent ? 
parent->envelope_file : NULL,
 cryptoctx, 
encrypteddata, _result, );
 }
diff --git a/util/crypto.c b/util/crypto.c
index bb587571..338f1d5d 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -140,7 +140,8 @@ void _notmuch_crypto_cleanup (unused(_notmuch_crypto_t 
*crypto))
 #endif
 
 GMimeObject *
-_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
+_notmuch_crypto_decrypt (bool *attempted,
+notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
 g_mime_3_unused(GMimeCryptoContext* crypto_ctx),
 GMimeMultipartEncrypted *part,
@@ -162,6 +163,8 @@ _notmuch_crypto_decrypt (notmuch_decryption_policy_t 
decrypt,
g_error_free (*err);
*err = NULL;
}
+   if (attempted)
+   *attempted = true;
 #if (GMIME_MAJOR_VERSION < 3)
ret = g_mime_multipart_encrypted_decrypt_session (part,
  crypto_ctx,
@@ -191,6 +194,8 @@ _notmuch_crypto_decrypt (notmuch_decryption_policy_t 
decrypt,
 if (decrypt == NOTMUCH_DECRYPT_AUTO)
return ret;
 
+if (attempted)
+   *attempted = true;
 #if (GMIME_MAJOR_VERSION < 3)
 ret = g_mime_multipart_encrypted_decrypt(part, crypto_ctx,
 decrypt_result, err);
diff --git a/util/crypto.h b/util/crypto.h
index dc95b4ca..c384601c 100644
--- a/util/crypto.h
+++ b/util/crypto.h
@@ -16,7 +16,8 @@ typedef struct _notmuch_crypto {
 } _notmuch_crypto_t;
 
 GMimeObject *
-_notmuch_crypto_decrypt (notmuch_decryption_policy_t decrypt,
+_notmuch_crypto_decrypt (bool *attempted,
+notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
 GMimeCryptoContext* crypto_ctx,
 GMimeMultipartEncrypted *part,
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 13/15] crypto: add --decrypt=nostash to avoid stashing session keys

[Daniel Kahn Gillmor]

Here's the configuration choice for people who want a cleartext index,
but don't want stashed session keys.

Interestingly, this "nostash" decryption policy is actually the same
policy that should be used by "notmuch show" and "notmuch reply",
since they never modify the index or database when they are invoked
with --decrypt.

We take advantage of this parallel to tune the behavior of those
programs so that we're not requesting session keys from GnuPG during
"show" and "reply" that we would then otherwise just throw away.
---
 completion/notmuch-completion.bash |  6 +++---
 doc/man1/notmuch-config.rst| 17 -
 doc/man1/notmuch-insert.rst| 16 ++--
 doc/man1/notmuch-new.rst   |  7 ---
 doc/man1/notmuch-reindex.rst   | 16 ++--
 lib/indexopts.c|  2 ++
 lib/notmuch.h  |  1 +
 notmuch-reply.c|  2 +-
 notmuch-show.c |  3 ++-
 notmuch.c  |  1 +
 test/T357-index-decryption.sh  | 23 +++
 util/crypto.c  |  4 ++--
 12 files changed, 71 insertions(+), 27 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 272131e6..948c153b 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -288,7 +288,7 @@ _notmuch_insert()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
@@ -320,7 +320,7 @@ _notmuch_new()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
@@ -442,7 +442,7 @@ _notmuch_reindex()
 $split &&
 case "${prev}" in
--decrypt)
-   COMPREPLY=( $( compgen -W "true false auto" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true false auto nostash" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index dabf269f..773fd9da 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -141,6 +141,9 @@ The available configuration items are described below.
 **index.decrypt**
 
 **[STORED IN DATABASE]**
+
+One of ``false``, ``auto``, ``nostash``, or ``true``.
+
 When indexing an encrypted e-mail message, if this variable is
 set to ``true``, notmuch will try to decrypt the message and
 index the cleartext, stashing a copy of any discovered session
@@ -150,11 +153,15 @@ The available configuration items are described below.
 secret keys.  Use ``false`` to avoid decrypting even when a
 stashed session key is already present.
 
-Be aware that the notmuch index is likely sufficient to
-reconstruct the cleartext of the message itself, so please
-ensure that the notmuch message index is adequately protected.
-DO NOT USE ``index.decrypt=true`` without considering the
-security of your index.
+``nostash`` is the same as ``true`` except that it will not
+stash newly-discovered session keys in the database.
+
+Be aware that the notmuch index is likely sufficient (and a
+stashed session key is certainly sufficient) to reconstruct
+the cleartext of the message itself, so please ensure that the
+notmuch message index is adequately protected.  DO NOT USE
+``index.decrypt=true`` or ``index.decrypt=nostash`` without
+considering the security of your index.
 
 Default: ``auto``.
 
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index 214f261b..1a3dfe98 100644
--- a/doc/man1/notmuch-insert.rst
+++ b/doc/man1/notmuch-insert.rst
@@ -51,10 +51,10 @@ Supported options for **insert** include
 ``--no-hooks``
 Prevent hooks from being run.
 
-``--decrypt=(true|auto|false)``
+``--decrypt=(true|nostash|auto|false)``
 
 If ``true`` and the message is encrypted, try to decrypt the
-message while indexing, storing any session keys discovered.
+message while indexing, stashing any session keys discovered.
 If ``auto``, and notmuch already knows about a session key for
 the message, it will try decrypting using that session key but
 will not try to access the user's secret keys.  If decryption
@@ -62,11 +62,15 @@ Supported options for **insert** include
 message is always stored to disk in its original form
 (ciphertext).
 
-Be aware that the index is likely sufficient to reconstruct
-the cleartext of the message itself, so please ensure that the
+ 

[PATCH v3 05/15] cli/reply: use decryption policy "auto" by default.

[Daniel Kahn Gillmor]

If the user doesn't specify --decrypt= at all, but a stashed session
key is known to notmuch, when replying to an encrypted message,
notmuch should just go ahead and decrypt.

The user can disable this at the command line with --decrypt=false,
though it's not clear why they would ever want to do that.
---
 completion/notmuch-completion.bash |  6 +-
 doc/man1/notmuch-reply.rst |  6 +-
 notmuch-reply.c|  9 +
 test/T357-index-decryption.sh  | 10 ++
 4 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index e462a82a..1cd616b3 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -350,12 +350,16 @@ _notmuch_reply()
COMPREPLY=( $( compgen -W "all sender" -- "${cur}" ) )
return
;;
+   --decrypt)
+   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
 case "${cur}" in
-*)
-   local options="--format= --format-version= --reply-to= --decrypt 
${_notmuch_shared_options}"
+   local options="--format= --format-version= --reply-to= --decrypt= 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst
index b6aec3c8..ede77930 100644
--- a/doc/man1/notmuch-reply.rst
+++ b/doc/man1/notmuch-reply.rst
@@ -80,8 +80,12 @@ Supported options for **reply** include
 multipart/encrypted part will be replaced by the decrypted
 content.
 
+If a session key is already known for the message, then it
+will be decrypted automatically unless the user explicitly
+sets ``--decrypt=false``.
+
 Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
+needed credentials. Without one, the decryption will likely fail.
 
 See **notmuch-search-terms(7)** for details of the supported syntax for
 .
diff --git a/notmuch-reply.c b/notmuch-reply.c
index eec34bed..fd990a9a 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -700,11 +700,12 @@ notmuch_reply_command (notmuch_config_t *config, int 
argc, char *argv[])
 int opt_index;
 notmuch_show_params_t params = {
.part = -1,
-   .crypto = { .decrypt = NOTMUCH_DECRYPT_FALSE },
+   .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO },
 };
 int format = FORMAT_DEFAULT;
 int reply_all = true;
 bool decrypt = false;
+bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -718,7 +719,7 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "all", true },
  { "sender", false },
  { 0, 0 } } },
-   { .opt_bool = , .name = "decrypt" },
+   { .opt_bool = , .name = "decrypt", .present = _set },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -728,8 +729,8 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 
 notmuch_process_shared_options (argv[0]);
-if (decrypt)
-   params.crypto.decrypt = NOTMUCH_DECRYPT_TRUE;
+if (decrypt_set)
+   params.crypto.decrypt = decrypt ? NOTMUCH_DECRYPT_TRUE : 
NOTMUCH_DECRYPT_FALSE;
 
 notmuch_exit_if_unsupported_format ();
 
diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 7996ec67..31991e22 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -200,6 +200,16 @@ test_expect_equal \
 "$output" \
 "$expected"
 
+test_begin_subtest "notmuch reply should show cleartext if session key is 
present"
+output=$(notmuch reply id:simple-encryp...@crypto.notmuchmail.org | grep '^>')
+expected='> This is a top sekrit message.'
+if [ $NOTMUCH_HAVE_GMIME_SESSION_KEYS -eq 0 ]; then
+test_subtest_known_broken
+fi
+test_expect_equal \
+"$output" \
+"$expected"
+
 
 # TODO: test removal of a message from the message store between
 # indexing and reindexing.
-- 
2.15.0

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 01/15] indexing: Change from try_decrypt to decrypt

[Daniel Kahn Gillmor]

the command-line interface for indexing (reindex, new, insert) used
--try-decrypt; and the configuration records used index.try_decrypt.
But by comparison with "show" and "reply", there doesn't seem to be
any reason for the "try" prefix.

This changeset adjusts the command-line interface and the
configuration interface.

For the moment, i've left indexopts_{set,get}_try_decrypt alone.  The
subsequent changeset will address those.
---
 NEWS   |  4 ++--
 completion/notmuch-completion.bash | 12 ++--
 doc/man1/notmuch-config.rst|  4 ++--
 doc/man1/notmuch-insert.rst|  6 +++---
 doc/man1/notmuch-new.rst   |  6 +++---
 doc/man1/notmuch-reindex.rst   |  6 +++---
 doc/man7/notmuch-properties.rst|  2 +-
 lib/indexopts.c| 14 +++---
 notmuch-config.c   |  2 +-
 notmuch.c  |  4 ++--
 test/T357-index-decryption.sh  | 18 +-
 test/test-lib.sh   |  2 +-
 12 files changed, 40 insertions(+), 40 deletions(-)

diff --git a/NEWS b/NEWS
index 412c678d..0465b9e8 100644
--- a/NEWS
+++ b/NEWS
@@ -19,9 +19,9 @@ Indexing cleartext of encrypted e-mails
   It's now possible to include the cleartext of encrypted e-mails in
   the notmuch index.  This makes it possible to search your encrypted
   e-mails with the same ease as searching cleartext.  This can be done
-  on a per-message basis with the --try-decrypt argument to indexing
+  on a per-message basis with the --decrypt argument to indexing
   commands (new, insert, reindex), or by default by running "notmuch
-  config set index.try_decrypt true".
+  config set index.decrypt true".
 
   Note that the contents of the index are sufficient to roughly
   reconstruct the cleartext of the message itself, so please ensure
diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 7aae4297..e462a82a 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -287,7 +287,7 @@ _notmuch_insert()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
return
;;
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -296,7 +296,7 @@ _notmuch_insert()
 ! $split &&
 case "${cur}" in
--*)
-   local options="--create-folder --folder= --keep --no-hooks 
--try-decrypt= ${_notmuch_shared_options}"
+   local options="--create-folder --folder= --keep --no-hooks 
--decrypt= ${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
return
@@ -319,7 +319,7 @@ _notmuch_new()
 
 $split &&
 case "${prev}" in
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -328,7 +328,7 @@ _notmuch_new()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--no-hooks --try-decrypt= --quiet 
${_notmuch_shared_options}"
+   local options="--no-hooks --decrypt= --quiet 
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "${options}" -- ${cur}) )
;;
@@ -437,7 +437,7 @@ _notmuch_reindex()
 
 $split &&
 case "${prev}" in
-   --try-decrypt)
+   --decrypt)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
@@ -446,7 +446,7 @@ _notmuch_reindex()
 ! $split &&
 case "${cur}" in
-*)
-   local options="--try-decrypt= ${_notmuch_shared_options}"
+   local options="--decrypt= ${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst
index 6961737f..ea3d9754 100644
--- a/doc/man1/notmuch-config.rst
+++ b/doc/man1/notmuch-config.rst
@@ -138,7 +138,7 @@ The available configuration items are described below.
 
 Default: ``gpg``.
 
-**index.try_decrypt**
+**index.decrypt**
 
 **[STORED IN DATABASE]**
 When indexing an encrypted e-mail message, if this variable is
@@ -146,7 +146,7 @@ The available configuration items are described below.
 the cleartext.  Be aware that the index is likely sufficient
 to reconstruct the cleartext of the message itself, so please
 ensure that the notmuch message index is adequately protected.
-DO NOT USE ``index.try_decrypt=true`` without considering the
+DO NOT USE ``index.decrypt=true`` without considering the
 security of your index.
 
 Default: ``false``.
diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst
index c500b251..eb9ff11b 100644
--- a/doc/man1/notmuch-insert.rst
+++ 

[PATCH 6/6] test: Add test to unset config items with the python bindings

[l-m-h]

---
 test/T390-python.sh | 13 +
 1 file changed, 13 insertions(+)

diff --git a/test/T390-python.sh b/test/T390-python.sh
index c6f395e4..312d61e8 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -142,4 +142,17 @@ cat <<'EOF' >EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "set_config with no value will unset config entries"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+db.set_config('testkey1', '')
+db.set_config('testkey2', '')
+db.set_config("zzzafter", '')
+db.set_config("aaabefore", '')
+v = db.get_configs()
+print(list(v) == [])
+EOF
+test_expect_equal "$(cat OUTPUT)" "True"
+
 test_done
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/6] python: add default arg to get_config_list

[l-m-h]

It makes the function a little more intuitive to use and does not
diverge much from the original function signature.

Also an example is added to the docstring.
---
 bindings/python/notmuch/database.py | 13 ++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index 2866b860..54966307 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -313,7 +313,7 @@ class Database(object):
 """
 self._assert_db_is_initialized()
 status = Database._upgrade(self._db, None, None)
-#TODO: catch exceptions, document return values and etc
+# TODO: catch exceptions, document return values and etc
 return status
 
 _begin_atomic = nmlib.notmuch_database_begin_atomic
@@ -689,12 +689,19 @@ class Database(object):
 _config_list_destroy.argtypes = [NotmuchConfigListP]
 _config_list_destroy.restype = None
 
-def get_config_list(self, prefix):
+def get_config_list(self, prefix=''):
 """Return a list of key, value pairs where the start of key matches the
 given prefix
 
 Note that only config values that are stored in the database are
-searched and returned.  The config file is not read.
+searched and returned.  The config file is not read.  If no `prefix` is
+given all config values are returned.
+
+This could be used to get all config values or all named queries into a
+dict for example::
+
+config = {k: v for k, v in db.get_config_list()}
+queries = {k[6:]: v for k, v in db.get_config_list('query.')}
 
 :param prefix: a string by which the keys should be selected
 :type prefix:  str
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/6] python: Rename get_config_list to get_configs

[l-m-h]

The old name has a bit of a feeling of hungarian notation.  Also many
generators in the core are named with the suffix "s" to indicate
iterables: dict.items, dict.keys for example.
---
 bindings/python/notmuch/database.py | 18 ++
 test/T390-python.sh | 12 ++--
 2 files changed, 8 insertions(+), 22 deletions(-)

diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index 32566620..fe09b330 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -689,7 +689,7 @@ class Database(object):
 _config_list_destroy.argtypes = [NotmuchConfigListP]
 _config_list_destroy.restype = None
 
-def get_config_list(self, prefix=''):
+def get_configs(self, prefix=''):
 """Return a generator of key, value pairs where the start of key
 matches the given prefix
 
@@ -699,7 +699,7 @@ class Database(object):
 
 This could be used to get all named queries into a dict for example::
 
-queries = {k[6:]: v for k, v in db.get_config_list('query.')}
+queries = {k[6:]: v for k, v in db.get_configs('query.')}
 
 :param prefix: a string by which the keys should be selected
 :type prefix:  str
@@ -721,20 +721,6 @@ class Database(object):
 yield key, value
 self._config_list_move_to_next(config_list_p)
 
-def get_configs(self, prefix=''):
-"""Return a dict of key, value pairs where the start of key matches the
-given prefix
-
-:param prefix: a string by which the keys should be selected
-:type prefix:  str
-:returns:  all key-value pairs where `prefix` matches the beginning
-   of the key
-:rtype:a dict of str: str
-:raises:  :exc:`NotmuchError` in case of failure.
-
-"""
-return dict(self.get_config_list(prefix))
-
 """notmuch_database_set_config"""
 _set_config = nmlib.notmuch_database_set_config
 _set_config.argtypes = [NotmuchDatabaseP, c_char_p, c_char_p]
diff --git a/test/T390-python.sh b/test/T390-python.sh
index 725a00c9..c6f395e4 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -97,22 +97,22 @@ testkey2 = testvalue2
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
-test_begin_subtest "get_config_list with no match returns empty generator"
+test_begin_subtest "get_configs with no match returns empty generator"
 test_python <<'EOF'
 import notmuch
 db = notmuch.Database()
-v = db.get_config_list('nonexistent')
+v = db.get_configs('nonexistent')
 print(list(v) == [])
 EOF
 test_expect_equal "$(cat OUTPUT)" "True"
 
-test_begin_subtest "get_config_list with no arguments returns all pairs"
+test_begin_subtest "get_configs with no arguments returns all pairs"
 test_python <<'EOF'
 import notmuch
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
 db.set_config("zzzafter", "afterval")
 db.set_config("aaabefore", "beforeval")
-v = db.get_config_list()
+v = db.get_configs()
 for index, keyval in enumerate(v):
 key, val = keyval
 print('{}: {} => {}'.format(index, key, val))
@@ -125,13 +125,13 @@ cat <<'EOF' >EXPECTED
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
-test_begin_subtest "get_config_list prefix is used to match keys"
+test_begin_subtest "get_configs prefix is used to match keys"
 test_python <<'EOF'
 import notmuch
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
 db.set_config('testkey1', 'testvalue1')
 db.set_config('testkey2', 'testvalue2')
-v = db.get_config_list('testkey')
+v = db.get_configs('testkey')
 for index, keyval in enumerate(v):
 key, val = keyval
 print('{}: {} => {}'.format(index, key, val))
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/6] python: add bindings to access config

[l-m-h]

The C functions notmuch_database_get_config,
notmuch_database_get_config_list and notmuch_database_set_config are
part of the official C bindings.  So there should also be some python
bindings for them.

Also they are the only way to access the named queries introduced in
b9bf3f44.

The interface of the python functions is designed to be close to the C
functions.
---
 bindings/python/docs/source/database.rst |   6 ++
 bindings/python/notmuch/database.py  | 106 +++
 bindings/python/notmuch/globals.py   |   5 ++
 3 files changed, 117 insertions(+)

diff --git a/bindings/python/docs/source/database.rst b/bindings/python/docs/source/database.rst
index 079dc754..f9567949 100644
--- a/bindings/python/docs/source/database.rst
+++ b/bindings/python/docs/source/database.rst
@@ -37,6 +37,12 @@
 
.. automethod:: create_query
 
+   .. automethod:: get_config
+
+   .. automethod:: get_config_list
+
+   .. automethod:: set_config
+
.. attribute:: Database.MODE
 
  Defines constants that are used as the mode in which to open a database.
diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index 1279804a..2866b860 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -26,6 +26,7 @@ from .globals import (
 nmlib,
 Enum,
 _str,
+NotmuchConfigListP,
 NotmuchDatabaseP,
 NotmuchDirectoryP,
 NotmuchMessageP,
@@ -634,3 +635,108 @@ class Database(object):
 raise NotmuchError(message="No DB path specified"
" and no user default found")
 return config.get('database', 'path')
+
+"""notmuch_database_get_config"""
+_get_config = nmlib.notmuch_database_get_config
+_get_config.argtypes = [NotmuchDatabaseP, c_char_p, POINTER(c_char_p)]
+_get_config.restype = c_uint
+
+def get_config(self, key):
+"""Return the value of the given config key.
+
+Note that only config values that are stored in the database are
+searched and returned.  The config file is not read.
+
+:param key: the config key under which a value should be looked up, it
+should probably be in the form "section.key"
+:type key:  str
+:returns:   the config value or the empty string if no value is present
+for that key
+:rtype: str
+:raises::exc:`NotmuchError` in case of failure.
+
+"""
+self._assert_db_is_initialized()
+return_string = c_char_p()
+status = self._get_config(self._db, _str(key), byref(return_string))
+if status != STATUS.SUCCESS:
+raise NotmuchError(status)
+return return_string.value.decode('utf-8')
+
+"""notmuch_database_get_config_list"""
+_get_config_list = nmlib.notmuch_database_get_config_list
+_get_config_list.argtypes = [NotmuchDatabaseP, c_char_p,
+ POINTER(NotmuchConfigListP)]
+_get_config_list.restype = c_uint
+
+_config_list_valid = nmlib.notmuch_config_list_valid
+_config_list_valid.argtypes = [NotmuchConfigListP]
+_config_list_valid.restype = bool
+
+_config_list_key = nmlib.notmuch_config_list_key
+_config_list_key.argtypes = [NotmuchConfigListP]
+_config_list_key.restype = c_char_p
+
+_config_list_value = nmlib.notmuch_config_list_value
+_config_list_value.argtypes = [NotmuchConfigListP]
+_config_list_value.restype = c_char_p
+
+_config_list_move_to_next = nmlib.notmuch_config_list_move_to_next
+_config_list_move_to_next.argtypes = [NotmuchConfigListP]
+_config_list_move_to_next.restype = None
+
+_config_list_destroy = nmlib.notmuch_config_list_destroy
+_config_list_destroy.argtypes = [NotmuchConfigListP]
+_config_list_destroy.restype = None
+
+def get_config_list(self, prefix):
+"""Return a list of key, value pairs where the start of key matches the
+given prefix
+
+Note that only config values that are stored in the database are
+searched and returned.  The config file is not read.
+
+:param prefix: a string by which the keys should be selected
+:type prefix:  str
+:returns:  all key-value pairs where `prefix` matches the beginning
+   of the key
+:rtype:a list of pairs of str
+:raises:  :exc:`NotmuchError` in case of failure.
+
+"""
+self._assert_db_is_initialized()
+config_list_p = NotmuchConfigListP()
+status = self._get_config_list(self._db, _str(prefix),
+   byref(config_list_p))
+if status != STATUS.SUCCESS:
+raise NotmuchError(status)
+config_list = []
+while self._config_list_valid(config_list_p):
+key = self._config_list_key(config_list_p).decode('utf-8')
+value = 

[PATCH 4/6] test: Add tests for new python bindings

[l-m-h]

The tests where adopted from the tests for the corresponding C functions
in test/T590-libconfig.sh.
---
 test/T390-python.sh | 68 +
 1 file changed, 68 insertions(+)

diff --git a/test/T390-python.sh b/test/T390-python.sh
index a93a7f34..725a00c9 100755
--- a/test/T390-python.sh
+++ b/test/T390-python.sh
@@ -74,4 +74,72 @@ EOF
 notmuch search --sort=oldest-first --output=messages "tučňáččí" | sed s/^id:// > EXPECTED
 test_expect_equal_file EXPECTED OUTPUT
 
+# TODO currently these tests for setting and getting config values are
+# somewhat interdependent.  This is because the config values stored in the
+# database are not cleaned up after each test, so they remain there for the
+# next test.  The ./README file states that this can happen so it seems kind
+# of ok.
+
+test_begin_subtest "set and get config values"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+db.set_config('testkey1', 'testvalue1')
+db.set_config('testkey2', 'testvalue2')
+v1 = db.get_config('testkey1')
+v2 = db.get_config('testkey2')
+print('testkey1 = ' + v1)
+print('testkey2 = ' + v2)
+EOF
+cat <<'EOF' >EXPECTED
+testkey1 = testvalue1
+testkey2 = testvalue2
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "get_config_list with no match returns empty generator"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database()
+v = db.get_config_list('nonexistent')
+print(list(v) == [])
+EOF
+test_expect_equal "$(cat OUTPUT)" "True"
+
+test_begin_subtest "get_config_list with no arguments returns all pairs"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+db.set_config("zzzafter", "afterval")
+db.set_config("aaabefore", "beforeval")
+v = db.get_config_list()
+for index, keyval in enumerate(v):
+key, val = keyval
+print('{}: {} => {}'.format(index, key, val))
+EOF
+cat <<'EOF' >EXPECTED
+0: aaabefore => beforeval
+1: testkey1 => testvalue1
+2: testkey2 => testvalue2
+3: zzzafter => afterval
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "get_config_list prefix is used to match keys"
+test_python <<'EOF'
+import notmuch
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+db.set_config('testkey1', 'testvalue1')
+db.set_config('testkey2', 'testvalue2')
+v = db.get_config_list('testkey')
+for index, keyval in enumerate(v):
+key, val = keyval
+print('{}: {} => {}'.format(index, key, val))
+EOF
+cat <<'EOF' >EXPECTED
+0: testkey1 => testvalue1
+1: testkey2 => testvalue2
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/6] python: turn get_config_list into a generator

[l-m-h]

This mimics the behaviour of the underlying C function more closely as
it also does not store all values in memory.
---
 bindings/python/notmuch/database.py | 30 --
 1 file changed, 20 insertions(+), 10 deletions(-)

diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py
index 54966307..32566620 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -690,24 +690,22 @@ class Database(object):
 _config_list_destroy.restype = None
 
 def get_config_list(self, prefix=''):
-"""Return a list of key, value pairs where the start of key matches the
-given prefix
+"""Return a generator of key, value pairs where the start of key
+matches the given prefix
 
 Note that only config values that are stored in the database are
 searched and returned.  The config file is not read.  If no `prefix` is
 given all config values are returned.
 
-This could be used to get all config values or all named queries into a
-dict for example::
+This could be used to get all named queries into a dict for example::
 
-config = {k: v for k, v in db.get_config_list()}
 queries = {k[6:]: v for k, v in db.get_config_list('query.')}
 
 :param prefix: a string by which the keys should be selected
 :type prefix:  str
-:returns:  all key-value pairs where `prefix` matches the beginning
+:yields:   all key-value pairs where `prefix` matches the beginning
of the key
-:rtype:a list of pairs of str
+:ytype:pairs of str
 :raises:  :exc:`NotmuchError` in case of failure.
 
 """
@@ -717,13 +715,25 @@ class Database(object):
byref(config_list_p))
 if status != STATUS.SUCCESS:
 raise NotmuchError(status)
-config_list = []
 while self._config_list_valid(config_list_p):
 key = self._config_list_key(config_list_p).decode('utf-8')
 value = self._config_list_value(config_list_p).decode('utf-8')
-config_list.append((key, value))
+yield key, value
 self._config_list_move_to_next(config_list_p)
-return config_list
+
+def get_configs(self, prefix=''):
+"""Return a dict of key, value pairs where the start of key matches the
+given prefix
+
+:param prefix: a string by which the keys should be selected
+:type prefix:  str
+:returns:  all key-value pairs where `prefix` matches the beginning
+   of the key
+:rtype:a dict of str: str
+:raises:  :exc:`NotmuchError` in case of failure.
+
+"""
+return dict(self.get_config_list(prefix))
 
 """notmuch_database_set_config"""
 _set_config = nmlib.notmuch_database_set_config
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 0/6] python: add bindings for notmuch_database_get_config{, _list}

[l-m-h]

Comming back after a long time (sorry for the wait).

I now changed the binding for notmuch_database_get_config_list into a
generator.  It is called get_configs in the python bindings (the "s"
should indicate the iterable/generator nature like for dict.items or
dict.keys).

Tests and the set_config entry point were also added.

If you want you can merge it as is or I can squash the commits in any
way you want.

Lucas Hoffmann (6):
  python: add bindings to access config
  python: add default arg to get_config_list
  python: turn get_config_list into a generator
  test: Add tests for new python bindings
  python: Rename get_config_list to get_configs
  test: Add test to unset config items with the python bindings

 bindings/python/docs/source/database.rst |   6 ++
 bindings/python/notmuch/database.py  | 111 ++-
 bindings/python/notmuch/globals.py   |   5 ++
 test/T390-python.sh  |  81 ++
 4 files changed, 202 insertions(+), 1 deletion(-)

-- 
2.15.1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 5/5] debian/control: put elpa-notmuch in Section: lisp.

[Daniel Kahn Gillmor]

On Thu 2017-11-30 04:40:43 -0500, Daniel Kahn Gillmor wrote:
> This makes lintian stop complaining about:
>
> W: elpa-notmuch: wrong-section-according-to-package-name elpa-notmuch => lisp

I'm withdrawing this patch for consideration.  I'm convinced by
bremner's argument in https://bugs.debian.org/883772 that
language-specific sections are a bad idea!

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test: session keys are known broken without session key support

[David Bremner]

Daniel Kahn Gillmor  writes:

> On Tue 2017-12-05 13:40:27 -0500, Daniel Kahn Gillmor wrote:
>> If the version of GMime we're building against doesn't support session
>> key extraction or re-use, mark the tests that rely on session key
>> capabilities as known-broken.
>>
>> This should resolve test suite failures on ubuntu trusty and debian
>> jessie and earlier, which have GMime 2.6.20 -- session key support was
>> introduced in GMime 2.6.21.
>
> This should fix the current travis build failure.  If it's accepted,
> i'll use the same technique to mark other tests in my session-keys
> series as dependent upon session-key capability.

pushed,

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 06/21] crypto: Test restore of cleartext index from stashed session keys

[Jameson Graef Rollins]

On Mon, Dec 04 2017, David Bremner  wrote:
> Pushed patches 1 to 6. I seem to recall 7 and 8 basically adressed
> concerns/suggestions Jamie had, so I'm hoping he can have a quick look
> at those.

Yes, this new series is great and definitely addresses all my concerns.
I'm stoked to see that the first part of it has been pushed, and looking
forward to the full series!

This is really great progress, Daniel.  Thanks for pushing on this.

jamie.


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] cli/help: give a hint about notmuch-emacs-mua

[David Bremner]

Daniel Kahn Gillmor  writes:

> "notmuch help" doesn't mention "notmuch-emacs-mua" even though we
> support it through the try_external_command() mechanism.
>
> In addition, "notmuch help emacs-mua" doesn't work, even though we
> ship the appropriate manpage.
>
> This changeset fixes both of these problems.

pushed,

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] debian/control: build-depend on python3-sphinx instead of python-sphinx

[David Bremner]

Daniel Kahn Gillmor  writes:

> python2 is going to be deprecated, and python3-sphinx is available all
> the way back to oldoldstable.  let's use the more modern version.
>
> To make this work and still ship the manpages, tell ./configure to
> prefer python3 over python, if it exists.

pushed.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch