cope with inline PGP encrypted messages

2017-12-11 Thread Daniel Kahn Gillmor 
Inline PGP encrypted messages are clearly worse than PGP/MIME
structured encrypted messages.  There are no standards for how they
are formed, and they don't offer any structured metadata about how to
interpret the bytestream produced by decrypting them.

However, some other MUAs and end-user workflows may make creation of
inline PGP encrypted messages the only available option for message
encryption, and when Notmuch encounters such a message, it should make
a reasonable best-effort to render the cleartext to the user.

Due to ambiguities in interpretation of signatures on inline messages
(e.g. which parts of the message were actually signed?  what character
encoding should the bytestream be interpreted as), we continue to
ignore inline-signed messages entirely, and we do not look at the
validity of any signatures that might be found when decrypting inline
PGP encrypted messages.

We make use here of GMime's optimization function for detecting the
presence of inline PGP encrypted content, which is only found in GMime
3.0 or later.

This series is currently based n top of the "notmuch show
--decrypt=stash" series, which it needs to be able to apply cleanly.
If that series proves controversial, i could rebase this patch
manually against some earlier commit.

If you have applied this series, and you know you have some inline PGP
messages already in your message store, you can try to retroactively
reindex them with something like:

notmuch reindex --decrypt=true BEGIN-PGP-MESSAGE and not tag:encrypted

I welcome review and feedback about this series.

  --dkg

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 4/5] index: _index_encrypted_mime_part returns success or failure

2017-12-11 Thread Daniel Kahn Gillmor 
This change prepares us to know whether or not
_index_encrypted_mime_part succeeded or not on a given MIME part.

We don't currently make use of the information, but we will in
subsequent changes.
---
 lib/index.cc | 19 +++
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index e03f5230..29ede685 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -364,7 +364,7 @@ _index_content_type (notmuch_message_t *message, 
GMimeObject *part)
 }
 }
 
-static void
+static bool
 _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t 
*indexopts,
GMimeContentType *content_type,
GMimeObject *part);
@@ -419,6 +419,8 @@ _index_mime_part (notmuch_message_t *message,
_index_content_type (message,
 g_mime_multipart_get_part (multipart, i));
if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) {
+   /* deliberately ignore return value here: if it fails to 
decrypt,
+  we have nothing else to try */
_index_encrypted_mime_part(message, indexopts,
   content_type,
   part);
@@ -519,8 +521,9 @@ _index_mime_part (notmuch_message_t *message,
 }
 
 /* descend (if desired) into the cleartext part of an encrypted MIME
- * part while indexing. */
-static void
+ * part while indexing.  Returns true if there was a successful
+ * decryption, false if there was not.*/
+static bool
 _index_encrypted_mime_part (notmuch_message_t *message,
notmuch_indexopts_t *indexopts,
g_mime_3_unused(GMimeContentType *content_type),
@@ -532,7 +535,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 GMimeObject *clear = NULL;
 
 if (!indexopts || (notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE))
-   return;
+   return false;
 
 notmuch = notmuch_message_get_database (message);
 
@@ -550,7 +553,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
if (status)
_notmuch_database_log_append (notmuch, "failed to add 
index.decryption "
  "property (%d)\n", status);
-   return;
+   return false;
}
 }
 #endif
@@ -560,7 +563,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 clear = _notmuch_crypto_decrypt (, 
notmuch_indexopts_get_decrypt_policy (indexopts),
 message, crypto_ctx, encrypted_data, 
get_sk ? _result : NULL, );
 if (!attempted)
-   return;
+   return false;
 if (err || !clear) {
if (decrypt_result)
g_object_unref (decrypt_result);
@@ -576,7 +579,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
if (status)
_notmuch_database_log_append (notmuch, "failed to add 
index.decryption "
  "property (%d)\n", status);
-   return;
+   return false;
 }
 if (decrypt_result) {
 #if HAVE_GMIME_SESSION_KEYS
@@ -597,7 +600,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 if (status)
_notmuch_database_log (notmuch, "failed to add index.decryption "
   "property (%d)\n", status);
-
+return true;
 }
 
 notmuch_status_t
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/5] index: tag text parts with inline PGP encryption as "encrypted"

2017-12-11 Thread Daniel Kahn Gillmor 
Assuming we have GMime 3.0 (which has efficient detection of inline
PGP encrypted blobs) we should be able to mark those messages with the
same tag that we mark PGP/MIME and S/MIME encrypted messages.
---
 lib/index.cc   | 6 ++
 test/T359-inline-pgp-decryption.sh | 4 ++--
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index f144b9fb..e03f5230 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -468,6 +468,12 @@ _index_mime_part (notmuch_message_t *message,
return;
 }
 
+#if (GMIME_MAJOR_VERSION >= 3)
+if (GMIME_IS_TEXT_PART (part) && g_mime_part_get_openpgp_data (GMIME_PART 
(part)) == GMIME_OPENPGP_DATA_ENCRYPTED) {
+   _notmuch_message_add_term (message, "tag", "encrypted");
+}
+#endif
+
 byte_array = g_byte_array_new ();
 
 stream = g_mime_stream_mem_new_with_byte_array (byte_array);
diff --git a/test/T359-inline-pgp-decryption.sh 
b/test/T359-inline-pgp-decryption.sh
index c0db8eaf..314ca786 100755
--- a/test/T359-inline-pgp-decryption.sh
+++ b/test/T359-inline-pgp-decryption.sh
@@ -43,7 +43,7 @@ expected='
   },
  "id": "X",
  "match": true,
- "tags": ["inbox", "unread"],
+ "tags": ["encrypted", "inbox", "unread"],
  "timestamp": 946728000
  },
  ['
@@ -74,7 +74,7 @@ expected='
   },
  "id": "X",
  "match": false,
- "tags": ["inbox", "unread"],
+ "tags": ["encrypted", "inbox", "unread"],
  "timestamp": 946728000
  },
  "reply-headers": {
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/5] crypto: prepare for decryption of inline PGP encrypted messages

2017-12-11 Thread Daniel Kahn Gillmor 
Inline PGP encrypted messages are clearly worse than PGP/MIME
structured encrypted messages.  There are no standards for how they
are formed, and they don't offer any structured metadata about how to
interpret the bytestream produced by decrypting them.

However, some other MUAs and end-user workflows may make creation of
inline PGP encrypted messages the only available option for message
encryption, and when Notmuch encounters such a message, it should make
a reasonable best-effort to render the cleartext to the user.

Due to ambiguities in interpretation of signatures on inline messages
(e.g. which parts of the message were actually signed?  what character
encoding should the bytestream be interpreted as), we continue to
ignore inline-signed messages entirely, and we do not look at the
validity of any signatures that might be found when decrypting inline
PGP encrypted messages.

We make use here of GMime's optimization function for detecting the
presence of inline PGP encrypted content, which is only found in GMime
3.0 or later.

This change prepares the internal codebase for decrypting inline
encrypted messages, but does not yet actually use the capability.
---
 lib/index.cc  |  6 +++---
 mime-node.c   | 24 ++--
 util/crypto.c | 35 +++
 util/crypto.h |  2 +-
 4 files changed, 45 insertions(+), 22 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 22ca9ec1..f144b9fb 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -367,7 +367,7 @@ _index_content_type (notmuch_message_t *message, 
GMimeObject *part)
 static void
 _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t 
*indexopts,
GMimeContentType *content_type,
-   GMimeMultipartEncrypted *part);
+   GMimeObject *part);
 
 /* Callback to generate terms for each mime part of a message. */
 static void
@@ -421,7 +421,7 @@ _index_mime_part (notmuch_message_t *message,
if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) {
_index_encrypted_mime_part(message, indexopts,
   content_type,
-  GMIME_MULTIPART_ENCRYPTED 
(part));
+  part);
} else {
if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) {
_notmuch_database_log (notmuch_message_get_database 
(message),
@@ -518,7 +518,7 @@ static void
 _index_encrypted_mime_part (notmuch_message_t *message,
notmuch_indexopts_t *indexopts,
g_mime_3_unused(GMimeContentType *content_type),
-   GMimeMultipartEncrypted *encrypted_data)
+   GMimeObject *encrypted_data)
 {
 notmuch_status_t status;
 GError *err = NULL;
diff --git a/mime-node.c b/mime-node.c
index 75b79f98..973133d9 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -196,10 +196,10 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
 {
 GError *err = NULL;
 GMimeDecryptResult *decrypt_result = NULL;
-GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
 notmuch_message_t *message = NULL;
 
-if (! node->decrypted_child) {
+if (GMIME_IS_PART (part) || /* must be inline */
+   (GMIME_IS_MULTIPART_ENCRYPTED (part) && ! node->decrypted_child)) {
for (mime_node_t *parent = node; parent; parent = parent->parent)
if (parent->envelope_file) {
message = parent->envelope_file;
@@ -209,7 +209,7 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
node->decrypted_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
 
node->ctx->crypto->decrypt,
 message,
-cryptoctx, 
encrypteddata, _result, );
+cryptoctx, part, 
_result, );
 }
 if (! node->decrypted_child) {
fprintf (stderr, "Failed to decrypt part: %s\n",
@@ -217,15 +217,19 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
goto DONE;
 }
 
-node->decrypt_success = true;
-node->verify_attempted = true;
 
 if (decrypt_result) {
-   /* This may be NULL if the part is not signed. */
-   node->sig_list = g_mime_decrypt_result_get_signatures (decrypt_result);
-   if (node->sig_list) {
-   g_object_ref (node->sig_list);
-   set_signature_list_destructor (node);
+   node->decrypt_success = true;
+   if (GMIME_IS_MULTIPART_ENCRYPTED (part)) {
+   /* Only check signatures on PGP/MIME messages, not inline
+  messages. To understand why, see
+  https://dkg.fifthhorseman.net/notes/inline-pgp-harmful/ */
+

[PATCH 5/5] index: try indexing the cleartext of inline PGP encrypted text parts

2017-12-11 Thread Daniel Kahn Gillmor 
Assuming that we're using GMime 3.0 or later, and the user has asked
for decryption of some sort, we should go ahead and index the
cleartext.
---
 lib/index.cc   | 7 +++
 test/T359-inline-pgp-decryption.sh | 7 +++
 2 files changed, 14 insertions(+)

diff --git a/lib/index.cc b/lib/index.cc
index 29ede685..49f7cfbf 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -473,6 +473,13 @@ _index_mime_part (notmuch_message_t *message,
 #if (GMIME_MAJOR_VERSION >= 3)
 if (GMIME_IS_TEXT_PART (part) && g_mime_part_get_openpgp_data (GMIME_PART 
(part)) == GMIME_OPENPGP_DATA_ENCRYPTED) {
_notmuch_message_add_term (message, "tag", "encrypted");
+   if (_index_encrypted_mime_part(message, indexopts,
+  content_type,
+  part))
+   return;
+   /* if decryption on inline PGP encrypted message fails, we
+* should still fall through and try indexing the MIME part
+* anyway (this is what we did before inline PGP decryption) */
 }
 #endif
 
diff --git a/test/T359-inline-pgp-decryption.sh 
b/test/T359-inline-pgp-decryption.sh
index 314ca786..66b85d5b 100755
--- a/test/T359-inline-pgp-decryption.sh
+++ b/test/T359-inline-pgp-decryption.sh
@@ -94,4 +94,11 @@ output=$(notmuch search 'sekrit')
 expected=''
 test_expect_equal "$output" "$expected"
 
+test_begin_subtest "reindexing cleartext of inline PGP encrypted message 
should succeed"
+test_subtest_broken_gmime_2
+notmuch reindex --decrypt=true 
id:inline-pgp-encryp...@testsuite.notmuchmail.org
+output=$(notmuch search 'sekrit')
+expected='thread:0001   2000-01-01 [1/1] 
test_su...@notmuchmail.org; inline PGP encrypted message (encrypted inbox 
unread)'
+test_expect_equal "$output" "$expected"
+
 test_done
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/5] cli/{show, reply}: try to decrypt inline PGP encrypted messages

2017-12-11 Thread Daniel Kahn Gillmor 
We try this only for leaf parts that are explicitly marked as
Content-Type: text/*, since we don't want to accidentally match on any
other weird part that happens to contain the magic string, or on the
payload child of a multipart/encrypted part.

Of course, this only works for GMime 3.0 and later, because of how
we're detecting the presence of the OpenPGP inline encrypted blob.
---
 mime-node.c|  4 ++
 test/T359-inline-pgp-decryption.sh | 97 ++
 2 files changed, 101 insertions(+)
 create mode 100755 test/T359-inline-pgp-decryption.sh

diff --git a/mime-node.c b/mime-node.c
index 973133d9..3c94bb62 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -325,6 +325,10 @@ _mime_node_create (mime_node_t *parent, GMimeObject *part)
} else {
node_verify (node, part, cryptoctx);
}
+#if (GMIME_MAJOR_VERSION >= 3)
+} else if (GMIME_IS_TEXT_PART (part) && g_mime_part_get_openpgp_data 
(GMIME_PART (part)) == GMIME_OPENPGP_DATA_ENCRYPTED) {
+   node_decrypt_and_verify (node, part, cryptoctx);
+#endif
 }
 
 return node;
diff --git a/test/T359-inline-pgp-decryption.sh 
b/test/T359-inline-pgp-decryption.sh
new file mode 100755
index ..c0db8eaf
--- /dev/null
+++ b/test/T359-inline-pgp-decryption.sh
@@ -0,0 +1,97 @@
+#!/usr/bin/env bash
+
+test_description='Decryption of inline PGP messages'
+. $(dirname "$0")/test-lib.sh || exit 1
+
+##
+
+add_gnupg_home
+
+test_begin_subtest "Adding inline PGP encrypted message"
+mkdir -p "$MAIL_DIR/cur"
+cat < "$MAIL_DIR/cur/inline-pgp-encrypted.eml"
+Message-Id: inline-pgp-encryp...@testsuite.notmuchmail.org
+Content-Type: text/plain
+Subject: inline PGP encrypted message
+Date: Sat, 01 Jan 2000 12:00:00 +
+From: test_su...@notmuchmail.org
+To: test_su...@notmuchmail.org
+
+$(echo "this is the sekrit message" | gpg --no-tty --batch --quiet 
--trust-model=always --encrypt --armor --recipient test_su...@notmuchmail.org)
+EOF
+test_expect_success 'notmuch new'
+
+test_begin_subtest "inline PGP decryption, --format=json"
+test_subtest_broken_gmime_2
+output=$(notmuch show --format=json --decrypt=true 
id:inline-pgp-encryp...@testsuite.notmuchmail.org \
+| notmuch_json_show_sanitize)
+expected='
+ [[[{"body": [{
+ "content": "this is the sekrit message\n",
+ "content-type": "text/plain",
+ "encstatus": [{"status": "good" }],
+ "id": 1
+ }],
+ "date_relative": "2000-01-01",
+ "excluded": false,
+ "filename": ["Y"],
+ "headers": {
+   "Date": "Sat, 01 Jan 2000 12:00:00 +",
+   "From": "test_su...@notmuchmail.org",
+   "Subject": "inline PGP encrypted message",
+   "To": "test_su...@notmuchmail.org"
+  },
+ "id": "X",
+ "match": true,
+ "tags": ["inbox", "unread"],
+ "timestamp": 946728000
+ },
+ ['
+
+test_expect_equal_json \
+"$output" \
+"$expected"
+
+test_begin_subtest "inline PGP decryption for reply"
+test_subtest_broken_gmime_2
+output=$(notmuch reply --format=json --decrypt=true 
id:inline-pgp-encryp...@testsuite.notmuchmail.org \
+| notmuch_json_show_sanitize)
+expected='
+ {"original": {"body": [{
+ "content": "this is the sekrit message\n",
+ "content-type": "text/plain",
+ "encstatus": [{"status": "good" }],
+ "id": 1
+ }],
+ "date_relative": "2000-01-01",
+ "excluded": false,
+ "filename": ["Y"],
+ "headers": {
+   "Date": "Sat, 01 Jan 2000 12:00:00 +",
+   "From": "test_su...@notmuchmail.org",
+   "Subject": "inline PGP encrypted message",
+   "To": "test_su...@notmuchmail.org"
+  },
+ "id": "X",
+ "match": false,
+ "tags": ["inbox", "unread"],
+ "timestamp": 946728000
+ },
+ "reply-headers": {
+   "From": "Notmuch Test Suite ",
+   "In-reply-to": "",
+   "References": "",
+   "Subject": "Re: inline PGP encrypted message"
+ }
+}'
+
+test_expect_equal_json \
+"$output" \
+"$expected"
+
+test_begin_subtest "searching for cleartext of inline PGP encrypted message 
should fail"
+output=$(notmuch search 'sekrit')
+expected=''
+test_expect_equal "$output" "$expected"
+
+test_done
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] cli/reply: make --decrypt take a keyword

2017-12-11 Thread Daniel Kahn Gillmor 
On Tue 2017-12-12 01:52:52 -0500, Daniel Kahn Gillmor wrote:
> This brings the --decrypt argument to "notmuch reply" into line with
> the other --decrypt arguments (in "show", "new", "insert", and
> "reindex").  This patch is really just about bringing consistency to
> the user interface.

v2 of this patch covers:

> We also use the recommended form in the emacs MUA when replying, and
> update test T350 to match.

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] cli/show: make --decrypt take a keyword.

2017-12-11 Thread Daniel Kahn Gillmor 
On Tue 2017-12-12 01:51:57 -0500, Daniel Kahn Gillmor wrote:
> We also expand tab completion for it, update the emacs bindings, and
> update T350, T357, and T450 to match.
>
> Make use of the bool-to-keyword backward-compatibility feature.

and now revision 3 of this patch includes transitioning the emacs
bindings to use explicit arguments for --decrypt, and covering the rest
of the test suite as well.

I think it's settled down now. :)

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] cli/show: make --decrypt take a keyword.

2017-12-11 Thread Daniel Kahn Gillmor 
We also expand tab completion for it, update the emacs bindings, and
update T350, T357, and T450 to match.

Make use of the bool-to-keyword backward-compatibility feature.
---
 completion/notmuch-completion.bash |  6 +-
 doc/man1/notmuch-show.rst  | 37 +
 emacs/notmuch-lib.el   |  2 +-
 emacs/notmuch-query.el |  2 +-
 notmuch-show.c | 22 +-
 test/T350-crypto.sh| 12 ++--
 test/T357-index-decryption.sh  |  6 +++---
 test/T450-emacs-show.sh|  2 +-
 8 files changed, 47 insertions(+), 42 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index fb093de8..4ab2e5f6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,10 +517,14 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body|--decrypt)
+   --exclude|--body)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
+--decrypt)
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 64caa7a6..7d2b38cb 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -115,22 +115,27 @@ Supported options for **show** include
 supported with --format=json and --format=sexp), and the
 multipart/signed part will be replaced by the signed data.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
-
-Implies --verify.
+``--decrypt=(false|auto|true)``
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e. "multipart/encrypted" parts). Status of
+the decryption will be reported (currently only supported
+with --format=json and --format=sexp) and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will fail.
+
+Note: ``true`` implies --verify.
+
+Default: ``auto``
 
 ``--exclude=(true|false)``
 Specify whether to omit threads only matching
diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 010be454..a7e02710 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -593,7 +593,7 @@ the given type."
   (set-buffer-multibyte nil))
 (let ((args `("show" "--format=raw"
   ,(format "--part=%s" (plist-get part :id))
-  ,@(when process-crypto '("--decrypt"))
+  ,@(when process-crypto '("--decrypt=true"))
   ,(notmuch-id-to-query (plist-get msg :id
   (coding-system-for-read
(if binaryp 'no-conversion
diff --git a/emacs/notmuch-query.el b/emacs/notmuch-query.el
index 592fd8f1..563e4acf 100644
--- a/emacs/notmuch-query.el
+++ b/emacs/notmuch-query.el
@@ -32,7 +32,7 @@ is a possibly empty forest of replies.
 "
   (let ((args '("show" "--format=sexp" "--format-version=4")))
 (if notmuch-show-process-crypto
-   (setq args (append args '("--decrypt"
+   (setq args (append args '("--decrypt=true"
 (setq args (append args search-terms))
 (apply #'notmuch-call-notmuch-sexp args)))
 
diff --git a/notmuch-show.c b/notmuch-show.c
index d5adc370..9871159d 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1085,8 +1085,6 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {

[PATCH v2] cli/reply: make --decrypt take a keyword

2017-12-11 Thread Daniel Kahn Gillmor 
This brings the --decrypt argument to "notmuch reply" into line with
the other --decrypt arguments (in "show", "new", "insert", and
"reindex").  This patch is really just about bringing consistency to
the user interface.

We also use the recommended form in the emacs MUA when replying, and
update test T350 to match.
---
 completion/notmuch-completion.bash |  2 +-
 doc/man1/notmuch-reply.rst | 34 --
 emacs/notmuch-mua.el   |  2 +-
 notmuch-reply.c| 11 ++-
 test/T350-crypto.sh|  2 +-
 5 files changed, 29 insertions(+), 22 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 4ab2e5f6..a24b8a08 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -351,7 +351,7 @@ _notmuch_reply()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst
index ede77930..1b62e075 100644
--- a/doc/man1/notmuch-reply.rst
+++ b/doc/man1/notmuch-reply.rst
@@ -72,20 +72,26 @@ Supported options for **reply** include
 in this order, and copy values from the first that contains
 something other than only the user's addresses.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will likely fail.
+``--decrypt=(false|auto|true)``
+
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e., "multipart/encrypted" parts). Status
+of the decryption will be reported (currently only supported
+with --format=json and --format=sexp), and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's secret keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will likely fail.
+
+Default: ``auto``
 
 See **notmuch-search-terms(7)** for details of the supported syntax for
 .
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 7a341ebf..59b546a6 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -181,7 +181,7 @@ mutiple parts get a header."
reply
original)
 (when process-crypto
-  (setq args (append args '("--decrypt"
+  (setq args (append args '("--decrypt=true"
 
 (if reply-all
(setq args (append args '("--reply-to=all")))
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 5cdf642b..75cf7ecb 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -704,8 +704,6 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
 };
 int format = FORMAT_DEFAULT;
 int reply_all = true;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -719,7 +717,12 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "all", true },
  { "sender", false },
  { 0, 0 } } },
-   { .opt_bool = , .name = "decrypt", .present = _set },
+   { .opt_keyword = (int*)(), .name = "decrypt",
+ .keyword_no_arg_value = "true", .keywords =
+ (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
+ { "auto", NOTMUCH_DECRYPT_AUTO },
+ { "true", NOTMUCH_DECRYPT_NOSTASH },
+ { 0, 0 } } },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -729,8 +732,6 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 
 notmuch_process_shared_options (argv[0]);
-if (decrypt_set)
-   params.crypto.decrypt = decrypt ? NOTMUCH_DECRYPT_NOSTASH :

[PATCH] make release archive: common (or no) timestamps

2017-12-11 Thread Tomi Ollila 
The appended file 'version' has the same timestamp as the files added
by `git archive`.

The original file name and time stamp are no longer saved to the
gzip header in resulting $(PACKAGE)-$(VERSION).tar.gz file.

When build environment is close enough to another, this may
provide mutually reproducible release archive files.
---

The best way to realize improvements in a patch set is to send it >;/

V2 of id:20171211173102.6279-1-tomi.oll...@iki.fi

Replaces file touch with --mtime= GNU tar(1) option (diffdiff with V1):

: +++ b/Makefile.local
: @@ -34,6 +34,5 @@ $(TAR_FILE):
: ct=`git --no-pager log -1 --pretty=format:%ct $$ref` ; \
: -   touch -d @$$ct version.tmp
: tar --owner root --group root --append -f $(TAR_FILE).tmp \
: --transform s_^_$(PACKAGE)-$(VERSION)/_  \
: -   --transform 's_.tmp$$__' version.tmp
: +   --transform 's_.tmp$$__' --mtime=@$$ct version.tmp
: rm version.tmp


 Makefile.local | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index 9505b7fee70b..1535c2ae8a7b 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -31,11 +31,12 @@ $(TAR_FILE):
fi ; \
git archive --format=tar --prefix=$(PACKAGE)-$(VERSION)/ $$ref > 
$(TAR_FILE).tmp
echo $(VERSION) > version.tmp
+   ct=`git --no-pager log -1 --pretty=format:%ct $$ref` ; \
tar --owner root --group root --append -f $(TAR_FILE).tmp \
--transform s_^_$(PACKAGE)-$(VERSION)/_  \
-   --transform 's_.tmp$$__' version.tmp
+   --transform 's_.tmp$$__' --mtime=@$$ct version.tmp
rm version.tmp
-   gzip < $(TAR_FILE).tmp > $(TAR_FILE)
+   gzip -n < $(TAR_FILE).tmp > $(TAR_FILE)
@echo "Source is ready for release in $(TAR_FILE)"
 
 $(SHA256_FILE): $(TAR_FILE)
-- 
2.13.3

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] properties: add notmuch_message_count_properties

2017-12-11 Thread Daniel Kahn Gillmor 
The user can already do this manually, of course, but (a) it's nice to
have a convenience function, and (b) exposing this interface means
that someone more clever with a _notmuch_string_map_t than i am can
write a more efficient version if they like, and it will just
accelerate the users of the convenience function.
---
 lib/message-property.cc | 25 +
 lib/notmuch.h   | 16 
 2 files changed, 41 insertions(+)

diff --git a/lib/message-property.cc b/lib/message-property.cc
index 2e44a386..7894016c 100644
--- a/lib/message-property.cc
+++ b/lib/message-property.cc
@@ -36,6 +36,31 @@ notmuch_message_get_property (notmuch_message_t *message, 
const char *key, const
 return NOTMUCH_STATUS_SUCCESS;
 }
 
+notmuch_status_t
+notmuch_message_count_properties (notmuch_message_t *message, const char *key, 
unsigned int *count)
+{
+if (! count || ! key || ! message)
+   return NOTMUCH_STATUS_NULL_POINTER;
+
+notmuch_string_map_t *map;
+map = _notmuch_message_property_map (message);
+if (! map)
+   return NOTMUCH_STATUS_NULL_POINTER;
+
+notmuch_string_map_iterator_t *matcher = 
_notmuch_string_map_iterator_create (map, key, true);
+if (! matcher)
+   return NOTMUCH_STATUS_OUT_OF_MEMORY;
+
+*count = 0;
+while (_notmuch_string_map_iterator_valid (matcher)) {
+   (*count)++;
+   _notmuch_string_map_iterator_move_to_next (matcher);
+}
+
+_notmuch_string_map_iterator_destroy (matcher);
+return NOTMUCH_STATUS_SUCCESS;
+}
+
 static notmuch_status_t
 _notmuch_message_modify_property (notmuch_message_t *message, const char *key, 
const char *value,
  bool delete_it)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index bcf9b68a..141425ee 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1890,6 +1890,22 @@ typedef struct _notmuch_string_map_iterator 
notmuch_message_properties_t;
 notmuch_message_properties_t *
 notmuch_message_get_properties (notmuch_message_t *message, const char *key, 
notmuch_bool_t exact);
 
+/**
+ * Return the number of properties named "key" belonging to the specific 
message.
+ *
+ * @param[in] message  The message to examine
+ * @param[in] key  key to count
+ * @param[out] count   The number of matching properties associated with this 
message.
+ *
+ * @returns
+ *
+ * NOTMUCH_STATUS_SUCCESS: successful count, possibly some other error.
+ *
+ * @since libnotmuch 5.1 (notmuch 0.26)
+ */
+notmuch_status_t
+notmuch_message_count_properties (notmuch_message_t *message, const char *key, 
unsigned int *count);
+
 /**
  * Is the given *properties* iterator pointing at a valid (key,value)
  * pair.
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/5] properties: add notmuch_message_count_properties

2017-12-11 Thread Daniel Kahn Gillmor 
The user can already do this manually, of course, but (a) it's nice to
have a convenience function, and (b) exposing this interface means
that someone more clever with a _notmuch_string_map_t than i am can
write a more efficient version if they like, and it will just
accelerate the users of the convenience function.
---
 lib/message-property.cc | 25 +
 lib/notmuch.h   | 15 +++
 2 files changed, 40 insertions(+)

diff --git a/lib/message-property.cc b/lib/message-property.cc
index 2e44a386..7894016c 100644
--- a/lib/message-property.cc
+++ b/lib/message-property.cc
@@ -36,6 +36,31 @@ notmuch_message_get_property (notmuch_message_t *message, 
const char *key, const
 return NOTMUCH_STATUS_SUCCESS;
 }
 
+notmuch_status_t
+notmuch_message_count_properties (notmuch_message_t *message, const char *key, 
unsigned int *count)
+{
+if (! count || ! key || ! message)
+   return NOTMUCH_STATUS_NULL_POINTER;
+
+notmuch_string_map_t *map;
+map = _notmuch_message_property_map (message);
+if (! map)
+   return NOTMUCH_STATUS_NULL_POINTER;
+
+notmuch_string_map_iterator_t *matcher = 
_notmuch_string_map_iterator_create (map, key, true);
+if (! matcher)
+   return NOTMUCH_STATUS_OUT_OF_MEMORY;
+
+*count = 0;
+while (_notmuch_string_map_iterator_valid (matcher)) {
+   (*count)++;
+   _notmuch_string_map_iterator_move_to_next (matcher);
+}
+
+_notmuch_string_map_iterator_destroy (matcher);
+return NOTMUCH_STATUS_SUCCESS;
+}
+
 static notmuch_status_t
 _notmuch_message_modify_property (notmuch_message_t *message, const char *key, 
const char *value,
  bool delete_it)
diff --git a/lib/notmuch.h b/lib/notmuch.h
index bcf9b68a..9e62766b 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -1890,6 +1890,21 @@ typedef struct _notmuch_string_map_iterator 
notmuch_message_properties_t;
 notmuch_message_properties_t *
 notmuch_message_get_properties (notmuch_message_t *message, const char *key, 
notmuch_bool_t exact);
 
+/**
+ * Return the number of properties named "key" belonging to the specific 
message.
+ *
+ * @param[in] message  The message to examine
+ * @param[in] key  key to count
+ *
+ * @returns
+ *
+ * NOTMUCH_STATUS_SUCCESS: successful count, possibly some other error.
+ *
+ * @since libnotmuch 5.1 (notmuch 0.26)
+ */
+notmuch_status_t
+notmuch_message_count_properties (notmuch_message_t *message, const char *key, 
unsigned int *count);
+
 /**
  * Is the given *properties* iterator pointing at a valid (key,value)
  * pair.
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 4/5] cli/show: reindex when we learned new session keys about a message

2017-12-11 Thread Daniel Kahn Gillmor 
If the number of session keys for a given message increased after
running "notmuch show" then we just learned something new that might
let us do automatic decryption.  We should reindex this message using
our newfound knowledge.
---
 notmuch-show.c | 20 
 1 file changed, 20 insertions(+)

diff --git a/notmuch-show.c b/notmuch-show.c
index 9871159d..90e45cd9 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -873,6 +873,11 @@ show_message (void *ctx,
 void *local = talloc_new (ctx);
 mime_node_t *root, *part;
 notmuch_status_t status;
+unsigned int session_keys = 0;
+notmuch_status_t session_key_count_error = NOTMUCH_STATUS_SUCCESS;
+
+if (params->crypto.decrypt == NOTMUCH_DECRYPT_TRUE)
+   session_key_count_error = notmuch_message_count_properties (message, 
"session-key", _keys);
 
 status = mime_node_open (local, message, &(params->crypto), );
 if (status)
@@ -880,6 +885,21 @@ show_message (void *ctx,
 part = mime_node_seek_dfs (root, (params->part < 0 ? 0 : params->part));
 if (part)
status = format->part (local, sp, part, indent, params);
+
+if (params->crypto.decrypt == NOTMUCH_DECRYPT_TRUE && 
session_key_count_error == NOTMUCH_STATUS_SUCCESS) {
+   unsigned int new_session_keys = 0;
+   if (notmuch_message_count_properties (message, "session-key", 
_session_keys) == NOTMUCH_STATUS_SUCCESS &&
+   new_session_keys > session_keys) {
+   /* try a quiet re-indexing */
+   notmuch_indexopts_t *indexopts = 
notmuch_database_get_default_indexopts (notmuch_message_get_database (message));
+   if (indexopts) {
+   notmuch_indexopts_set_decrypt_policy (indexopts, 
NOTMUCH_DECRYPT_AUTO);
+   status = notmuch_message_reindex (message, indexopts);
+   if (status)
+   fprintf (stderr, "Error re-indexing message with 
--decrypt=stash. (%d) %s\n", status, notmuch_status_to_string (status));
+   }
+   }
+}
   DONE:
 talloc_free (local);
 return status;
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/5] cli: write session keys to database, if asked to do so

2017-12-11 Thread Daniel Kahn Gillmor 
If the decryption policy is NOTMUCH_DECRYPT_TRUE, that means we want
to stash session keys in the database.  Note that there is currently
no way from the command line to set it this way, though, so it is not
yet included in the test suite.
---
 mime-node.c | 24 
 1 file changed, 20 insertions(+), 4 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index 11df082b..75b79f98 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -197,16 +197,18 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
 GError *err = NULL;
 GMimeDecryptResult *decrypt_result = NULL;
 GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
+notmuch_message_t *message = NULL;
 
 if (! node->decrypted_child) {
-   mime_node_t *parent;
-   for (parent = node; parent; parent = parent->parent)
-   if (parent->envelope_file)
+   for (mime_node_t *parent = node; parent; parent = parent->parent)
+   if (parent->envelope_file) {
+   message = parent->envelope_file;
break;
+   }
 
node->decrypted_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
 
node->ctx->crypto->decrypt,
-parent ? 
parent->envelope_file : NULL,
+message,
 cryptoctx, 
encrypteddata, _result, );
 }
 if (! node->decrypted_child) {
@@ -225,6 +227,20 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part,
g_object_ref (node->sig_list);
set_signature_list_destructor (node);
}
+
+#if HAVE_GMIME_SESSION_KEYS
+   if (node->ctx->crypto->decrypt == NOTMUCH_DECRYPT_TRUE && message) {
+   notmuch_database_t *db = notmuch_message_get_database (message);
+   const char *sk = g_mime_decrypt_result_get_session_key 
(decrypt_result);
+   if (db && sk) {
+   notmuch_status_t status;
+   status = notmuch_message_add_property (message, "session-key", 
sk);
+   if (status)
+   fprintf (stderr, "Failed to stash session key in the 
database (%d) %s\n",
+status, notmuch_status_to_string (status));
+   }
+   }
+#endif
g_object_unref (decrypt_result);
 }
 
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/5] cli/show: enable --decrypt=stash

2017-12-11 Thread Daniel Kahn Gillmor 
Add fancy new feature, which makes "notmuch show" capable of actually
indexing messages that it just decrypted.

This enables a workflow where messages can come in in the background
and be indexed using "--decrypt=auto".  But when showing an encrypted
message for the first time, it gets automatically indexed.

This is something of a departure for "notmuch show" -- in particular,
because it requires read/write access to the database.  However, this
might be a common use case -- people get mail delivered and indexed in
the background, but only want access to their secret key to happen
when they're directly interacting with notmuch itself.

In such a scenario, they couldn't search newly-delivered, encrypted
messages, but they could search for them once they've read them.

Documentation of this new feature also uses a table form, similar to
that found in the description of index.decrypt in notmuch-config(1).

A notmuch UI that wants to facilitate this workflow while also
offering an interactive search interface might instead make use of
these additional commands while the user is at the console:

Count received encrypted messages (if > 0, there are some things we
haven't yet tried to index, and therefore can't yet search):

 notmuch count tag:encrypted and \
 not property:index.decryption=success and \
 not property:index.decryption=failure

Reindex those messages:

 notmuch reindex --try-decrypt=true tag:encrypted and \
 not property:index.decryption=success and \
 not property:index.decryption=failure
---
 completion/notmuch-completion.bash |  2 +-
 doc/man1/notmuch-show.rst  | 35 ---
 notmuch-show.c |  9 +++--
 test/T357-index-decryption.sh  | 18 ++
 4 files changed, 58 insertions(+), 6 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index a24b8a08..16ae3992 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -522,7 +522,7 @@ _notmuch_show()
return
;;
 --decrypt)
-   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true auto false stash" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 7d2b38cb..5fd9876e 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -115,7 +115,7 @@ Supported options for **show** include
 supported with --format=json and --format=sexp), and the
 multipart/signed part will be replaced by the signed data.
 
-``--decrypt=(false|auto|true)``
+``--decrypt=(false|auto|true|stash)``
 If ``true``, decrypt any MIME encrypted parts found in the
 selected content (i.e. "multipart/encrypted" parts). Status of
 the decryption will be reported (currently only supported
@@ -123,17 +123,46 @@ Supported options for **show** include
 decryption the multipart/encrypted part will be replaced by
 the decrypted content.
 
+``stash`` behaves like ``true``, but upon successful
+decryption it will also stash the message's session key in the
+database, and index the cleartext of the message, enabling
+automatic decryption in the future.
+
 If ``auto``, and a session key is already known for the
 message, then it will be decrypted, but notmuch will not try
 to access the user's keys.
 
 Use ``false`` to avoid even automatic decryption.
 
-Non-automatic decryption expects a functioning
+Non-automatic decryption (``stash`` or ``true``, in the
+absence of a stashed session key) expects a functioning
 **gpg-agent(1)** to provide any needed credentials. Without
 one, the decryption will fail.
 
-Note: ``true`` implies --verify.
+Note: setting either ``true`` or ``stash`` here implies
+``--verify``.
+
+Here is a table that summarizes each of these policies:
+
+++---+--+--+---+
+|| false | auto | true | stash |
+++===+==+==+===+
+| Show cleartext if  |   |  X   |  X   |   X   |
+| session key is |   |  |  |   |
+| already known  |   |  |  |   |
+++---+--+--+---+
+| Use secret keys to |   |  |  X   |   X   |
+| show cleartext |   |  |  |   |
+++---+--+--+---+
+| Stash any newly|   |  |  |   X   |
+| recovered session keys,|   |  |  |   |
+| reindexing message if  |   |  |  |   |
+|

notmuch show --decrypt=stash

2017-12-11 Thread Daniel Kahn Gillmor 
This series allows "notmuch show" to index the cleartext and stash the
session keys of an encrypted message while displaying it.

Because it uses a keyword argument to --decrypt for "notmuch show", It
needs to be applied *after* the series with the subject:

Encourage explicit arguments for --decrypt in "show" and "reply"


Background
--

The cleartext indexing and session-keys series make working with
encrypted e-mail significantly easier in notmuch.  However, their
underlying assumption is that at the time of message ingestion (and
"notmuch new" in particular), the user is likely to have access to
their long-term secret keys.

In practice, many people using GnuPG today have their secret keys
locked behind a passphrase, or on a smartcard, and also run "notmuch
new" in some sort of scheduled, backgrounded process.

The result is that for users with this workflow, GnuPG prompts for
their passphrase (or to trigger their smartcard) at unpredictable
times, depending on when their mail delivery happens, and on how many
encrypted messages they receive.  This is both unfriendly and bad for
security (we should not train users to approve random prompts for
secret key access when nothing they're doing interactively seems to
warrant it).

Outline
---

For a friendlier experience, some users may prefer incoming encrypted
mail to stay in their inbox *without* being decrypted, until they
choose to look at it.  At the moment that they're looking at it, their
MUA is in the foreground and they're interacting with it, so being
prompted for their password or smartcard interactively makes sense at
that time.

This series makes it possible for this interaction to to actually
decrypt the message, index it, and stash any session keys the first
time the user interacts with the message through "notmuch show".

This is not a workflow that every MUA will choose to use (e.g. users
whose decryption-capable secret key is already cheaply available
without hassling the user at "notmuch new" shouldn't use it), but it
is a sensible workflow for some users that notmuch should support.

Furthermore, it is a more efficient use of secret key material -- a
user that wants to stash session keys of a message, but whose
long-term decryption secret key is on a smartcard should only be obliged
to trigger the smartcard once per message, ever.

Implementation details
--

The most controversial part of this series is that it makes "notmuch
show" potentially not a read-only operation on the database.  This is
a tradeoff that the users of this workflow will need to consider,
since they are explicitly asking "notmuch show" to potentially modify
their index.

Note that i've made this R/O-to-R/W switch fairly coarse.  If the user
requests --decrypt=stash, then "notmuch show" will operate on a
read/write database, regardless of whether the message is actually
encrypted.  I used this coarse approach because i couldn't figure out
a safe way to reopen an existing read-only database in read-write
mode.  If someone more clever with Xapian than me wants to suggest a
way to do this in a more fine-grained fashion, i'd welcome patches or
pointers.

I welcome review and feedback.

  --dkg

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/5] lib: expose notmuch_message_get_database()

2017-12-11 Thread Daniel Kahn Gillmor 
We've had _notmuch_message_database() internally for a while, and it's
useful.  It turns out to be useful on the other side of the library
interface as well (i'll use it later in this series for "notmuch
show"), so we expose it publicly now.
---
 lib/index.cc| 10 +-
 lib/message-property.cc |  4 ++--
 lib/message.cc  | 14 +++---
 lib/notmuch-private.h   |  2 --
 lib/notmuch.h   |  8 
 5 files changed, 22 insertions(+), 16 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 0ad683fa..22ca9ec1 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -385,7 +385,7 @@ _index_mime_part (notmuch_message_t *message,
 const char *charset;
 
 if (! part) {
-   _notmuch_database_log (_notmuch_message_database (message),
+   _notmuch_database_log (notmuch_message_get_database (message),
  "Warning: Not indexing empty mime part.\n");
return;
 }
@@ -411,7 +411,7 @@ _index_mime_part (notmuch_message_t *message,
 g_mime_multipart_get_part (multipart, 
i));
continue;
} else if (i != GMIME_MULTIPART_SIGNED_CONTENT) {
-   _notmuch_database_log (_notmuch_message_database (message),
+   _notmuch_database_log (notmuch_message_get_database 
(message),
   "Warning: Unexpected extra parts of 
multipart/signed. Indexing anyway.\n");
}
}
@@ -424,7 +424,7 @@ _index_mime_part (notmuch_message_t *message,
   GMIME_MULTIPART_ENCRYPTED 
(part));
} else {
if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) {
-   _notmuch_database_log (_notmuch_message_database 
(message),
+   _notmuch_database_log (notmuch_message_get_database 
(message),
   "Warning: Unexpected extra parts 
of multipart/encrypted.\n");
}
}
@@ -447,7 +447,7 @@ _index_mime_part (notmuch_message_t *message,
 }
 
 if (! (GMIME_IS_PART (part))) {
-   _notmuch_database_log (_notmuch_message_database (message),
+   _notmuch_database_log (notmuch_message_get_database (message),
  "Warning: Not indexing unknown mime part: %s.\n",
  g_type_name (G_OBJECT_TYPE (part)));
return;
@@ -528,7 +528,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 if (!indexopts || (notmuch_indexopts_get_decrypt_policy (indexopts) == 
NOTMUCH_DECRYPT_FALSE))
return;
 
-notmuch = _notmuch_message_database (message);
+notmuch = notmuch_message_get_database (message);
 
 GMimeCryptoContext* crypto_ctx = NULL;
 #if (GMIME_MAJOR_VERSION < 3)
diff --git a/lib/message-property.cc b/lib/message-property.cc
index 35eaf3c6..2e44a386 100644
--- a/lib/message-property.cc
+++ b/lib/message-property.cc
@@ -44,7 +44,7 @@ _notmuch_message_modify_property (notmuch_message_t *message, 
const char *key, c
 notmuch_status_t status;
 char *term = NULL;
 
-status = _notmuch_database_ensure_writable (_notmuch_message_database 
(message));
+status = _notmuch_database_ensure_writable (notmuch_message_get_database 
(message));
 if (status)
return status;
 
@@ -92,7 +92,7 @@ _notmuch_message_remove_all_properties (notmuch_message_t 
*message, const char *
 notmuch_status_t status;
 const char * term_prefix;
 
-status = _notmuch_database_ensure_writable (_notmuch_message_database 
(message));
+status = _notmuch_database_ensure_writable (notmuch_message_get_database 
(message));
 if (status)
return status;
 
diff --git a/lib/message.cc b/lib/message.cc
index d5db89b6..0886b22d 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -268,7 +268,7 @@ _notmuch_message_create_for_message_id (notmuch_database_t 
*notmuch,
 
doc_id = _notmuch_database_generate_doc_id (notmuch);
 } catch (const Xapian::Error ) {
-   _notmuch_database_log(_notmuch_message_database (message), "A Xapian 
exception occurred creating message: %s\n",
+   _notmuch_database_log(notmuch_message_get_database (message), "A Xapian 
exception occurred creating message: %s\n",
 error.get_msg().c_str());
notmuch->exception_reported = true;
*status_ret = NOTMUCH_PRIVATE_STATUS_XAPIAN_EXCEPTION;
@@ -495,7 +495,7 @@ _notmuch_message_ensure_message_file (notmuch_message_t 
*message)
return;
 
 message->message_file = _notmuch_message_file_open_ctx (
-   _notmuch_message_database (message), message, filename);
+   notmuch_message_get_database (message), message, filename);
 }
 
 const char *
@@ -525,7 +525,7 @@ notmuch_message_get_header (notmuch_message_t *message, 
const char *header)
return talloc_strdup (message, value.c_str ());
 
} catch

Re: [PATCH v2] cli/show: make --decrypt take a keyword.

2017-12-11 Thread Daniel Kahn Gillmor 
On Mon 2017-12-11 21:33:57 -0500, Daniel Kahn Gillmor wrote:
> We also expand tab completion for it, and update T357 to match.
>
> Make use of the bool-to-keyword backward-compatibility feature.

v2 of this patch just removes an unnecessary attempt to open the
database read-write.  That isn't necessary here, though it will be
necessary if we want to enable future versions of "notmuch show" that
can also interactively add a message to the index.

I removed it from this series so that the interface normalization here
(which is valuable whether or not we decide to augment "notmuch show" as
described) doesn't get distracted by the change.

sorry for the quick revision.

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] cli/show: make --decrypt take a keyword.

2017-12-11 Thread Daniel Kahn Gillmor 
We also expand tab completion for it, and update T357 to match.

Make use of the bool-to-keyword backward-compatibility feature.
---
 completion/notmuch-completion.bash |  6 +-
 doc/man1/notmuch-show.rst  | 37 +
 notmuch-show.c | 22 +-
 test/T357-index-decryption.sh  |  6 +++---
 4 files changed, 38 insertions(+), 33 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index fb093de8..4ab2e5f6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,10 +517,14 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body|--decrypt)
+   --exclude|--body)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
+--decrypt)
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 64caa7a6..7d2b38cb 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -115,22 +115,27 @@ Supported options for **show** include
 supported with --format=json and --format=sexp), and the
 multipart/signed part will be replaced by the signed data.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
-
-Implies --verify.
+``--decrypt=(false|auto|true)``
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e. "multipart/encrypted" parts). Status of
+the decryption will be reported (currently only supported
+with --format=json and --format=sexp) and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will fail.
+
+Note: ``true`` implies --verify.
+
+Default: ``auto``
 
 ``--exclude=(true|false)``
 Specify whether to omit threads only matching
diff --git a/notmuch-show.c b/notmuch-show.c
index d5adc370..9871159d 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1085,8 +1085,6 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -1101,7 +1099,12 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
{ .opt_bool = _thread, .name = "entire-thread",
  .present = _thread_set },
{ .opt_int = , .name = "part" },
-   { .opt_bool = , .name = "decrypt", .present = _set },
+   { .opt_keyword = (int*)(), .name = "decrypt",
+ .keyword_no_arg_value = "true", .keywords =
+ (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
+ { "auto", NOTMUCH_DECRYPT_AUTO },
+ { "true", NOTMUCH_DECRYPT_NOSTASH },
+ { 0, 0 } } },
{ .opt_bool = , .name = "verify" },
{ .opt_bool = _body, .name = "body" },
{ .opt_bool = _html, .name = "include-html" },
@@ -1115,16 +1118,9 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
-if (decrypt_set) {
-   if (decrypt) {
-   /* we do not need or want to ask for session keys */
-   params.crypto.decrypt = NOTMUCH_DECRYPT_NOSTASH;
-   /* decryption implies verification */
-   params.crypto.verify = true;
-   } else {
-   params.crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
-   }
-}
+/* explicit decryption implies verification */
+if (params.crypto.decrypt

Encourage explicit arguments for --decrypt in "show" and "reply"

2017-12-11 Thread Daniel Kahn Gillmor 
The notmuch indexing subcommands ("new", "insert", and "reindex") now
have a --decrypt option that takes an argument (the decryption
policy), since the session-keys patches have landed.

But the viewing subcommands ("show" and "reply") have their
traditional --decrypt option that (as a boolean) need not take an
argument, having --decrypt not present means something different from
either --decrypt=true or --decrypt=false.

This series allows the user to explicitly choose --decrypt=auto for
the viewing subcommands, while allowing people to use the
argument-free form (as an alias for --decrypt=true), but warns the
user to encourage them to switch to using an explicit argument
instead.

This is useful normalizing work for the interface, so it's worthwhile
on its own. It is also necessary preparation in the event that we
decide we want to:

 * set up a notmuch configuration option that changes the default for
   --decrypt for the viewing subcommands
 
 * allow "notmuch show" to actually index encrypted messages upon
   their first encounter (e.g., via a new decryption policy, which
   i'll propose separately)

As always, review and feedback welcome!

   --dkg


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/3] cli/show: make --decrypt take a keyword.

2017-12-11 Thread Daniel Kahn Gillmor 
We also expand tab completion for it, and update T357 to match.

Make use of the bool-to-keyword backward-compatibility feature.
---
 completion/notmuch-completion.bash |  6 +-
 doc/man1/notmuch-show.rst  | 37 +
 notmuch-show.c | 27 +--
 test/T357-index-decryption.sh  |  6 +++---
 4 files changed, 42 insertions(+), 34 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index fb093de8..4ab2e5f6 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -517,10 +517,14 @@ _notmuch_show()
COMPREPLY=( $( compgen -W "text json sexp mbox raw" -- "${cur}" ) )
return
;;
-   --exclude|--body|--decrypt)
+   --exclude|--body)
COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
return
;;
+--decrypt)
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
+   return
+   ;;
 esac
 
 ! $split &&
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 64caa7a6..7d2b38cb 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -115,22 +115,27 @@ Supported options for **show** include
 supported with --format=json and --format=sexp), and the
 multipart/signed part will be replaced by the signed data.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will fail.
-
-Implies --verify.
+``--decrypt=(false|auto|true)``
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e. "multipart/encrypted" parts). Status of
+the decryption will be reported (currently only supported
+with --format=json and --format=sexp) and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will fail.
+
+Note: ``true`` implies --verify.
+
+Default: ``auto``
 
 ``--exclude=(true|false)``
 Specify whether to omit threads only matching
diff --git a/notmuch-show.c b/notmuch-show.c
index d5adc370..ddd3c8c5 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1085,8 +1085,6 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
 bool exclude = true;
 bool entire_thread_set = false;
 bool single_message;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -1101,7 +1099,12 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
{ .opt_bool = _thread, .name = "entire-thread",
  .present = _thread_set },
{ .opt_int = , .name = "part" },
-   { .opt_bool = , .name = "decrypt", .present = _set },
+   { .opt_keyword = (int*)(), .name = "decrypt",
+ .keyword_no_arg_value = "true", .keywords =
+ (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
+ { "auto", NOTMUCH_DECRYPT_AUTO },
+ { "true", NOTMUCH_DECRYPT_NOSTASH },
+ { 0, 0 } } },
{ .opt_bool = , .name = "verify" },
{ .opt_bool = _body, .name = "body" },
{ .opt_bool = _html, .name = "include-html" },
@@ -1115,16 +1118,9 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
 
 notmuch_process_shared_options (argv[0]);
 
-if (decrypt_set) {
-   if (decrypt) {
-   /* we do not need or want to ask for session keys */
-   params.crypto.decrypt = NOTMUCH_DECRYPT_NOSTASH;
-   /* decryption implies verification */
-   params.crypto.verify = true;
-   } else {
-   params.crypto.decrypt = NOTMUCH_DECRYPT_FALSE;
-   }
-}
+/* explicit decryption implies verification */
+if

[PATCH 3/3] cli/reply: make --decrypt take a keyword

2017-12-11 Thread Daniel Kahn Gillmor 
This brings the --decrypt argument to "notmuch reply" into line with
the other --decrypt arguments (in "show", "new", "insert", and
"reindex").  This patch is really just about bringing consistency to
the user interface.
---
 completion/notmuch-completion.bash |  2 +-
 doc/man1/notmuch-reply.rst | 34 --
 notmuch-reply.c| 11 ++-
 3 files changed, 27 insertions(+), 20 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index 4ab2e5f6..a24b8a08 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -351,7 +351,7 @@ _notmuch_reply()
return
;;
--decrypt)
-   COMPREPLY=( $( compgen -W "true false" -- "${cur}" ) )
+   COMPREPLY=( $( compgen -W "true auto false" -- "${cur}" ) )
return
;;
 esac
diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst
index ede77930..1b62e075 100644
--- a/doc/man1/notmuch-reply.rst
+++ b/doc/man1/notmuch-reply.rst
@@ -72,20 +72,26 @@ Supported options for **reply** include
 in this order, and copy values from the first that contains
 something other than only the user's addresses.
 
-``--decrypt``
-Decrypt any MIME encrypted parts found in the selected content
-(ie. "multipart/encrypted" parts). Status of the decryption will
-be reported (currently only supported with --format=json and
---format=sexp) and on successful decryption the
-multipart/encrypted part will be replaced by the decrypted
-content.
-
-If a session key is already known for the message, then it
-will be decrypted automatically unless the user explicitly
-sets ``--decrypt=false``.
-
-Decryption expects a functioning **gpg-agent(1)** to provide any
-needed credentials. Without one, the decryption will likely fail.
+``--decrypt=(false|auto|true)``
+
+If ``true``, decrypt any MIME encrypted parts found in the
+selected content (i.e., "multipart/encrypted" parts). Status
+of the decryption will be reported (currently only supported
+with --format=json and --format=sexp), and on successful
+decryption the multipart/encrypted part will be replaced by
+the decrypted content.
+
+If ``auto``, and a session key is already known for the
+message, then it will be decrypted, but notmuch will not try
+to access the user's secret keys.
+
+Use ``false`` to avoid even automatic decryption.
+
+Non-automatic decryption expects a functioning
+**gpg-agent(1)** to provide any needed credentials. Without
+one, the decryption will likely fail.
+
+Default: ``auto``
 
 See **notmuch-search-terms(7)** for details of the supported syntax for
 .
diff --git a/notmuch-reply.c b/notmuch-reply.c
index 5cdf642b..75cf7ecb 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -704,8 +704,6 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
 };
 int format = FORMAT_DEFAULT;
 int reply_all = true;
-bool decrypt = false;
-bool decrypt_set = false;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -719,7 +717,12 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
  (notmuch_keyword_t []){ { "all", true },
  { "sender", false },
  { 0, 0 } } },
-   { .opt_bool = , .name = "decrypt", .present = _set },
+   { .opt_keyword = (int*)(), .name = "decrypt",
+ .keyword_no_arg_value = "true", .keywords =
+ (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
+ { "auto", NOTMUCH_DECRYPT_AUTO },
+ { "true", NOTMUCH_DECRYPT_NOSTASH },
+ { 0, 0 } } },
{ .opt_inherit = notmuch_shared_options },
{ }
 };
@@ -729,8 +732,6 @@ notmuch_reply_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 
 notmuch_process_shared_options (argv[0]);
-if (decrypt_set)
-   params.crypto.decrypt = decrypt ? NOTMUCH_DECRYPT_NOSTASH : 
NOTMUCH_DECRYPT_FALSE;
 
 notmuch_exit_if_unsupported_format ();
 
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/3] cli: some keyword options can be supplied with no argument

2017-12-11 Thread Daniel Kahn Gillmor 
We might change some notmuch command line tools that used to be
booleans into keyword arguments.

In that case, there are some legacy tools that will expect to be able
to do "notmuch foo --bar" instead of "notmuch foo --bar=baz".

This patch makes it possible to support that older API, while
providing a warning and an encouragement to upgrade.
---
 command-line-arguments.c  | 59 +++
 command-line-arguments.h  |  4 +++
 test/T410-argument-parsing.sh | 24 ++
 test/arg-test.c   | 12 -
 4 files changed, 82 insertions(+), 17 deletions(-)

diff --git a/command-line-arguments.c b/command-line-arguments.c
index db73ca5e..d0e21693 100644
--- a/command-line-arguments.c
+++ b/command-line-arguments.c
@@ -4,13 +4,19 @@
 #include "error_util.h"
 #include "command-line-arguments.h"
 
+typedef enum {
+OPT_FAILED, /* false */
+OPT_OK, /* good */
+OPT_GIVEBACK, /* pop one of the arguments you thought you were getting off 
the stack */
+} opt_handled;
+
 /*
   Search the array of keywords for a given argument, assigning the
   output variable to the corresponding value.  Return false if nothing
   matches.
 */
 
-static bool
+static opt_handled
 _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
 
 const notmuch_keyword_t *keywords;
@@ -29,16 +35,32 @@ _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
else
*arg_desc->opt_keyword = keywords->value;
 
-   return true;
+   return OPT_OK;
 }
+
+if (arg_desc->opt_keyword && arg_desc->keyword_no_arg_value && next != ':' 
&& next != '=') {
+   for (keywords = arg_desc->keywords; keywords->name; keywords++) {
+   if (strcmp (arg_desc->keyword_no_arg_value, keywords->name) != 0)
+   continue;
+
+   *arg_desc->opt_keyword = keywords->value;
+   fprintf (stderr, "Warning: No known keyword option given for 
\"%s\", choosing value \"%s\"."
+"  Please specify the argument explicitly!\n", 
arg_desc->name, arg_desc->keyword_no_arg_value);
+
+   return OPT_GIVEBACK;
+   }
+   fprintf (stderr, "No matching keyword for option \"%s\" and default 
value \"%s\" is invalid.\n", arg_str, arg_desc->name);
+   return OPT_FAILED;
+}
+
 if (next != '\0')
fprintf (stderr, "Unknown keyword argument \"%s\" for option 
\"%s\".\n", arg_str, arg_desc->name);
 else
fprintf (stderr, "Option \"%s\" needs a keyword argument.\n", 
arg_desc->name);
-return false;
+return OPT_FAILED;
 }
 
-static bool
+static opt_handled
 _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, char next, const 
char *arg_str) {
 bool value;
 
@@ -48,45 +70,45 @@ _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, 
char next, const char
value = false;
 } else {
fprintf (stderr, "Unknown argument \"%s\" for (boolean) option 
\"%s\".\n", arg_str, arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 
 *arg_desc->opt_bool = value;
 
-return true;
+return OPT_OK;
 }
 
-static bool
+static opt_handled
 _process_int_arg (const notmuch_opt_desc_t *arg_desc, char next, const char 
*arg_str) {
 
 char *endptr;
 if (next == '\0' || arg_str[0] == '\0') {
fprintf (stderr, "Option \"%s\" needs an integer argument.\n", 
arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 
 *arg_desc->opt_int = strtol (arg_str, , 10);
 if (*endptr == '\0')
-   return true;
+   return OPT_OK;
 
 fprintf (stderr, "Unable to parse argument \"%s\" for option \"%s\" as an 
integer.\n",
 arg_str, arg_desc->name);
-return false;
+return OPT_FAILED;
 }
 
-static bool
+static opt_handled
 _process_string_arg (const notmuch_opt_desc_t *arg_desc, char next, const char 
*arg_str) {
 
 if (next == '\0') {
fprintf (stderr, "Option \"%s\" needs a string argument.\n", 
arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 if (arg_str[0] == '\0' && ! arg_desc->allow_empty) {
fprintf (stderr, "String argument for option \"%s\" must be 
non-empty.\n", arg_desc->name);
-   return false;
+   return OPT_FAILED;
 }
 *arg_desc->opt_string = arg_str;
-return true;
+return OPT_OK;
 }
 
 /* Return number of non-NULL opt_* fields in opt_desc. */
@@ -186,13 +208,15 @@ parse_option (int argc, char **argv, const 
notmuch_opt_desc_t *options, int opt_
if (next != '=' && next != ':' && next != '\0')
continue;
 
+   bool incremented = false;
if (next == '\0' && next_arg != NULL && ! try->opt_bool) {
next = ' ';
value = next_arg;
+   incremented = true;
opt_index ++;
}
 
-   bool opt_status = false;
+   opt_handled opt_status = OPT_FAILED;
if (try->opt_keyword ||

[PATCH] Standards-Version: bumped to 4.1.2 (no changes needed)

2017-12-11 Thread Daniel Kahn Gillmor 
---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index f644695b..bf7eb647 100644
--- a/debian/control
+++ b/debian/control
@@ -28,7 +28,7 @@ Build-Depends:
  gpgsm ,
  gnupg ,
  bash-completion (>=1.9.0~)
-Standards-Version: 4.1.1
+Standards-Version: 4.1.2
 Homepage: https://notmuchmail.org/
 Vcs-Git: git://notmuchmail.org/git/notmuch
 Vcs-Browser: https://git.notmuchmail.org/git/notmuch
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4] python: add decrypt_policy argument to Database.index_file()

2017-12-11 Thread Daniel Kahn Gillmor 
We adopt a pythonic idiom here with an optional argument, rather than
exposing the user to the C indexopts object directly.

This now includes a simple test to ensure that the decrypt_policy
argument works as expected.
---
 bindings/python/notmuch/database.py | 45 +++--
 bindings/python/notmuch/globals.py  |  5 +
 test/T390-python.sh | 39 
 3 files changed, 87 insertions(+), 2 deletions(-)

diff --git a/bindings/python/notmuch/database.py 
b/bindings/python/notmuch/database.py
index 1279804a..2a07e346 100644
--- a/bindings/python/notmuch/database.py
+++ b/bindings/python/notmuch/database.py
@@ -28,6 +28,7 @@ from .globals import (
 _str,
 NotmuchDatabaseP,
 NotmuchDirectoryP,
+NotmuchIndexoptsP,
 NotmuchMessageP,
 NotmuchTagsP,
 )
@@ -72,6 +73,9 @@ class Database(object):
 MODE = Enum(['READ_ONLY', 'READ_WRITE'])
 """Constants: Mode in which to open the database"""
 
+DECRYPTION_POLICY = Enum(['FALSE', 'TRUE', 'AUTO', 'NOSTASH'])
+"""Constants: policies for decrypting messages during indexing"""
+
 """notmuch_database_get_directory"""
 _get_directory = nmlib.notmuch_database_get_directory
 _get_directory.argtypes = [NotmuchDatabaseP, c_char_p, 
POINTER(NotmuchDirectoryP)]
@@ -400,13 +404,25 @@ class Database(object):
 # return the Directory, init it with the absolute path
 return Directory(abs_dirpath, dir_p, self)
 
+_get_default_indexopts = nmlib.notmuch_database_get_default_indexopts
+_get_default_indexopts.argtypes = [NotmuchDatabaseP]
+_get_default_indexopts.restype = NotmuchIndexoptsP
+
+_indexopts_set_decrypt_policy = nmlib.notmuch_indexopts_set_decrypt_policy
+_indexopts_set_decrypt_policy.argtypes = [NotmuchIndexoptsP, c_uint]
+_indexopts_set_decrypt_policy.restype = None
+
+_indexopts_destroy = nmlib.notmuch_indexopts_destroy
+_indexopts_destroy.argtypes = [NotmuchIndexoptsP]
+_indexopts_destroy.restype = None
+
 _index_file = nmlib.notmuch_database_index_file
 _index_file.argtypes = [NotmuchDatabaseP, c_char_p,
  c_void_p,
  POINTER(NotmuchMessageP)]
 _index_file.restype = c_uint
 
-def index_file(self, filename, sync_maildir_flags=False):
+def index_file(self, filename, sync_maildir_flags=False, 
decrypt_policy=None):
 """Adds a new message to the database
 
 :param filename: should be a path relative to the path of the
@@ -427,6 +443,23 @@ class Database(object):
 API. You might want to look into the underlying method
 :meth:`Message.maildir_flags_to_tags`.
 
+:param decrypt_policy: If the message contains any encrypted
+parts, and decrypt_policy is set to
+:attr:`DECRYPTION_POLICY`.TRUE, notmuch will try to
+decrypt the message and index the cleartext, stashing any
+discovered session keys.  If it is set to
+:attr:`DECRYPTION_POLICY`.FALSE, it will never try to
+decrypt during indexing.  If it is set to
+:attr:`DECRYPTION_POLICY`.AUTO, then it will try to use
+any stashed session keys it knows about, but will not try
+to access the user's secret keys.
+:attr:`DECRYPTION_POLICY`.NOSTASH behaves the same as
+:attr:`DECRYPTION_POLICY`.TRUE except that no session keys
+are stashed in the database.  If decrypt_policy is set to
+None (the default), then the database itself will decide
+whether to decrypt, based on the `index.decrypt`
+configuration setting (see notmuch-config(1)).
+
 :returns: On success, we return
 
1) a :class:`Message` object that can be used for things
@@ -457,7 +490,15 @@ class Database(object):
 """
 self._assert_db_is_initialized()
 msg_p = NotmuchMessageP()
-status = self._index_file(self._db, _str(filename), c_void_p(None), 
byref(msg_p))
+indexopts = c_void_p(None)
+if decrypt_policy is not None:
+indexopts = self._get_default_indexopts(self._db)
+self._indexopts_set_decrypt_policy(indexopts, decrypt_policy)
+
+status = self._index_file(self._db, _str(filename), indexopts, 
byref(msg_p))
+
+if indexopts:
+self._indexopts_destroy(indexopts)
 
 if not status in [STATUS.SUCCESS, STATUS.DUPLICATE_MESSAGE_ID]:
 raise NotmuchError(status)
diff --git a/bindings/python/notmuch/globals.py 
b/bindings/python/notmuch/globals.py
index b1eec2cf..71426c84 100644
--- a/bindings/python/notmuch/globals.py
+++ b/bindings/python/notmuch/globals.py
@@ -88,3 +88,8 @@ NotmuchDirectoryP = POINTER(NotmuchDirectoryS)
 class NotmuchFilenamesS(Structure):
 pass
 NotmuchFilenamesP = POINTER(NotmuchFilenamesS)
+
+
+class NotmuchIndexoptsS(Structure):
+pass

Re: [PATCH 3/4] nmbug: Auto-checkout in clone if it wouldn't clobber

2017-12-11 Thread David Bremner 
"W. Trevor King"  writes:

> On Mon, Dec 11, 2017 at 09:10:47AM -0400, David Bremner wrote:
>> Not sure what happened to patch 4/4?
>
> Ah sorry.  My local branch has a commit to bump nmbug's internal
> version to 0.3 (see my comments in [1]), but I ended up deciding to
> punt that until before the next notmuch release (in case other nmbug
> changes landed in the meantime) and forgot to update the denominator.
>
> Based on [2], perhaps it is now time?  Or is [3] likely to land before
> the freeze?
>

I think bumping the version is something reasonable to do during the
feature freeze.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/4] nmbug: Auto-checkout in clone if it wouldn't clobber

2017-12-11 Thread W. Trevor King 
On Mon, Dec 11, 2017 at 09:10:47AM -0400, David Bremner wrote:
> Not sure what happened to patch 4/4?

Ah sorry.  My local branch has a commit to bump nmbug's internal
version to 0.3 (see my comments in [1]), but I ended up deciding to
punt that until before the next notmuch release (in case other nmbug
changes landed in the meantime) and forgot to update the denominator.

Based on [2], perhaps it is now time?  Or is [3] likely to land before
the freeze?

Cheers,
Trevor

[1]: id:cover.1507675236.git.wk...@tremily.us
 Subject: [PATCH 0/3] nmbug:
 Date: Tue, 10 Oct 2017 15:49:48 -0700
[2]: id:87efo222nr@tethera.net
 Subject: Notmuch 0.26 release schedule
 Date: Sun, 10 Dec 2017 08:22:32 -0400
[3]: id:4487e001b350aa8e343a1201d869cceca2a03ab6.1508176853.git.wk...@tremily.us
 Subject: [PATCH] nmbug: Only error for invalid diff lines in tags/
 Date: Mon, 16 Oct 2017 11:01:47 -0700  

   

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy


signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] make release archive: common (or no) timestamps

2017-12-11 Thread Tomi Ollila 
The appended file 'version' has the same timestamp as the files added
by `git archive`.

The original file name and time stamp are no longer saved to the
gzip header in resulting $(PACKAGE)-$(VERSION).tar.gz file.

When build environment is close enough to another, this may
provide mutually reproducible release archive files.
---

tested somewhat like:

$ rm -f test.tar.gz
$ make test.tar.gz TAR_FILE=test.tar.gz
$ md5sum test.tar.gz
59b22212dd006d24d09acd98cd18570a
$ sleep 2
$ rm test.tar.gz
$ make test.tar.gz TAR_FILE=test.tar.gz
$ md5sum test.tar.gz
59b22212dd006d24d09acd98cd18570a
$ rm test.tar.gz

( second try, checksums changed :O .. the reason was git-am'ed content
  *huh* ;)

 Makefile.local | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Makefile.local b/Makefile.local
index 9505b7fee70b..49f83bc3feea 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -31,11 +31,13 @@ $(TAR_FILE):
fi ; \
git archive --format=tar --prefix=$(PACKAGE)-$(VERSION)/ $$ref > 
$(TAR_FILE).tmp
echo $(VERSION) > version.tmp
+   ct=`git --no-pager log -1 --pretty=format:%ct $$ref` ; \
+   touch -d @$$ct version.tmp
tar --owner root --group root --append -f $(TAR_FILE).tmp \
--transform s_^_$(PACKAGE)-$(VERSION)/_  \
--transform 's_.tmp$$__' version.tmp
rm version.tmp
-   gzip < $(TAR_FILE).tmp > $(TAR_FILE)
+   gzip -n < $(TAR_FILE).tmp > $(TAR_FILE)
@echo "Source is ready for release in $(TAR_FILE)"
 
 $(SHA256_FILE): $(TAR_FILE)
-- 
2.13.3

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli/help, completion: added pointers to notmuch-properties(7)

2017-12-11 Thread David Bremner 
Daniel Kahn Gillmor  writes:

> ---
>  completion/notmuch-completion.bash | 2 +-
>  notmuch.c  | 2 ++

pushed to master

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 3/4] nmbug: Auto-checkout in clone if it wouldn't clobber

2017-12-11 Thread David Bremner 
"W. Trevor King"  writes:

> We currently auto-checkout after pull and merge to make those more
> convenient.  They're guarded against data-loss with a leading
> _insist_committed().  This commit adds the same convenience to clone,
> since in most cases users will have no NMBPREFIX-prefixed tags in
> their database when they clone.  Users that *do* have
> NMBPREFIX-prefixed tags will get a warning (and I've bumped the
> default log level to warning so folks who don't set --log-level will
> see it) like:

pushed 1-3 to master. Not sure what happened to patch 4/4?
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch