Re: Release 0.29.

2019-05-26 Thread David Bremner
David Bremner writes: > I know there are several things "in progress", but we've also > accumulated a fair amount of change since 0.28. I am planning a feature > freeze for 0.29 on May 31 and (hopefully) a release on June 7. > > d It's that time again, time to add NEWS items for user facing

Re: revision 3: easing access to the cryptographic envelope

2019-05-26 Thread David Bremner
Daniel Kahn Gillmor writes: > This is the third revision of the series originally posted at > id:20190424183113.29242-1-...@fifthhorseman.net (revision 2 was at > id:20190520032228.27420-1-...@fifthhorseman.net) > > This series addresses comments raised by David Bremner in his review. > Thanks,

start to replace dependence of test suite on gdb with LD_PRELOAD shims

2019-05-26 Thread David Bremner
These patches are somewhere in between WIP and proposed for merging. I'm sure there are some shell scripting tweaks needed, and it's a potentially disruptive enough change that I want to wait until after 0.29. On the other hand, it works for me, and the API seems like an improvement on what it

[PATCH 1/2] test: provide machinery to make and use test_shims

2019-05-26 Thread David Bremner
These can be used e.g. to override return values for functions, in place of the existing scripting of gdb. --- test/test-lib.sh | 16 1 file changed, 16 insertions(+) diff --git a/test/test-lib.sh b/test/test-lib.sh index ff18fae6..a423b7f4 100644 --- a/test/test-lib.sh +++

[PATCH 2/2] test: replace use of gdb with LD_PRELOAD shims in T070-insert.sh

2019-05-26 Thread David Bremner
This removes the dependency of this test script on gdb, and considerably speeds up the running of the tests. --- test/T070-insert.sh | 50 - 1 file changed, 18 insertions(+), 32 deletions(-) diff --git a/test/T070-insert.sh b/test/T070-insert.sh index

[PATCH] NEWS: News for my changes for 0.29

2019-05-26 Thread David Bremner
These are pretty terse overall, and could be expanded in future commits. --- NEWS | 33 + 1 file changed, 33 insertions(+) diff --git a/NEWS b/NEWS index 26b8160c..7a79e560 100644 --- a/NEWS +++ b/NEWS @@ -1,18 +1,51 @@ Notmuch 0.29 (UNRELEASED)

[PATCH] NEWS: The minimum supported major version of Emacs is now 24.

2019-05-26 Thread Tomi Ollila
--- NEWS | 2 ++ 1 file changed, 2 insertions(+) diff --git a/NEWS b/NEWS index 26b8160c8ee2..d8aa272f0093 100644 --- a/NEWS +++ b/NEWS @@ -10,6 +10,8 @@ Command Line Interface Emacs - +The minimum supported major version of Emacs is now 24. + Support for GNU Emacs older than 25.1 is

[PATCH v2] test: report summary even when aborting

2019-05-26 Thread Daniel Kahn Gillmor
In certain cases of test suite failure, the summary report was not being printed. In particular, any failure on the parallel test suite, and any aborted test in the serialized test suite would end up hiding the summary. It's better to always show the summary where we can (while preserving the

Re: [PATCH] test: report summary even when aborting

2019-05-26 Thread Daniel Kahn Gillmor
Thanks for the feedback, Tomi! On Sat 2019-05-25 22:41:58 +0300, Tomi Ollila wrote: > On Sat, May 25 2019, Daniel Kahn Gillmor wrote: > >> In certain cases of test suite failure, the summary report was not >> being printed. In particular, any failure on the parallel test suite, >> and any

Re: [PATCH] NEWS: The minimum supported major version of Emacs is now 24.

2019-05-26 Thread David Bremner
Tomi Ollila writes: > --- > NEWS | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/NEWS b/NEWS > index 26b8160c8ee2..d8aa272f0093 100644 > --- a/NEWS > +++ b/NEWS > @@ -10,6 +10,8 @@ Command Line Interface > Emacs > - > > +The minimum supported major version of Emacs is now

Re: [PATCH] NEWS: News for my changes for 0.29

2019-05-26 Thread David Bremner
David Bremner writes: > These are pretty terse overall, and could be expanded in future > commits. > --- > NEWS | 33 + > 1 file changed, 33 insertions(+) [snip] > + > +Add support for gzip compressed mail messages (/not/ mboxes); > +e.g. `gzip -9 Maildir/cur/*

Re: revision 3: easing access to the cryptographic envelope

2019-05-26 Thread David Bremner
Daniel Kahn Gillmor writes: > On Sun 2019-05-26 09:01:46 -0300, David Bremner wrote: >> Daniel Kahn Gillmor writes: >> >>> This is the third revision of the series originally posted at >>> id:20190424183113.29242-1-...@fifthhorseman.net (revision 2 was at >>>

Re: revision 3: easing access to the cryptographic envelope

2019-05-26 Thread Daniel Kahn Gillmor
On Sun 2019-05-26 09:01:46 -0300, David Bremner wrote: > Daniel Kahn Gillmor writes: > >> This is the third revision of the series originally posted at >> id:20190424183113.29242-1-...@fifthhorseman.net (revision 2 was at >> id:20190520032228.27420-1-...@fifthhorseman.net) >> >> This series

[PATCH v2 08/17] test: show cryptographic envelope information for signed mails

2019-05-26 Thread Daniel Kahn Gillmor
Make sure that we emit the correct cryptographic envelope status for cleartext signed messages. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh| 11 ++- .../signed-protected-header.eml | 29 +++

[PATCH v2 07/17] test: add test for missing external subject

2019-05-26 Thread Daniel Kahn Gillmor
Adding another test to ensure that we handle protected headers gracefully when no external subject is present. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh| 6 .../subjectless-protected-header.eml | 29 +++ 2 files changed,

[PATCH v2 02/17] util/crypto: add information about the payload part

2019-05-26 Thread Daniel Kahn Gillmor
When walking the MIME tree, if we discover that we are at the cryptographic payload, then we would like to record at least the Subject header of the current MIME part. In the future, we might want to record many other headers as well, but for now we will stick with just the Subject. See

[PATCH v2 01/17] cli/show: emit headers after emitting body

2019-05-26 Thread Daniel Kahn Gillmor
This paves the way for emitting protected headers after verification and decryption, because it means that the headers will only be emitted after the body has been parsed. Signed-off-by: Daniel Kahn Gillmor --- notmuch-show.c| 6 +++--- test/T170-sexp.sh | 10 +- 2 files changed, 8

[PATCH v2 12/17] test: after reindexing, only legitimate protected subjects are searchable

2019-05-26 Thread Daniel Kahn Gillmor
This test scans for all the possible protected headers (including bogus/broken ones) that are present in the protected-headers corpus, trying to make sure that only the ones that are not broken or malformed show up in a search after re-indexing. Signed-off-by: Daniel Kahn Gillmor ---

[PATCH v2 05/17] cli/show: emit payload subject instead of outside subject

2019-05-26 Thread Daniel Kahn Gillmor
Correctly fix the two outstanding tests so that the protected (hidden) subject is properly reported. Signed-off-by: Daniel Kahn Gillmor --- notmuch-client.h | 2 +- notmuch-reply.c| 4 +++- notmuch-show.c | 14 +-

[PATCH v2 15/17] test: emacs/show: ensure that protected headers appear as expected

2019-05-26 Thread Daniel Kahn Gillmor
This tests notmuch-show; headers appear appropriately based on the setting of notmuch-crypto-process-mime. Signed-off-by: Daniel Kahn Gillmor --- test/T358-emacs-protected-headers.sh | 36 +++- 1 file changed, 35 insertions(+), 1 deletion(-) diff --git

[PATCH v2 03/17] test: new test framework to compare json parts

2019-05-26 Thread Daniel Kahn Gillmor
From: Jameson Graef Rollins This makes it easier to write fairly compact, readable tests of json output, without needing to sanitize away parts that we don't care about. Signed-off-by: Daniel Kahn Gillmor --- test/json_check_nodes.py | 113 +++

Protected Headers (2nd major revision, more testing!)

2019-05-26 Thread Daniel Kahn Gillmor
Hi all-- Way back in id:20180511055544.13676-1-...@fifthhorseman.net, i proposed support for protected headers (in particular, for being able to read and search for subject lines of encrypted messages which protect the Subject). Although that series was reviewed by Bremner, i never managed to

[PATCH v2 11/17] test: protected headers should work when both encrypted and signed.

2019-05-26 Thread Daniel Kahn Gillmor
Up to this point, we've tested protected headers on messages that have either been encrypted or signed, but not both. This adds a couple tests of signed+encrypted messages, one where the subject line is masked (outside subject line is "Subject Unavailable") and another where it is not (outside

[PATCH v2 14/17] test: ensure that protected headers appear in notmuch-emacs search as expected

2019-05-26 Thread Daniel Kahn Gillmor
We initially test only notmuch-search; tests for other functionality come in different patchsets later. Signed-off-by: Daniel Kahn Gillmor --- test/T358-emacs-protected-headers.sh | 36 1 file changed, 36 insertions(+) create mode 100755

[PATCH v2 10/17] indexing: record protected subject when indexing cleartext

2019-05-26 Thread Daniel Kahn Gillmor
When indexing the cleartext of an encrypted message, record any protected subject in the database, which should make it findable and visible in search. Signed-off-by: Daniel Kahn Gillmor --- lib/index.cc | 42 ++ lib/message.cc |

[PATCH v2 06/17] cli/show: add information about which headers were protected

2019-05-26 Thread Daniel Kahn Gillmor
The header-mask member of the per-message crypto object allows a clever UI frontend to mark whether a header was protected (or not). And if it was protected, it contains enough information to show useful detail to an interested user. For example, an MUA could offer a "show what this message's

[PATCH v2 16/17] test: reply (in cli and emacs) should protect indexed sensitive headers

2019-05-26 Thread Daniel Kahn Gillmor
These tests are currently broken! When a protected subject is indexed in the clear, it leaks in the reply headers :( For emacs, we set up separate tests for when the protected header is indexed in the clear and when it is unindexed. neither case should leak, but the former wasn't tested yet.

[PATCH v2 09/17] cli/reply: ensure encrypted Subject: line does not leak in the clear

2019-05-26 Thread Daniel Kahn Gillmor
Now that we can decrypt headers, we want to make sure that clients using "notmuch reply" to prepare a reply don't leak cleartext in their subject lines. In particular, the ["reply-headers"]["Subject"] should by default show the external Subject. A replying MUA that intends to protect the Subject

[PATCH v2 13/17] test: try indexing nested messages and protected headers

2019-05-26 Thread Daniel Kahn Gillmor
We want to make sure that internally-forwarded messages don't end up "bubbling up" when they aren't actually the cryptographic payload. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh| 6 ...pted-message-with-forwarded-attachment.eml | 33

[PATCH v2 04/17] cli/show: add tests for viewing protected headers

2019-05-26 Thread Daniel Kahn Gillmor
Here we add several variant e-mail messages, some of which have correctly-structured protected headers, and some of which do not. The goal of the tests is to ensure that the right protected subjects get reported. Signed-off-by: Daniel Kahn Gillmor --- test/T356-protected-headers.sh

[PATCH v2 17/17] cli/reply: pull proposed subject line from the message, not the index

2019-05-26 Thread Daniel Kahn Gillmor
Protected subject lines were being emitted in reply when the cleartext of documents was indexed. create_reply_message() was pulling the subject line from the index, rather than pulling it from the GMimeMessage object that it already has on hand. This one-line fix to notmuch-reply.c solves that

Re: [PATCH v2] test: report summary even when aborting

2019-05-26 Thread Tomi Ollila
On Sun, May 26 2019, Daniel Kahn Gillmor wrote: > In certain cases of test suite failure, the summary report was not > being printed. In particular, any failure on the parallel test suite, > and any aborted test in the serialized test suite would end up hiding > the summary. > > It's better to

Re: [PATCH v2] test: report summary even when aborting

2019-05-26 Thread David Bremner
Daniel Kahn Gillmor writes: > In certain cases of test suite failure, the summary report was not > being printed. In particular, any failure on the parallel test suite, > and any aborted test in the serialized test suite would end up hiding > the summary. > pushed to master, d

Re: revision 3: easing access to the cryptographic envelope

2019-05-26 Thread Daniel Kahn Gillmor
On Sun 2019-05-26 13:54:29 -0300, David Bremner wrote: > % uncrustify -c devel/uncrustify.cfg --replace $files thanks for this pointer, i will experiment with it. > If we do decide to rip off the bandage, that will cause a certain amount > of rebasing pain for any patch series in flight; now