Re: [PATCH] emacs: Add notmuch-show-local-dates option

2020-07-05 Thread Daniel Kahn Gillmor
On Fri 2020-07-03 19:58:23 -0300, David Bremner wrote:
> The bikeshed must be blue! Uh, I mean what about narrowish screens (80
> columns or so) and or deeply indented threads?

You mean what should happen to messages with headers that are much
longer, like:

To: David Bremner , Kevin Foley , 
notmuch@notmuchmail.org

?  

If someone wants to propose an alternate compact representation that
includes both the local time and the sender's TZ offset from where the
reader is, but is shorter than what i'd proposed, i'm all ears…

I originally tried the more compact:

Date: Sat, 04 Jul 2020 09:25:26 -0400 (Sender's TZ +0200)

But that isn't quite as useful as knowing what time the sender thought
it was when they hit "send" (i don't want to have to do the math in my
head).  I also tried just showing the time (not the date) but then that
gets confusing around the international date line.

Time is hard, who knew‽

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs: Add notmuch-show-local-dates option

2020-07-03 Thread Daniel Kahn Gillmor
On Thu 2020-07-02 18:02:34 -0400, Kevin Foley wrote:
> Daniel Kahn Gillmor  writes:
>> and it could take three values:
>>
>>  - nil (default), shows the Date: header as received
>>  - t, shows the timestamp from the Date: header in local time, 
>>with the as-received header in parens afterward (see below)
>>  - "only", shows only the timestamp in localtime
>>
>
> I feel like "only" makes more sense as the option to be used for t, and
> having "both" as another option.

I'm fine either way.

>> so if your system is TZ=UTC, and notmuch-show-date-header-localtime is
>> set to t, and you're looking at a message sent from TZ=America/New_York,
>> you might see:
>>
>> Date: Thu, 02 Jul 2020 19:34:53 + (Thu, 02 Jul 2020 15:34:53 -0400)
>
> Actually, seeing it written out here makes me realize some people could
> potentially prefer:
>
> Date: {sent-tz-datetime} ({system-tz-datetime})
>
> or some other kind of formatting.
>
> Would it make sense to allow a function instead of "both", which would
> be passed the time and let the user return it formatted how they would
> like?  Or is that over-complicating things?

For a toolkit, i like the idea of a function.  For an end-user-facing
MUA, i like opinionated decisions that do obviously the right thing,
without requiring the user to fiddle with anything.

We're struggling a bit here because notmuch-emacs is sort of in the
middle of these two things -- sometimes the one, other times the other.

Pushing on the "just do the right thing" front:

What if there were no configuration variable at all, and it just always
shows "both" ?  Or, even cleverer, what if it only shows both if the
current TZ differs from the sender's TZ?

So if i'm in TZ=America/New_York, and the sender is in
TZ=America/New_York, i would just see the normal header:

Date: Fri, 03 Jul 2020 13:22:36 -0400

But if the sender is in TZ=Europe/Berlin, i would see:

Date: Fri, 03 Jul 2020 13:22:36 -0400 [Fri, 03 Jul 2020 19:22:36 +0200]

(Note that RFC 5322 Date format shows the hour offset, but not the
actual TZ -- i can't tell from -0400 whether someone is in
TZ=America/New_York or TZ=America/Manaus)

Is there anyone who would complain about this just being the default
behavior -- with no additional settings to change?

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] completion: remove "setup" from the list of possible completions

2020-07-02 Thread Daniel Kahn Gillmor
On Wed 2020-06-24 21:44:01 +1000, Peter Wang wrote:
> On Mon, 22 Jun 2020 12:22:50 +0200 Lukasz Stelmach  
> wrote:
>> It was <2020-06-20 sob 12:53>, when Reto wrote:
>> > On Fri, Jun 19, 2020 at 12:40:49PM +0200, Łukasz Stelmach wrote:
>> >> Having "setup" in the set requires entering three instad of two characters
>> >> for "search". Since "setup" is rearly used it makes little sense to have
>> >> it in the set and cripple UX for much more frequently used "search".
>> >
>> > I very much disagree with this patch.
>> > The completions should contain all possible values, saving a single 
>> > keystroke is
>> > certainly not a valid reason to remove a valid option from the completions.
>> >
>> > Write an alias into your bashrc if that bothers you so much... Then you 
>> > can save
>> > much more keystrokes.
>> 
>> I already have several aliases covering most of my use cases, however, I
>> still use "notmuch search" from time to time and I came to a conclusion
>> expressed in this patch. Of course, as a random user, I can only suggest
>> and by no means insist on applying it.
>
> Another possibility may be to rename "notmuch setup" to "notmuch init",
> treating "setup" as a deprecated synonym for "init". The completions
> would include "init" but not "setup".

I sympathize with everyone struggling with the first-world problems in
this thread. :P

If i had to choose between the status quo and Lukasz's suggestion of not
completing "notmuch setup", i'd choose the status quo.

I value having all non-deprecated subcommands show up in tab completion.
This is particularly important for someone who is just starting to use
notmuch, and may use tab completion for discoverability.  If they can't
find the very first expected subcommand to be used in tab completion
exploration, that is pretty weird.

That said, i appreciate Peter's clever attempt to thread the needle.

Unfortunately, changing "setup" to "init" moves "notmuch insert" from
"notmuch i" to "notmuch in", so you're sort of robbing from
Peter to pay Paul.  And I'm having difficulty coming up with another
good subcommand name with an unambiguous prefix to move "setup" to.

I also note that we have no independent manpage for "notmuch-setup",
it's just symlinked from notmuch.1.gz.

Another "clever" approach to assuage the tab-completion-for-conveience
advocates would be to introduce a (non-deprecated) alias for "search"
that itself would be fewer keystrokes before tab completion (e.g. "srch"
is two keystrokes because "sr" is unambiguous, "query" is just one,
because "q" is unambiguous).

Overall, i value consistency and completeness and i would not like to
see the tab completion be either an inconsistent or incomplete
representation of the options available to the user from the command
line.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs: Add notmuch-show-local-dates option

2020-07-02 Thread Daniel Kahn Gillmor
This proposed notmuch-show-local-dates feature is a nice one.  It
renders the Date header in a format that is likely to be more useful to
the viewer.  I certainly find this more useful than having to do the
TZ conversions in my head.

But, setting this to t hides the sender's TZ from the viewer -- and i
often find it useful to learn the sender's TZ from the Date: header.

What would be really useful for me is to see the Date header represented
both ways: in my local time *and* the Date header that the sender sent.
But a boolean setting doesn't give me much room to express that
preference.

On Thu 2020-06-25 09:30:16 -0400, Kevin J. Foley wrote:
> Allows users to specify they'd like dates to be displayed in local time

This comment says "time", but the emacs config variable name says
"dates".  This confusion is worse because the Date: header actually
contains a timestamp, not a date ☹

Perhaps we should rename the variable notmuch-show-date-header-localtime,
and it could take three values:

 - nil (default), shows the Date: header as received
 - t, shows the timestamp from the Date: header in local time, 
   with the as-received header in parens afterward (see below)
 - "only", shows only the timestamp in localtime

so if your system is TZ=UTC, and notmuch-show-date-header-localtime is
set to t, and you're looking at a message sent from TZ=America/New_York,
you might see:

Date: Thu, 02 Jul 2020 19:34:53 + (Thu, 02 Jul 2020 15:34:53 -0400)

Would that be too much configurability?

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: introduce exception handling at top level of libnotmuch

2020-07-02 Thread Daniel Kahn Gillmor
Hi Bremner, all--

On Mon 2020-06-29 22:14:07 -0300, David Bremner wrote:
> I know that some of you are not C++ fans, but at the moment this is
> the cleanest fix I can think of to uncaught xapian exceptions causing
> calls to the library to die. Floris reminded me of this recently with
> the discussion about operations on closed databases, but cleaning up
> the handling of exceptions in libnotmuch has been on my mind for a
> while. It will be bit laborious so I did a few functions for
> discussion purposes before getting too carried away.

I've read through the series and it looks reasonable to me.

I've also tested them, and they behave as expected.

If someone has a more nuanced approach to dealing with some of the
subtle exceptions that might be raised, i'd be happy to see those
approaches laid on top of this series.

> There is still a certain amount of boilerplate with more or less
> identical try/catch blocks (yes, I really miss scheme macros here). I
> could mostly eliminate that with C++11 lambdas, but I wasn't sure the
> result was more maintainable or nicer.

I think this looks fine, and it isn't a huge amount of boilerplate.

Please merge.

  --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Newbie: On Outlook Exchange how to use notmuch to connect to it?

2020-07-02 Thread Daniel Kahn Gillmor
On Thu 2020-07-02 10:36:34 +0300, Tomer B wrote:
> Hi,  I tried searching but the mailing list is huge..

You might try searching at the new mailing list interface:
https://nmbug.notmuchmail.org/nmweb/ -- put your search terms in the
"terms" box!

> I'm a newbie and impressed by notmuch.  I want to start using it, i'm
> currently on outlook exchange, is there any guide that shows how to use
> notmuch for searching my outlook both exchange and local folders,

Outlook is a Mail User Agent (a local e-mail program) or a webapp. It
typically accesses Microsoft Exchange (a mail server) using one of several
different possible mechanisms (MAPI, EWS, etc) to fetch representations
of each e-mail message.

Notmuch only works with a local maildir.  So what you need is a way to
get a copy of your messages from Exchange and build a local maildir.

I'm afraid i don't have a good answer for that -- it will typically
depend on how Exchange is configured.  If Exchange has an IMAP connector
enabled, you can use a tool like imap-dl (from the mailscripts package),
getmail, or offlineimap to retrieve the messages via IMAP and stash them
in a local maildir.

If Exchange doesn't have an IMAP connector enabled, then you'll need to
learn more about what interfaces are available for you to fetch the
underlying messages.

Without more information about how your mail server is configured, it's
hard to know how to give you better guidance here, though.

 --dkg



signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] doc: update name of gmime package to install

2020-07-02 Thread Daniel Kahn Gillmor
On Thu 2020-07-02 10:01:07 -0300, David Bremner wrote:
> Tomi Ollila  writes:
>
>> On Fri, Jun 26 2020, David Bremner wrote:
>>
>>> Fedora still has an old gmime-devel which is 2.6.x. This is no longer
>>> supported by notmuch.
>>> ---
>>>  INSTALL   | 2 +-
>>>  configure | 2 +-
>>>  2 files changed, 2 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/INSTALL b/INSTALL
>>> index f1236e71..3f11c2b7 100644
>>> --- a/INSTALL
>>> +++ b/INSTALL
>>> @@ -95,7 +95,7 @@ dependencies with a single simple command line. For 
>>> example:
>>>  
>>>For Fedora and similar:
>>>  
>>> -   sudo yum install xapian-core-devel gmime-devel libtalloc-devel 
>>> zlib-devel python3-sphinx texinfo info
>>> +   sudo yum install xapian-core-devel gmime30-devel libtalloc-devel 
>>> zlib-devel python3-sphinx texinfo info
>>
>> Since 'Fedora', change 'yum' to 'dnf' (no yum nomore on no fedora system)
>
> Do you mean dnf is better or newer than yum? FWIW I still have yum in my
> fedora32 (cloud) image.

I've been using dnf on fedora 32.  I concur that we need gmime30-devel,
not gmime-devel.

My understanding is that dnf is the future:

   https://www.2daygeek.com/comparison-difference-between-dnf-vs-yum/

Might as well s/yum/dnf/ while you're s/gmime-devel/gmime30-devel/ too.

LGTM,

  --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 1/2] configure: can gpgme can verify signatures when decrypting with a session key?

2020-07-02 Thread Daniel Kahn Gillmor
If https://dev.gnupg.org/T3464 is unresolved in the version of gpgme
we are testing against, then we should know about it, because it
affects the behavior of notmuch.

Signed-off-by: Daniel Kahn Gillmor 
---
 configure | 78 ++-
 1 file changed, 77 insertions(+), 1 deletion(-)

diff --git a/configure b/configure
index 2e01034b..2caa08c8 100755
--- a/configure
+++ b/configure
@@ -620,6 +620,78 @@ EOF
 if [ -n "$TEMP_GPG" -a -d "$TEMP_GPG" ]; then
 rm -rf "$TEMP_GPG"
 fi
+
+# see https://dev.gnupg.org/T3464
+# there are problems verifying signatures when decrypting with session 
keys with GPGME 1.13.0 and 1.13.1
+printf "Checking signature verification when decrypting using session 
keys... "
+
+cat > _verify_sig_with_session_key.c <
+#include 
+
+int main () {
+GError *error = NULL;
+GMimeParser *parser = NULL;
+GMimeMultipartEncrypted *body = NULL;
+GMimeDecryptResult *result = NULL;
+GMimeSignatureList *sig_list = NULL;
+GMimeSignature *sig = NULL;
+GMimeObject *output = NULL;
+GMimeSignatureStatus status;
+int len;
+
+g_mime_init ();
+parser = g_mime_parser_new ();
+g_mime_parser_init_with_stream (parser, 
g_mime_stream_file_open("$srcdir/test/corpora/crypto/encrypted-signed.eml", 
"r", ));
+if (error) return !! fprintf (stderr, "failed to instantiate parser with 
test/corpora/pkcs7/smime-onepart-signed.eml\n");
+
+body = GMIME_MULTIPART_ENCRYPTED(g_mime_message_get_mime_part 
(g_mime_parser_construct_message (parser, NULL)));
+if (body == NULL) return !!fprintf (stderr, "did not find a 
multipart/encrypted message\n");
+
+output = g_mime_multipart_encrypted_decrypt (body, GMIME_DECRYPT_NONE, 
"9:13607E4217515A70EC8DF9DBC16C5327B94577561D98AD1246FA8756659C7899", , 
);
+if (error || output == NULL) return !! fprintf (stderr, "decrypt 
failed\n");
+
+sig_list = g_mime_decrypt_result_get_signatures (result);
+if (sig_list == NULL) return !! fprintf (stderr, "sig_list is NULL\n");
+
+if (sig_list == NULL) return !! fprintf (stderr, "no GMimeSignatureList 
found\n");
+len = g_mime_signature_list_length (sig_list);
+if (len != 1) return !! fprintf (stderr, "expected 1 signature, got %d\n", 
len);
+sig = g_mime_signature_list_get_signature (sig_list, 0);
+if (sig == NULL) return !! fprintf (stderr, "no GMimeSignature found at 
position 0\n");
+status = g_mime_signature_get_status (sig);
+if (status & GMIME_SIGNATURE_STATUS_KEY_MISSING) return !! fprintf 
(stderr, "signature status contains KEY_MISSING (see 
https://dev.gnupg.org/T3464)\n");
+
+return 0;
+}
+EOF
+if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XX"); then
+printf 'No.\nCould not make tempdir for testing signature verification 
when decrypting with session keys.\n'
+errors=$((errors + 1))
+elif ${CC} ${CFLAGS} ${gmime_cflags} _verify_sig_with_session_key.c 
${gmime_ldflags} -o _verify_sig_with_session_key \
+&& GNUPGHOME=${TEMP_GPG} gpg --batch --quiet --import < 
"$srcdir"/test/gnupg-secret-key.asc \
+&& rm -f ${TEMP_GPG}/private-keys-v1.d/*.key
+then
+if GNUPGHOME=${TEMP_GPG} ./_verify_sig_with_session_key; then
+gmime_verify_with_session_key=1
+printf "Yes.\n"
+else
+gmime_verify_with_session_key=0
+printf "No.\n"
+cat <https://dev.gnupg.org/T3464 for more details.
+EOF
+fi
+else
+printf 'No.\nFailed to set up gpg for testing signature verification 
while decrypting with a session key.\n'
+errors=$((errors + 1))
+fi
+if [ -n "$TEMP_GPG" -a -d "$TEMP_GPG" ]; then
+rm -rf "$TEMP_GPG"
+fi
 else
 have_gmime=0
 printf "No.\n"
@@ -1144,7 +1216,8 @@ for flag in -Wmissing-declarations; do
 done
 printf "\n\t%s\n" "${WARN_CFLAGS}"
 
-rm -f minimal minimal.c _time_t.c _libversion.c _libversion _libversion.sh 
_check_session_keys.c _check_session_keys _check_x509_validity.c 
_check_x509_validity
+rm -f minimal minimal.c _time_t.c _libversion.c _libversion _libversion.sh 
_check_session_keys.c _check_session_keys _check_x509_validity.c 
_check_x509_validity \
+   _verify_sig_with_session_key.c _verify_sig_with_session_key
 
 # construct the Makefile.config
 cat > Makefile.config <

[PATCH 2/2] tests: mark sig verification known-broken with session keys on buggy gpgme

2020-07-02 Thread Daniel Kahn Gillmor
We make use of the just-introduced configure test.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T357-index-decryption.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh
index 1ac2836a..1ed5f28c 100755
--- a/test/T357-index-decryption.sh
+++ b/test/T357-index-decryption.sh
@@ -306,6 +306,9 @@ test_json_nodes <<<"$output" "$goodsig"
 
 test_begin_subtest "verify signature with stashed session key"
 output=$(notmuch show --format=json id:encrypted-sig...@crypto.notmuchmail.org)
+if [ $NOTMUCH_GMIME_VERIFY_WITH_SESSION_KEY -ne 1 ]; then
+test_subtest_known_broken
+fi
 test_json_nodes <<<"$output" "$goodsig"
 
 # TODO: test removal of a message from the message store between
-- 
2.27.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: crypto test failures on Fedora and OpenSUSE

2020-07-01 Thread Daniel Kahn Gillmor
Hi folks--

On Sun 2020-06-28 08:33:42 -0300, David Bremner wrote:
> I dug a bit further down, and this is what is returned from gpgme
> (line 345 in g_mime_gpgme_get_signatures)
>
> sig = {next = 0x0,
> summary = GPGME_SIGSUM_KEY_MISSING, 
> fpr = 0x4ac480 "5AEAB11F5E33DCE875DDB75B6D92612D94E46381", status = 9, 
> notations = 0x0, timestamp = 1559167762, exp_timestamp = 0, 
> wrong_key_usage = 0, 
> pka_trust = 0, chain_model = 0, is_de_vs = 0, _unused = 0, 
> validity = GPGME_VALIDITY_UNKNOWN, validity_reason = 0, 
> pubkey_algo = GPGME_PK_RSA, hash_algo = GPGME_MD_SHA256, pka_address = 
> 0x0, 
> key = 0x0}
>
> At this point I'm leaning towards declaring it a gpgme problem in
> fedora32, and suggesting that relevant distros mark the test broken. I
> am of course open to more informed opinions.

The problem does indeed appear to be with gpgme, in versions 1.13.0 and
1.13.1.

In particular, it is a problem with the resolution of
https://dev.gnupg.org/T3464, which is ultimately fixed upstream, but is
not yet fixed in a released version of gpgme.

The upstream commit, which should be patched into gpgme on Fedora and
OpenSUSE (and anywhere else that depends on gpgme) is:

https://dev.gnupg.org/rMae4d7761a15b82eb98b0bcc72af2ae2e8973e1f9

(patch attached here as well)

We don't see this on Debian because gpgme in debian has carried this
patch for over a year now.

In gpgme 1.12.0 and earlier, this bug did not exist.  But gpgme 1.13.0
introduced the bug in an attempt to avoid error diagnostics when *not*
trying to verify a signature while using a session key.  The fix in
1.13.0 inadvertently introduced an error when the caller does actually
try to verify a signature, which is what we see here.

  --dkg

From ae4d7761a15b82eb98b0bcc72af2ae2e8973e1f9 Mon Sep 17 00:00:00 2001
From: Daniel Kahn Gillmor 
Date: Wed, 29 May 2019 17:56:01 -0400
Subject: [GPGME PATCH] gpg: Avoid error diagnostics with
 --override-session-key when verifying

* src/engine-gpg.c (gpg_decrypt): only send --no-keyring when we are
not verifying.

--

Without this change, the signature verification would fail.  This
problem was introduced in bded8ebc59c7fdad2617f4c9232a58047656834c in
an attempt to avoid an error when *not* verifying.  Clearly more test
suite coverage is needed to avoid introducing this sort of problem in
the future.

GnuPG-bug-id: 3464
Signed-off-by: Daniel Kahn Gillmor 
---
 src/engine-gpg.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/src/engine-gpg.c b/src/engine-gpg.c
index 5c335cb2..223404ed 100644
--- a/src/engine-gpg.c
+++ b/src/engine-gpg.c
@@ -1717,12 +1717,15 @@ gpg_decrypt (void *engine,
  strlen (override_session_key), 1);
   if (!err)
 {
-  /* We add --no-keyring because a keyring is not required
-   * when we are overriding the session key.  It would
+  /* When we are not trying to verify signatures as well,
+   * we add --no-keyring because a keyring is not required
+   * for decryption when overriding the session key.  It would
* work without that option but --no-keyring avoids that
* gpg return a failure due to a missing key log_error()
* diagnostic.  --no-keyring is supported since 2.1.14. */
-  err = add_arg (gpg, "--no-keyring");
+
+  if (!(flags & GPGME_DECRYPT_VERIFY))
+  err = add_arg (gpg, "--no-keyring");
   if (!err)
 err = add_arg (gpg, "--override-session-key-fd");
   if (!err)
-- 
2.27.0



signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


notmuch mailing list administrivia

2020-07-01 Thread Daniel Kahn Gillmor
Hi folks--

David Bremner, Carl Worth, and i did some maintenance work today.  We
hope that what we did was to successfully move the notmuch mailing list
to a mailman3 instance.

Please let us know if you have any problems with it (such as not
receiving this message, ha ha).

(feel free to send mail privately to me and bremner, we don't need a
huge reply-all on-list thread)

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


difficulties with notmuch2 python bindings for alot

2020-06-09 Thread Daniel Kahn Gillmor
Hi all--

I see over on github that alot is trying to port to the notmuch2
bindings, and having a few problems with it:

 https://github.com/pazz/alot/pull/1511

alot is an important consumer of the notmuch python bindings, and it
would be really great to see them successfully transition to the
notmuch2 module.

Floris, if you (or anyone else with this particular knowledge) has a
chance to take a look and help them sort out the remaining issues, that
would be much appreciated!

  --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] add NEWS for 0.30

2020-06-01 Thread Daniel Kahn Gillmor
Signed-off-by: Daniel Kahn Gillmor 
---
 NEWS | 35 +++
 1 file changed, 35 insertions(+)

diff --git a/NEWS b/NEWS
index 14db7f6f..1921127f 100644
--- a/NEWS
+++ b/NEWS
@@ -1,6 +1,41 @@
 Notmuch 0.30 (2020-06-01)
 =
 
+S/MIME
+--
+
+Handle S/MIME (PKCS#7) messages -- one-part signed messages, encrypted
+messages, and multilayer messages.  Treat them symmetrically to
+OpenPGP messages.  This includes handling protected headers
+gracefully.
+
+If you're using Notmuch with S/MIME, you currently need to configure
+gpgsm appropriately.
+
+Mixed-up MIME Repair
+
+
+Detect and automatically repair a common form of message mangling
+created by Microsoft Exchange. (see index.repaired=mixedup in
+notmuch-properties(7).
+
+Protected Headers
+-
+
+Avoid indexing the legacy-display part of an encrypted message that
+has protected headers. (see
+index.repaired=skip-protected-headers-legacy-display in
+notmuch-properties(7).
+
+Python
+--
+
+Drop compatibility with python2, focus on python3.
+
+Introduce new CFFI-based python bindings in the python module named
+"notmuch2".  Officially deprecate (but still support) the older
+"notmuch" module.
+
 Notmuch 0.29.3 (2019-11-27)
 ===
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] tests/ruby: Ensure that test works for out-of-tree builds

2020-05-26 Thread Daniel Kahn Gillmor
On Tue 2020-05-26 13:06:02 -0400, Daniel Kahn Gillmor wrote:
> ---
>  test/test-lib.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 792b1cb9..78a90862 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -1081,7 +1081,7 @@ test_python() {
>  }
>  
>  test_ruby() {
> -MAIL_DIR=$MAIL_DIR $NOTMUCH_RUBY -I $NOTMUCH_SRCDIR/bindings/ruby> OUTPUT
> +MAIL_DIR=$MAIL_DIR $NOTMUCH_RUBY -I "$NOTMUCH_BUILDDIR/bindings/ruby"> 
> OUTPUT
>  }
>  
>  test_C () {

I should note that i'm not a ruby developer, but this seems to be the
correct patch to solve the problem, similar to what Floris proposed to
fix the python-cffi test suite.

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: python-cffi and ruby test suites fail in out-of-tree builds

2020-05-26 Thread Daniel Kahn Gillmor
On Sat 2020-05-23 21:52:18 +0200, Floris Bruynooghe wrote:
> modified   test/T391-python-cffi.sh
> @@ -8,7 +8,7 @@ fi
>  
>  
>  test_begin_subtest "python cffi tests"
> -pytest_dir=$NOTMUCH_SRCDIR/bindings/python-cffi/build/stage
> +pytest_dir=$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage
>  printf "[pytest]\nminversion = 3.0\naddopts = -ra\n" > $pytest_dir/pytest.ini
>  test_expect_success "(cd $pytest_dir && ${NOTMUCH_PYTHON} -m pytest 
> --log-file=$TMP_DIRECTORY/test.output)"
>  test_done

I've tested this and i can confirm that it works for both out-of-tree
and in-tree builds.  LGTM.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] tests/ruby: Ensure that test works for out-of-tree builds

2020-05-26 Thread Daniel Kahn Gillmor
---
 test/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 792b1cb9..78a90862 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -1081,7 +1081,7 @@ test_python() {
 }
 
 test_ruby() {
-MAIL_DIR=$MAIL_DIR $NOTMUCH_RUBY -I $NOTMUCH_SRCDIR/bindings/ruby> OUTPUT
+MAIL_DIR=$MAIL_DIR $NOTMUCH_RUBY -I "$NOTMUCH_BUILDDIR/bindings/ruby"> 
OUTPUT
 }
 
 test_C () {
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


doc build warnings when building out-of-tree

2020-05-26 Thread Daniel Kahn Gillmor
When building out-of-tree, the documentation (both manpages and info
files) are incomplete, but they do not explicitly fail.  build logs
follow from doing "mkdir build && ../configure && make".

If there's a way to make these warnings into hard failures, i think that
would be good -- we don't want to accidentally ship the output they
create.  And of course actually fixing it would be even better.

I'm afraid i'm not proficient enough in sphinx to know how to do either,
so i'm just reporting a bug for now.

--dkg

WITH_EMACS=1 sphinx-build -b man -d doc/_build/man_doctrees -q ../doc 
doc/_build/man
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:383: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:385: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-lib.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:387: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-show.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:389: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-tag.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:220: WARNING: Undefined 
substitution referenced: "docstring::notmuch-message-headers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:223: WARNING: Undefined 
substitution referenced: "docstring::notmuch-message-headers-visible".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:236: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-filename".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:239: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-git-send-email".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:242: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-message-id-stripped".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:245: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-mlarchive-link-and-go".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:248: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-tags".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:251: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-cc".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:254: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-date".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:257: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-from".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:260: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-message-id".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:263: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-mlarchive-link".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:266: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-subject".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:269: WARNING: Undefined 
substitution referenced: "docstring::notmuch-show-stash-to".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:347: WARNING: Undefined 
substitution referenced: "docstring::notmuch-tagging-keys".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:353: WARNING: Undefined 
substitution referenced: "docstring::notmuch-cycle-notmuch-buffers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:364: WARNING: Undefined 
substitution referenced: "docstring::notmuch-poll".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:367: WARNING: Undefined 
substitution referenced: "docstring::notmuch-poll-script".
[…]


WITH_EMACS=1 sphinx-build -b texinfo -d doc/_build/texinfo_doctrees -q ../doc 
doc/_build/texinfo
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:383: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:385: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-lib.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:387: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-show.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:389: WARNING: Problems with 
"include" directive path:
InputError: [Errno 2] No such file or directory: '../emacs/notmuch-tag.rsti'.
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:220: WARNING: Undefined 
substitution referenced: "docstring::notmuch-message-headers".
/home/dkg/src/notmuch/notmuch/doc/notmuch-emacs.rst:223: WARNING: Undefined 
substitution 

Re: [PATCH] tests: fix test_json_nodes() in out-of-tree builds

2020-05-26 Thread Daniel Kahn Gillmor
On Sun 2020-05-24 20:59:29 +0300, Tomi Ollila wrote:
> On Thu, May 21 2020, Daniel Kahn Gillmor wrote:
>
>> In out-of-tree builds, $TEST_DIRECTORY doesn't contain
>> json_check_nodes.py.  This caused 27 tests to fail in such an
>> environment.
>
> are you shure the run_emacs.sh change in 
>
> id:"20200423212643.864-1-tomi.oll...@iki.fi"
>
> is not needed ? =D

Sorry, i'd missed that message entirely.  Looks like we were barking up
the same tree :)

I do not mean to cast doubt on the run_emacs.sh change -- i haven't seen
the particular failure with two different versions of emacs, so i don't
know that i can comment on it.

At any rate, these seem like two distinct fixes.  I'd be happy to see
them both applied, so maybe just drop my patch and merge Tomi's?

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] python-cffi: enable out-of-tree builds

2020-05-26 Thread Daniel Kahn Gillmor
On Thu 2020-05-21 21:03:59 -0400, Daniel Kahn Gillmor wrote:
> This is a simple hack to enable out-of-tree builds, a concern raised
> by Tomi in id:m24kzjib9a@guru.guru-group.fi
>
> This change at least enables "make check" to complete without error,
> but I'm sure it could be improved.  I am not expert enough in
> setuptools to know how.

in id:87d06usa31@powell.devork.be, Floris said:

>> It probably is indeed the unfortunate case that copying the python
>> source is currently the easiest.  I had a quick look and it seemed like
>> one'd have to dig into the cffi setuptools support to make this work and
>> I'm not sure how successful that would be, but I admit I didn't feel
>> like trying.

So i've removed notmuch::needs-review -- sounds like this is currently
the way to go.

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs: split-window-sensibly in tree mode with open message

2020-05-23 Thread Daniel Kahn Gillmor
On Fri 2020-05-22 19:58:19 +0100, Mark Walters wrote:
> As far as I can see the split-window-sensibly version splits the
> window into two equal parts, whereas the split-window-vertically
> version (deliberately) makes the message pane bigger than the thread
> pane.

I find this argument compelling.  When i've used split-pane MUAs, i
pretty much always want the tree view smaller than the message pane.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


python-cffi and ruby test suites fail in out-of-tree builds

2020-05-21 Thread Daniel Kahn Gillmor
Hey folks--

I just did a bit of testing and cleanup for out-of-tree builds (see the
minor patches that should have landed on the list in the last hour or
two).

For me, "make check" in an out-of-tree build works fine now, with the
exception of T391-python-cffi.sh and T395-ruby.sh.

I'm afraid i don't know enough about either python or ruby to see the
obvious fix for these builds, but i do note that they're the only builds
which depend on copying their source into the out-of-tree location
(assuming no one proposes an improvement for
id:20200522010359.715688-1-...@fifthhorseman.net).

Just thought i'd flag this as an outstanding problem that hopefully
someone clever feels like digging into.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] tests: fix test_json_nodes() in out-of-tree builds

2020-05-21 Thread Daniel Kahn Gillmor
In out-of-tree builds, $TEST_DIRECTORY doesn't contain
json_check_nodes.py.  This caused 27 tests to fail in such an
environment.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 58972339..792b1cb9 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -556,7 +556,7 @@ test_json_nodes () {
 
if ! test_skip "$test_subtest_name"
then
-   output=$(PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON 
"$TEST_DIRECTORY"/json_check_nodes.py "$@")
+   output=$(PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON 
"$NOTMUCH_SRCDIR"/test/json_check_nodes.py "$@")
if [ "$?" = 0 ]
then
test_ok_
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 3/5] build: optionally build python-cffi bindings

2020-05-21 Thread Daniel Kahn Gillmor
On Mon 2019-11-04 23:26:25 +0200, Tomi Ollila wrote:
> how bad does out-of-tree build break with this -- do we need to do
> the same as with ruby bindings (copy sources -- do we still do so)? or does
> python provide better alternative..?

I've just posted id:20200522010359.715688-1-...@fifthhorseman.net to
apply the same hack as the ruby bindings here.  I welcome any better
proposals, but think we should at least have the build complete!

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] python-cffi: enable out-of-tree builds

2020-05-21 Thread Daniel Kahn Gillmor
This is a simple hack to enable out-of-tree builds, a concern raised
by Tomi in id:m24kzjib9a@guru.guru-group.fi

This change at least enables "make check" to complete without error,
but I'm sure it could be improved.  I am not expert enough in
setuptools to know how.

Signed-off-by: Daniel Kahn Gillmor 
---
 configure | 8 
 1 file changed, 8 insertions(+)

diff --git a/configure b/configure
index 37368bda..f0472e8e 100755
--- a/configure
+++ b/configure
@@ -70,6 +70,14 @@ if [ "$srcdir" != "." ]; then
 mkdir bindings/ruby
 cp -a "$srcdir"/bindings/ruby/*.[ch] bindings/ruby
 cp -a "$srcdir"/bindings/ruby/extconf.rb bindings/ruby
+
+# Use the same hack to replicate python-cffi source for
+# out-of-tree builds (again, not ideal).
+mkdir bindings/python-cffi
+cp -a "$srcdir"/bindings/python-cffi/tests \
+   "$srcdir"/bindings/python-cffi/notmuch2 \
+   "$srcdir"/bindings/python-cffi/setup.py \
+   bindings/python-cffi/
 fi
 
 # Set several defaults (optionally specified by the user in
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: test suite: FIXED messages are misordered with tests

2020-05-21 Thread Daniel Kahn Gillmor
On Fri 2020-05-22 00:57:14 +0300, Tomi Ollila wrote:
> We've accumulated quite a bit of mess during these years to the test
> system, which makes it harder to do larger adjustments (and not (yet)
> mentioning even larger refactorings...).

The last significant refactoring was probably when jrollins and i got
the test suite running in parallel, and that might not even qualify as
"major".  Of course, the most effective tool for encouraging any
refactoring is having a robust and reliable test suite to make sure that
nothing broke.  But now we're talking about a test suite for the test
suite…  i dunno how far we want to go down the rabbit hole!

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2 v3] smime: tests of X.509 certificate validity are known-broken on GMime < 3.2.7

2020-05-21 Thread Daniel Kahn Gillmor
When checking cryptographic signatures, Notmuch relies on GMime to
tell it whether the certificate that signs a message has a valid User
ID or not.

If the User ID is not valid, then notmuch does not report the signer's
User ID to the user.  This means that the consumer of notmuch's
cryptographic summary of a message (or of its protected headers) can
be confident in relaying the reported identity to the user.

However, some versions of GMime before 3.2.7 cannot report Certificate
validity for X.509 certificates.  This is resolved upstream in GMime
at https://github.com/jstedfast/gmime/pull/90.

We adapt to this by marking tests of reported User IDs for
S/MIME-signed messages as known-broken if GMime is older than 3.2.7
and has not been patched.

If GMime >= 3.2.7 and certificate validity still doesn't work for
X.509 certs, then there has likely been a regression in GMime and we
should fail early, during ./configure.

To break out these specific User ID checks from other checks, i had to
split some tests into two parts, and reuse $output across the two
subtests.

Signed-off-by: Daniel Kahn Gillmor 
---
 configure  | 83 +-
 test/T355-smime.sh | 17 ---
 test/T356-protected-headers.sh | 13 +-
 3 files changed, 102 insertions(+), 11 deletions(-)

diff --git a/configure b/configure
index 0cfdaa6f..37368bda 100755
--- a/configure
+++ b/configure
@@ -494,7 +494,7 @@ int main () {
 if (error) return !! fprintf (stderr, "failed to instantiate parser with 
test/corpora/crypto/basic-encrypted.eml\n");
 
 body = GMIME_MULTIPART_ENCRYPTED(g_mime_message_get_mime_part 
(g_mime_parser_construct_message (parser, NULL)));
-if (body == NULL) return !!fprintf (stderr, "did not find a 
multipart encrypted message\n");
+if (body == NULL) return !! fprintf (stderr, "did not find a multipart 
encrypted message\n");
 
 output = g_mime_multipart_encrypted_decrypt (body, 
GMIME_DECRYPT_EXPORT_SESSION_KEY, NULL, _result, );
 if (error || output == NULL) return !! fprintf (stderr, "decryption 
failed\n");
@@ -536,6 +536,82 @@ EOF
 if [ -n "$TEMP_GPG" -a -d "$TEMP_GPG" ]; then
 rm -rf "$TEMP_GPG"
 fi
+
+# see https://github.com/jstedfast/gmime/pull/90
+# should be fixed in GMime in 3.2.7, but some distros might patch
+printf "Checking for GMime X.509 certificate validity... "
+
+cat > _check_x509_validity.c <
+#include 
+
+int main () {
+GError *error = NULL;
+GMimeParser *parser = NULL;
+GMimeApplicationPkcs7Mime *body = NULL;
+GMimeSignatureList *sig_list = NULL;
+GMimeSignature *sig = NULL;
+GMimeCertificate *cert = NULL;
+GMimeObject *output = NULL;
+GMimeValidity validity = GMIME_VALIDITY_UNKNOWN;
+int len;
+
+g_mime_init ();
+parser = g_mime_parser_new ();
+g_mime_parser_init_with_stream (parser, 
g_mime_stream_file_open("$srcdir/test/corpora/pkcs7/smime-onepart-signed.eml", 
"r", ));
+if (error) return !! fprintf (stderr, "failed to instantiate parser with 
test/corpora/pkcs7/smime-onepart-signed.eml\n");
+
+body = GMIME_APPLICATION_PKCS7_MIME(g_mime_message_get_mime_part 
(g_mime_parser_construct_message (parser, NULL)));
+if (body == NULL) return !!fprintf (stderr, "did not find a 
application/pkcs7 message\n");
+
+sig_list = g_mime_application_pkcs7_mime_verify (body, GMIME_VERIFY_NONE, 
, );
+if (error || output == NULL) return !! fprintf (stderr, "verify failed\n");
+
+if (sig_list == NULL) return !! fprintf (stderr, "no GMimeSignatureList 
found\n");
+len = g_mime_signature_list_length (sig_list);
+if (len != 1) return !! fprintf (stderr, "expected 1 signature, got %d\n", 
len);
+sig = g_mime_signature_list_get_signature (sig_list, 0);
+if (sig == NULL) return !! fprintf (stderr, "no GMimeSignature found at 
position 0\n");
+cert = g_mime_signature_get_certificate (sig);
+if (cert == NULL) return !! fprintf (stderr, "no GMimeCertificate 
found\n");
+validity = g_mime_certificate_get_id_validity (cert);
+if (validity != GMIME_VALIDITY_FULL) return !! fprintf (stderr, "Got 
validity %d, expected %d\n", validity, GMIME_VALIDITY_FULL);
+
+return 0;
+}
+EOF
+if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XX"); then
+printf 'No.\nCould not make tempdir for testing X.509 certificate 
validity support.\n'
+errors=$((errors + 1))
+elif ${CC} ${CFLAGS} ${gmime_cflags} _check_x509_validity.c 
${gmime_ldflags} -o _check_x509_validity \
+&& echo disable-crl-checks > "$TEMP_GPG/gpgsm.conf" \
+&& echo 
"4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$TEMP_GPG/trustlis

Re: [PATCH 2/2 v2] smime: tests of X.509 certificate validity are known-broken on GMime < 3.2.7

2020-05-21 Thread Daniel Kahn Gillmor
Thanks for the review, and for the poke about out-of-tree builds on IRC,
Bremner.  Another revision is coming in a minute.  Notes below…

On Thu 2020-05-21 20:29:05 -0300, David Bremner wrote:
> I find these long lines with !! in the middle pretty surprising. Is
> there some reason for this style? It doesn't seem to fit with the usual
> conventions.

Hm, do we even have conventions for inlined C in ./configure?

If you'd rather i expand these to something more verbose, i can do so,
but i was under the impression that we wanted to keep these C
interstitials fairly compact so that ./configure is still (somewhat)
readable as a shell script.

The "return !! fprintf…" idiom is a compact way to get a non-zero
process error code and an error message to stderr without introducing a
code block.  The only way for fprintf to return 0 (which would result in
the process returning 0) is if 0 bytes are written and no error
occurred, which isn't possible with any of the format strings supplied
here.

Another approach would be to pull the C entirely out of ./configure, but
that could have a problem when dealing with out-of-tree builds.  (i just
noticed a problem for this test with out-of-tree builds, which i'll
revise in a minute)

> This line in particular has a tab in the middle.

i dunno how that got there, i'll have it fixed in the upcoming revision.

>> +elif ${CC} ${CFLAGS} ${gmime_cflags} _check_x509_validity.c 
>> ${gmime_ldflags} -o _check_x509_validity \
>
> The other test files are cleaned up in configure (source and binary)
> once we are done with them.

good point, i'll ensure that they get cleaned up alongside
_check_session_keys* in the upcoming revision.

> As far as I could follow, the changes to the tests themselves look
> reasonable.

thanks for the thoughtful review!

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs: split-window-sensibly in tree mode with open message

2020-05-21 Thread Daniel Kahn Gillmor
On Sat 2020-05-02 20:11:09 -0400, Radu Butoi wrote:

> This uses the standard Emacs function `split-window-sensibly` to split a
> window horizontally or vertically depending on space when opening a
> message in tree view. By default, split-width-threshold is 160 columns
> (and -height- is nil), so screens wider than 160 will be split
> horizontally.
>
> This is based on an older proposal [1] which manually did the
> calculation of width. The main issues there were (1) lack of
> configurability and (2) lack of testing. I don't have an answer for
> testing, but this allows users to configure two thresholds using
> built-in variables, an improvement.

I like this proposal, and the simplification that it gives to the
notmuch-emacs codebase.  However, this thread is the first place i've
learned about split-window-sensibly, so i'm probably not eligible to
really judge the merits here.

As far as testing goes, a test would be nice -- is this something you
could add to test/T460-emacs-tree.sh ?  Testing UI/UX issues is always
pretty tough though, and it's not clear to me that we're actually
already testing the existing "(/ (window-height) 4)" business anyway.

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs: add notmuch-expr, sexp-style queries

2020-05-21 Thread Daniel Kahn Gillmor
On Wed 2020-05-13 20:00:24 +1000, Tom Fitzhenry wrote:
> notmuch-expr allows you to write notmuch search queries in sexp style like:
>
> (notmuch-expr
>   '(and
> (to "emacs-devel")
> "info manual"
> (or
>   (not (is "spam"))
>   (is "important"
>
> which will generate the textual query:
>
> "to:emacs-devel AND (NOT is:spam OR is:important) AND \"info manual\""

I like this idea!

> +(defun notmuch-expr--quote (s)
> +  ;; FIXME Escape s.
> +  (concat "\"" s "\""))

Shouldn't this FIXME be resolved before we consider merging?

  --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs docstrings: consistent indentation, newlines, periods

2020-05-21 Thread Daniel Kahn Gillmor
On Mon 2020-05-04 00:21:36 +0300, Tomi Ollila wrote:
> Fixed emacs docstrings to be consistent. No functional change.
>
> - removed some (accidental) indentation
> - removed some trailing newlines
> - added trailing periods where missing (some exclusions)

This all looks good to me, except for the following changes.

I think if the last line of the docstring is an example, deliberately
indented to indicate that it is verbatim code, adding a trailing period
is a mistake, because it encourages naive users to make the obvious
error of including the period in their text.

These were the examples i noticed in a skim of the changeset:

> --- a/emacs/notmuch-draft.el
> +++ b/emacs/notmuch-draft.el
> @@ -45,7 +45,7 @@ (defcustom notmuch-draft-tags '("+draft")
>  For example, if you wanted to give the message a \"draft\" tag
>  but not the (normally added by default) \"inbox\" tag, you would
>  set:
> -(\"+draft\" \"-inbox\")"
> +(\"+draft\" \"-inbox\")."
>:type '(repeat string)
>:group 'notmuch-draft)
>  


> --- a/emacs/notmuch-lib.el
> +++ b/emacs/notmuch-lib.el
> @@ -145,7 +145,7 @@ (defcustom notmuch-archive-tags '("-inbox")
>  
>  For example, if you wanted to remove an \"inbox\" tag and add an
>  \"archived\" tag, you would set:
> -(\"-inbox\" \"+archived\")"
> +(\"-inbox\" \"+archived\")."
>:type '(repeat string)
>:group 'notmuch-search
>:group 'notmuch-show)


> --- a/emacs/notmuch-message.el
> +++ b/emacs/notmuch-message.el
> @@ -33,7 +33,7 @@ (defcustom notmuch-message-replied-tags '("+replied")
>  
>  For example, if you wanted to add a \"replied\" tag and remove
>  the \"inbox\" and \"todo\" tags, you would set:
> -(\"+replied\" \"-inbox\" \"-todo\")"
> +(\"+replied\" \"-inbox\" \"-todo\")."
>:type '(repeat string)
>:group 'notmuch-send)
>  
> @@ -46,7 +46,7 @@ (defcustom notmuch-message-forwarded-tags '("+forwarded")
>  
>  For example, if you wanted to add a \"forwarded\" tag and remove
>  the \"inbox\" tag, you would set:
> -(\"+forwarded\" \"-inbox\")"
> +(\"+forwarded\" \"-inbox\")."
>:type '(repeat string)
>:group 'notmuch-send)


> --- a/emacs/notmuch-show.el
> +++ b/emacs/notmuch-show.el
> @@ -237,7 +237,7 @@ (defcustom notmuch-show-mark-read-tags '("-unread")
>  
>  For example, if you wanted to remove an \"unread\" tag and add a
>  \"read\" tag (which would make little sense), you would set:
> -(\"-unread\" \"+read\")"
> +(\"-unread\" \"+read\")."
>:type '(repeat string)
>:group 'notmuch-show)


I'd prefer this changeset to not have trailing periods when the
docstring ends in an example.   (and maybe also to clean up any trailing
periods that might already be in such a docstring, if they exist)

I recognize this is something of an aesthetic position with no
objectively correct answer, but so is the question of trailing periods,
i suppose.  I'm not going to die on this hill, though.

Regards,

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: waiting tag

2020-05-21 Thread Daniel Kahn Gillmor
On Mon 2020-05-04 09:25:59 +0200, Gregor Zattler wrote:
> * Keegan Carruthers-Smith  [2020-05-03; 22:37]:
>>   notmuch tag -waiting -- tag:waiting and 'thread:{tag:new}'
>
> but this removes the waiting tag if there is some response
> to some message in the thread, not necessary to the message
> tagged waiting, isn't it?

Yes, you're right, this is not exactly the thing you were asking for.

fwiw, i don't know how to provide the thing you were asking for either
:(

--dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: test suite: FIXED messages are misordered with tests

2020-05-21 Thread Daniel Kahn Gillmor
On Thu 2020-05-21 00:16:48 +0300, Tomi Ollila wrote:
> (just tested this latest works)

Thanks for looking into this, Tomi!

Do you have a patch to propose?

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: test suite: FIXED messages are misordered with tests

2020-05-19 Thread Daniel Kahn Gillmor
On Tue 2020-05-12 20:14:07 -0300, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>>
>> Clearly, that FIXED should come *after* the "T356-protected-headers:"
>> separator.
>>
>
> Can you reproduce this without running the tests in parallel?

Yep:

I've seen the misordered output when using "make -j4 check".

I've also seen it with plain old "make check".

And i can replicate it with:

make check NOTMUCH_TEST_SERIALIZE=1 NOTMUCH_TESTS=T355-smime.sh

(i don't think NOTMUCH_TEST_SERIALIZE=1 makes any difference when only
one item is present in NOTMUCH_TESTS)

I also tried "make check NOTMUCH_TEST_SERIALIZE=1" (so slow, i'm spoiled
by the parallel test suite!) and yes, i see the same problem there.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 3/9] cli: include wrapped part of PKCS#7 SignedData in the MIME tree

2020-05-12 Thread Daniel Kahn Gillmor
Unwrap a PKCS#7 SignedData part unconditionally when the cli is
traversing the MIME tree, and return it as a "child" of what would
otherwise be a leaf in the tree.

Unfortunately, this also breaks the JSON output.  We will fix that
next.

Signed-off-by: Daniel Kahn Gillmor 
---
 mime-node.c| 23 +--
 test/T355-smime.sh |  2 +-
 2 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index ff6805bf..b6431e3b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -220,8 +220,17 @@ node_verify (mime_node_t *node, GMimeObject *part)
 notmuch_status_t status;
 
 node->verify_attempted = true;
-node->sig_list = g_mime_multipart_signed_verify (
-   GMIME_MULTIPART_SIGNED (part), GMIME_VERIFY_NONE, );
+if (GMIME_IS_APPLICATION_PKCS7_MIME (part))
+   node->sig_list = g_mime_application_pkcs7_mime_verify (
+   GMIME_APPLICATION_PKCS7_MIME (part), GMIME_VERIFY_NONE, 
>unwrapped_child, );
+else
+   node->sig_list = g_mime_multipart_signed_verify (
+   GMIME_MULTIPART_SIGNED (part), GMIME_VERIFY_NONE, );
+
+if (node->unwrapped_child) {
+   node->nchildren = 1;
+   set_unwrapped_child_destructor (node);
+}
 
 if (node->sig_list)
set_signature_list_destructor (node);
@@ -376,6 +385,12 @@ _mime_node_set_up_part (mime_node_t *node, GMimeObject 
*part, int numchild)
} else {
node_verify (node, part);
}
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (part) &&
+  GMIME_SECURE_MIME_TYPE_SIGNED_DATA == 
g_mime_application_pkcs7_mime_get_smime_type (GMIME_APPLICATION_PKCS7_MIME 
(part))) {
+   /* If node->ctx->crypto->verify is false, it would be better
+* to just unwrap (instead of verifying), but
+* https://github.com/jstedfast/gmime/issues/67 */
+   node_verify (node, part);
 } else {
if (_notmuch_message_crypto_potential_payload (node->ctx->msg_crypto, 
part, node->parent ? node->parent->part : NULL, numchild) &&
node->ctx->msg_crypto->decryption_status == 
NOTMUCH_MESSAGE_DECRYPTED_FULL) {
@@ -409,6 +424,10 @@ mime_node_child (mime_node_t *parent, int child)
GMIME_MULTIPART (parent->part), child);
 } else if (GMIME_IS_MESSAGE (parent->part)) {
sub = g_mime_message_get_mime_part (GMIME_MESSAGE (parent->part));
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (parent->part) &&
+  parent->unwrapped_child &&
+  child == 0) {
+   sub = parent->unwrapped_child;
 } else {
/* This should have been caught by _mime_node_set_up_part */
INTERNAL_ERROR ("Unexpected GMimeObject type: %s",
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 7c28282a..4de0fbef 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -142,7 +142,6 @@ expected='#notmuch-dump batch-tag:3 config,properties,tags
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "show contents of PKCS#7 SignedData message"
-test_subtest_known_broken
 output=$(notmuch show --format=raw --part=2 
id:smime-onepart-signed@protected-headers.example)
 whitespace=' '
 expected="Bob, we need to cancel this contract.
@@ -178,6 +177,7 @@ On Tue, 26 Nov 2019 20:11:29 -0400, Alice Lovelace 
 wrote:
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "show PKCS#7 SignedData outputs valid JSON"
+test_subtest_known_broken
 output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
 test_valid_json "$output"
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Handle PKCS#7 S/MIME messages v2

2020-05-12 Thread Daniel Kahn Gillmor
This revision of the PKCS#7 S/MIME handling series is based on (and
very similar to) the series sent on the thread starting at
id:20200430201328.725651-1-...@fifthhorseman.net

However, it is rebased after more gracefully handling the subtle
errors in X.509 certificate validity when built against older versions
of GMime.

In particular, the patch found at
id:2020051010.371054-1-...@fifthhorseman.net must be applied
before this series can be applied.

Feedback and critiques welcome,

 --dkg



___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 1/9] lib: index PKCS7 SignedData parts

2020-05-12 Thread Daniel Kahn Gillmor
When we are indexing, we should treat SignedData parts the same way
that we treat a multipart object, indexing the wrapped part as a
distinct MIME object.

Unfortunately, this means doing some sort of cryptographic
verification whose results we throw away, because GMime doesn't offer
us any way to unwrap without doing signature verification.

I've opened https://github.com/jstedfast/gmime/issues/67 to request
the capability from GMime but for now, we'll just accept the
additional performance hit.

As we do this indexing, we also apply the "signed" tag, by analogy
with how we handle multipart/signed messages.  These days, that kind
of change should probably be done with a property instead, but that's
a different set of changes.  This one is just for consistency.

Note that we are currently *only* handling signedData parts, which are
basically clearsigned messages.  PKCS#7 parts can also be
envelopedData and authEnvelopedData (which are effectively encryption
layers), and compressedData (which afaict isn't implemented anywhere,
i've never encountered it).  We're laying the groundwork for indexing
these other S/MIME types here, but we're only dealing with signedData
for now.

Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc   | 57 ++
 test/T355-smime.sh |  2 --
 2 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 158ba5cf..bbf13dc5 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -372,6 +372,12 @@ _index_encrypted_mime_part (notmuch_message_t *message, 
notmuch_indexopts_t *ind
GMimeMultipartEncrypted *part,
_notmuch_message_crypto_t *msg_crypto);
 
+static void
+_index_pkcs7_part (notmuch_message_t *message,
+  notmuch_indexopts_t *indexopts,
+  GMimeObject *part,
+  _notmuch_message_crypto_t *msg_crypto);
+
 /* Callback to generate terms for each mime part of a message. */
 static void
 _index_mime_part (notmuch_message_t *message,
@@ -466,6 +472,11 @@ _index_mime_part (notmuch_message_t *message,
goto DONE;
 }
 
+if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) {
+   _index_pkcs7_part (message, indexopts, part, msg_crypto);
+   goto DONE;
+}
+
 if (! (GMIME_IS_PART (part))) {
_notmuch_database_log (notmuch_message_get_database (message),
   "Warning: Not indexing unknown mime part: %s.\n",
@@ -608,6 +619,52 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 
 }
 
+static void
+_index_pkcs7_part (notmuch_message_t *message,
+  notmuch_indexopts_t *indexopts,
+  GMimeObject *part,
+  _notmuch_message_crypto_t *msg_crypto)
+{
+GMimeApplicationPkcs7Mime *pkcs7;
+GMimeSecureMimeType p7type;
+GMimeObject *mimeobj = NULL;
+GMimeSignatureList *sigs = NULL;
+GError *err = NULL;
+notmuch_database_t *notmuch = NULL;
+
+pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part);
+p7type = g_mime_application_pkcs7_mime_get_smime_type (pkcs7);
+notmuch = notmuch_message_get_database (message);
+_index_content_type (message, part);
+
+if (p7type == GMIME_SECURE_MIME_TYPE_SIGNED_DATA) {
+   sigs = g_mime_application_pkcs7_mime_verify (pkcs7, GMIME_VERIFY_NONE, 
, );
+   if (sigs == NULL) {
+   _notmuch_database_log (notmuch, "Failed to verify PKCS#7 SignedData 
during indexing. (%d:%d) [%s]\n",
+  err->domain, err->code, err->message);
+   g_error_free (err);
+   goto DONE;
+   }
+   _notmuch_message_add_term (message, "tag", "signed");
+   GMimeObject *toindex = mimeobj;
+   if (_notmuch_message_crypto_potential_payload (msg_crypto, mimeobj, 
part, 0) &&
+   msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) {
+   toindex = _notmuch_repair_crypto_payload_skip_legacy_display 
(mimeobj);
+   if (toindex != mimeobj)
+   notmuch_message_add_property (message, "index.repaired", 
"skip-protected-headers-legacy-display");
+   }
+   _index_mime_part (message, indexopts, toindex, msg_crypto);
+} else {
+   _notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 
smime-type '%s'\n",
+  g_mime_object_get_content_type_parameter (part, 
"smime-type"));
+}
+ DONE:
+if (mimeobj)
+   g_object_unref (mimeobj);
+if (sigs)
+   g_object_unref (sigs);
+}
+
 static notmuch_status_t
 _notmuch_message_index_user_headers (notmuch_message_t *message, GMimeMessage 
*mime_message)
 {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index f8e8e396..a7eecedf 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -132,13 +132,11 @@ expected=''
 test_expect_equal "$expected" &q

[PATCH v2 2/9] smime: Identify encrypted S/MIME parts during indexing

2020-05-12 Thread Daniel Kahn Gillmor
We don't handle them correctly yet, but we can at least mark them as
being encrypted.

Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc   | 4 
 test/T355-smime.sh | 1 -
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/index.cc b/lib/index.cc
index bbf13dc5..f029b334 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -654,6 +654,10 @@ _index_pkcs7_part (notmuch_message_t *message,
notmuch_message_add_property (message, "index.repaired", 
"skip-protected-headers-legacy-display");
}
_index_mime_part (message, indexopts, toindex, msg_crypto);
+} else if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
+   _notmuch_message_add_term (message, "tag", "encrypted");
+   if (notmuch_indexopts_get_decrypt_policy (indexopts) != 
NOTMUCH_DECRYPT_FALSE)
+   _notmuch_database_log (notmuch, "Cannot decrypt PKCS#7 
envelopedData (S/MIME encrypted messages)\n");
 } else {
_notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 
smime-type '%s'\n",
   g_mime_object_get_content_type_parameter (part, 
"smime-type"));
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index a7eecedf..7c28282a 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -98,7 +98,6 @@ test_json_nodes <<<"$output" \
 
'crypto_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Notmuch 
Test Suite"'
 
 test_begin_subtest "encrypted+signed message is known to be encrypted, but 
signature is unknown"
-test_subtest_known_broken
 output=$(notmuch search subject:"test encrypted message 001")
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox)"
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 4/9] cli/show: If a leaf part has children, show them instead of omitting

2020-05-12 Thread Daniel Kahn Gillmor
Until we did PKCS#7 unwrapping, no leaf MIME part could have a child.

Now, we treat the unwrapped MIME part as the child of the PKCS#7
SignedData object.  So in that case, we want to show it instead of
deliberately omitting the content.

This fixes the test of the protected subject in
id:smime-onepart-signed@protected-headers.example.

Signed-off-by: Daniel Kahn Gillmor 
---
 notmuch-show.c | 11 ++-
 test/T355-smime.sh |  6 +++---
 test/T356-protected-headers.sh |  3 +--
 3 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index ab1cd144..36265043 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -759,7 +759,16 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
sp->string_len (sp, (char *) part_content->data, part_content->len);
g_object_unref (stream_memory);
} else {
-   format_omitted_part_meta_sprinter (sp, meta, GMIME_PART 
(node->part));
+   /* if we have a child part despite being a standard
+* (non-multipart) MIME part, that means there is
+* something to unwrap, which we will present in
+* content: */
+   if (node->nchildren) {
+   sp->map_key (sp, "content");
+   sp->begin_list (sp);
+   nclose = 1;
+   } else
+   format_omitted_part_meta_sprinter (sp, meta, GMIME_PART 
(node->part));
}
 } else if (GMIME_IS_MULTIPART (node->part)) {
sp->map_key (sp, "content");
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 4de0fbef..03aada20 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -177,12 +177,10 @@ On Tue, 26 Nov 2019 20:11:29 -0400, Alice Lovelace 
 wrote:
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "show PKCS#7 SignedData outputs valid JSON"
-test_subtest_known_broken
 output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
 test_valid_json "$output"
 
 test_begin_subtest "Verify signature on PKCS#7 SignedData message"
-test_subtest_known_broken
 output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
 
 test_json_nodes <<<"$output" \
@@ -192,7 +190,9 @@ test_json_nodes <<<"$output" \
 
'status:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"'
 
 test_begin_subtest "Verify signature on PKCS#7 SignedData message signer User 
ID"
-test_subtest_known_broken
+if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ]; then
+test_subtest_known_broken
+fi
 test_json_nodes <<<"$output" \
 
'userid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice 
Lovelace"'
 
diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh
index 5fd27434..5beffaf0 100755
--- a/test/T356-protected-headers.sh
+++ b/test/T356-protected-headers.sh
@@ -157,7 +157,6 @@ test_expect_equal "$output" 
id:protected-with-legacy-display@crypto.notmuchmail.
 
 for variant in multipart-signed onepart-signed; do
 test_begin_subtest "verify signed PKCS#7 subject ($variant)"
-[ "$variant" = multipart-signed ] || test_subtest_known_broken
 output=$(notmuch show --verify --format=json 
"id:smime-${variant}@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
@@ -165,7 +164,7 @@ for variant in multipart-signed onepart-signed; do
 
'sig_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"'
 \
 'not_encrypted:[0][0][0]["crypto"]!"decrypted"'
 test_begin_subtest "verify signed PKCS#7 subject ($variant) signer User ID"
-if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ] || [ "$variant" != 
multipart-signed ]; then
+if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ]; then
 test_subtest_known_broken
 fi
 test_json_nodes <<<"$output" \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 5/9] cli/reply: Ignore PKCS#7 wrapper parts when replying

2020-05-12 Thread Daniel Kahn Gillmor
When composing a reply, no one wants to see this line in the proposed
message:

Non-text part: application/pkcs7-mime

So we hide it, the same way we hide PGP/MIME cruft.

Signed-off-by: Daniel Kahn Gillmor 
---
 notmuch-reply.c| 5 +++--
 test/T355-smime.sh | 1 -
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index 2c30f6f9..ceb4f39b 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -65,8 +65,9 @@ format_part_reply (GMimeStream *stream, mime_node_t *node)
GMimeContentDisposition *disposition = 
g_mime_object_get_content_disposition (node->part);
 
if (g_mime_content_type_is_type (content_type, "application", 
"pgp-encrypted") ||
-   g_mime_content_type_is_type (content_type, "application", 
"pgp-signature")) {
-   /* Ignore PGP/MIME cruft parts */
+   g_mime_content_type_is_type (content_type, "application", 
"pgp-signature") ||
+   g_mime_content_type_is_type (content_type, "application", 
"pkcs7-mime")) {
+   /* Ignore PGP/MIME and S/MIME cruft parts */
} else if (g_mime_content_type_is_type (content_type, "text", "*") &&
   ! g_mime_content_type_is_type (content_type, "text", 
"html")) {
show_text_part_content (node->part, stream, 
NOTMUCH_SHOW_TEXT_PART_REPLY);
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 03aada20..8d225bc1 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -156,7 +156,6 @@ OpenPGP Example Corp"
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "reply to PKCS#7 SignedData message with proper quoting and 
attribution"
-test_subtest_known_broken
 output=$(notmuch reply id:smime-onepart-signed@protected-headers.example)
 expected="From: Notmuch Test Suite 
 Subject: Re: The FooCorp contract
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 7/9] crypto: handle PKCS#7 envelopedData in _notmuch_crypto_decrypt

2020-05-12 Thread Daniel Kahn Gillmor
In the two places where _notmuch_crypto_decrypt handles
multipart/encrypted messages (PGP/MIME), we should also handle PKCS#7
envelopedData (S/MIME).

This is insufficient for fully handling S/MIME encrypted data because
_notmuch_crypto_decrypt isn't yet actually invoked for envelopedData
parts, but that will happen in the following changes.

Signed-off-by: Daniel Kahn Gillmor 
---
 util/crypto.c | 32 ++--
 1 file changed, 26 insertions(+), 6 deletions(-)

diff --git a/util/crypto.c b/util/crypto.c
index fbd5f011..c09f467b 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -55,10 +55,21 @@ _notmuch_crypto_decrypt (bool *attempted,
}
if (attempted)
*attempted = true;
-   ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part),
- GMIME_DECRYPT_NONE,
- 
notmuch_message_properties_value (list),
- decrypt_result, err);
+   if (GMIME_IS_MULTIPART_ENCRYPTED (part)) {
+   ret = g_mime_multipart_encrypted_decrypt 
(GMIME_MULTIPART_ENCRYPTED (part),
+ GMIME_DECRYPT_NONE,
+ 
notmuch_message_properties_value (list),
+ decrypt_result, err);
+   } else if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) {
+   GMimeApplicationPkcs7Mime *pkcs7 = GMIME_APPLICATION_PKCS7_MIME 
(part);
+   GMimeSecureMimeType type = 
g_mime_application_pkcs7_mime_get_smime_type (pkcs7);
+   if (type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
+   ret = g_mime_application_pkcs7_mime_decrypt (pkcs7,
+
GMIME_DECRYPT_NONE,
+
notmuch_message_properties_value (list),
+
decrypt_result, err);
+   }
+   }
if (ret)
break;
}
@@ -81,8 +92,17 @@ _notmuch_crypto_decrypt (bool *attempted,
 GMimeDecryptFlags flags = GMIME_DECRYPT_NONE;
 if (decrypt == NOTMUCH_DECRYPT_TRUE && decrypt_result)
flags |= GMIME_DECRYPT_EXPORT_SESSION_KEY;
-ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part), flags, NULL,
- decrypt_result, err);
+if (GMIME_IS_MULTIPART_ENCRYPTED (part)) {
+   ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part), flags, NULL,
+ decrypt_result, err);
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) {
+   GMimeApplicationPkcs7Mime *pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part);
+   GMimeSecureMimeType p7type = 
g_mime_application_pkcs7_mime_get_smime_type (pkcs7);
+   if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
+   ret = g_mime_application_pkcs7_mime_decrypt (pkcs7, flags, NULL,
+decrypt_result, err);
+   }
+}
 return ret;
 }
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 6/9] crypto: Make _notmuch_crypto_decrypt take a GMimeObject

2020-05-12 Thread Daniel Kahn Gillmor
As we prepare to handle S/MIME-encrypted PKCS#7 EnvelopedData (which
is not multipart), we don't want to be limited to passing only
GMimeMultipartEncrypted MIME parts to _notmuch_crypto_decrypt.

There is no functional change here, just a matter of adjusting how we
pass arguments internally.

Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc  | 8 
 mime-node.c   | 3 +--
 util/crypto.c | 6 +++---
 util/crypto.h | 2 +-
 4 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index f029b334..da9a3abe 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -369,7 +369,7 @@ _index_content_type (notmuch_message_t *message, 
GMimeObject *part)
 
 static void
 _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t 
*indexopts,
-   GMimeMultipartEncrypted *part,
+   GMimeObject *part,
_notmuch_message_crypto_t *msg_crypto);
 
 static void
@@ -439,7 +439,7 @@ _index_mime_part (notmuch_message_t *message,
 g_mime_multipart_get_part (multipart, i));
if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) {
_index_encrypted_mime_part (message, indexopts,
-   GMIME_MULTIPART_ENCRYPTED 
(part),
+   part,
msg_crypto);
} else {
if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) {
@@ -551,7 +551,7 @@ _index_mime_part (notmuch_message_t *message,
 static void
 _index_encrypted_mime_part (notmuch_message_t *message,
notmuch_indexopts_t *indexopts,
-   GMimeMultipartEncrypted *encrypted_data,
+   GMimeObject *encrypted_data,
_notmuch_message_crypto_t *msg_crypto)
 {
 notmuch_status_t status;
@@ -603,7 +603,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
g_object_unref (decrypt_result);
 }
 GMimeObject *toindex = clear;
-if (_notmuch_message_crypto_potential_payload (msg_crypto, clear, 
GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT) &&
+if (_notmuch_message_crypto_potential_payload (msg_crypto, clear, 
encrypted_data, GMIME_MULTIPART_ENCRYPTED_CONTENT) &&
msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) {
toindex = _notmuch_repair_crypto_payload_skip_legacy_display (clear);
if (toindex != clear)
diff --git a/mime-node.c b/mime-node.c
index b6431e3b..c2ee858d 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -253,7 +253,6 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part)
 GError *err = NULL;
 GMimeDecryptResult *decrypt_result = NULL;
 notmuch_status_t status;
-GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
 notmuch_message_t *message = NULL;
 
 if (! node->unwrapped_child) {
@@ -266,7 +265,7 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part)
node->unwrapped_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
 
node->ctx->crypto->decrypt,
 message,
-encrypteddata, 
_result, );
+part, _result, 
);
if (node->unwrapped_child)
set_unwrapped_child_destructor (node);
 }
diff --git a/util/crypto.c b/util/crypto.c
index 0bb6f526..fbd5f011 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -34,7 +34,7 @@ GMimeObject *
 _notmuch_crypto_decrypt (bool *attempted,
 notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
-GMimeMultipartEncrypted *part,
+GMimeObject *part,
 GMimeDecryptResult **decrypt_result,
 GError **err)
 {
@@ -55,7 +55,7 @@ _notmuch_crypto_decrypt (bool *attempted,
}
if (attempted)
*attempted = true;
-   ret = g_mime_multipart_encrypted_decrypt (part,
+   ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part),
  GMIME_DECRYPT_NONE,
  
notmuch_message_properties_value (list),
  decrypt_result, err);
@@ -81,7 +81,7 @@ _notmuch_crypto_decrypt (bool *attempted,
 GMimeDecryptFlags flags = GMIME_DECRYPT_NONE;
 if (decrypt == NOTMUCH_DECRYPT_TRUE && decrypt_result)
flags |= GMIME_DECRYPT_EXPORT_SESSION_KEY;
-ret = g_mime_multipart_encrypted_decrypt (part, flags, NULL,
+ret = g_mime_multip

[PATCH v2 9/9] smime: Index cleartext of envelopedData when requested

2020-05-12 Thread Daniel Kahn Gillmor
Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc   | 5 +++--
 test/T355-smime.sh | 2 --
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index da9a3abe..826aa341 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -656,8 +656,9 @@ _index_pkcs7_part (notmuch_message_t *message,
_index_mime_part (message, indexopts, toindex, msg_crypto);
 } else if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
_notmuch_message_add_term (message, "tag", "encrypted");
-   if (notmuch_indexopts_get_decrypt_policy (indexopts) != 
NOTMUCH_DECRYPT_FALSE)
-   _notmuch_database_log (notmuch, "Cannot decrypt PKCS#7 
envelopedData (S/MIME encrypted messages)\n");
+   _index_encrypted_mime_part (message, indexopts,
+   part,
+   msg_crypto);
 } else {
_notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 
smime-type '%s'\n",
   g_mime_object_get_content_type_parameter (part, 
"smime-type"));
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 1f11725f..170f8649 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -107,12 +107,10 @@ test_begin_subtest "Reindex cleartext"
 test_expect_success "notmuch reindex --decrypt=true subject:'test encrypted 
message 001'"
 
 test_begin_subtest "signature is now known"
-test_subtest_known_broken
 output=$(notmuch search subject:"test encrypted message 001")
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
 
 test_begin_subtest "Encrypted body is indexed"
-test_subtest_known_broken
 output=$(notmuch search 'this is a test encrypted message')
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v2 8/9] smime: Pass PKCS#7 envelopedData to node_decrypt_and_verify

2020-05-12 Thread Daniel Kahn Gillmor
This change means we can support "notmuch show --decrypt=true" for
S/MIME encrypted messages, resolving several outstanding broken tests,
including all the remaining S/MIME protected header examples.

We do not yet handle indexing the cleartext of S/MIME encrypted
messages, though.

Signed-off-by: Daniel Kahn Gillmor 
---
 mime-node.c| 6 ++
 test/T355-smime.sh | 2 --
 test/T356-protected-headers.sh | 6 +++---
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index c2ee858d..f552e03a 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -390,6 +390,12 @@ _mime_node_set_up_part (mime_node_t *node, GMimeObject 
*part, int numchild)
 * to just unwrap (instead of verifying), but
 * https://github.com/jstedfast/gmime/issues/67 */
node_verify (node, part);
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (part) &&
+  GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA == 
g_mime_application_pkcs7_mime_get_smime_type (GMIME_APPLICATION_PKCS7_MIME 
(part)) &&
+  (node->ctx->crypto->decrypt != NOTMUCH_DECRYPT_FALSE)) {
+   node_decrypt_and_verify (node, part);
+   if (node->unwrapped_child && node->nchildren == 0)
+   node->nchildren = 1;
 } else {
if (_notmuch_message_crypto_potential_payload (node->ctx->msg_crypto, 
part, node->parent ? node->parent->part : NULL, numchild) &&
node->ctx->msg_crypto->decryption_status == 
NOTMUCH_MESSAGE_DECRYPTED_FULL) {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 8d225bc1..1f11725f 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -80,7 +80,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Decryption (notmuch CLI)"
-test_subtest_known_broken
 notmuch show --decrypt=true subject:"test encrypted message 001" |\
 grep "^This is a" > OUTPUT
 cat < EXPECTED
@@ -89,7 +88,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Cryptographic message status (encrypted+signed)"
-test_subtest_known_broken
 output=$(notmuch show --format=json --decrypt=true subject:"test encrypted 
message 001")
 test_json_nodes <<<"$output" \
 
'crypto_encrypted:[0][0][0]["crypto"]["decrypted"]["status"]="full"' \
diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh
index 5beffaf0..074a2345 100755
--- a/test/T356-protected-headers.sh
+++ b/test/T356-protected-headers.sh
@@ -173,7 +173,6 @@ done
 
 for variant in sign+enc sign+enc+legacy-disp; do
 test_begin_subtest "confirm signed and encrypted PKCS#7 subject ($variant)"
-test_subtest_known_broken
 output=$(notmuch show --decrypt=true --format=json 
"id:smime-${variant}@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
@@ -181,14 +180,15 @@ for variant in sign+enc sign+enc+legacy-disp; do
 
'sig_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"'
 \
 
'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}'
 test_begin_subtest "confirm signed and encrypted PKCS#7 subject ($variant) 
signer User ID"
-test_subtest_known_broken
+if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ]; then
+test_subtest_known_broken
+fi
 test_json_nodes <<<"$output" \
 
'sig_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice 
Lovelace"'
 
 done
 
 test_begin_subtest "confirm encryption-protected PKCS#7 subject 
(enc+legacy-disp)"
-test_subtest_known_broken
 output=$(notmuch show --decrypt=true --format=json 
"id:smime-enc+legacy-disp@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}'
 \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2 v2] smime: tests of X.509 certificate validity are known-broken on GMime < 3.2.7

2020-05-12 Thread Daniel Kahn Gillmor
When checking cryptographic signatures, Notmuch relies on GMime to
tell it whether the certificate that signs a message has a valid User
ID or not.

If the User ID is not valid, then notmuch does not report the signer's
User ID to the user.  This means that the consumer of notmuch's
cryptographic summary of a message (or of its protected headers) can
be confident in relaying the reported identity to the user.

However, some versions of GMime before 3.2.7 cannot report Certificate
validity for X.509 certificates.  This is resolved upstream in GMime
at https://github.com/jstedfast/gmime/pull/90.

We adapt to this by marking tests of reported User IDs for
S/MIME-signed messages as known-broken if GMime is older than 3.2.7
and has not been patched.

If GMime >= 3.2.7 and certificate validity still doesn't work for
X.509 certs, then there has likely been a regression in GMime and we
should fail early, during ./configure.

To break out these specific User ID checks from other checks, i had to
split some tests into two parts, and reuse $output across the two
subtests.

Signed-off-by: Daniel Kahn Gillmor 
---
 configure  | 79 ++
 test/T355-smime.sh | 17 +---
 test/T356-protected-headers.sh | 13 +-
 3 files changed, 100 insertions(+), 9 deletions(-)

diff --git a/configure b/configure
index 0cfdaa6f..92e5bd1b 100755
--- a/configure
+++ b/configure
@@ -536,6 +536,82 @@ EOF
 if [ -n "$TEMP_GPG" -a -d "$TEMP_GPG" ]; then
 rm -rf "$TEMP_GPG"
 fi
+
+# see https://github.com/jstedfast/gmime/pull/90
+# should be fixed in GMime in 3.2.7, but some distros might patch
+printf "Checking for GMime X.509 certificate validity... "
+
+cat > _check_x509_validity.c <
+#include 
+
+int main () {
+GError *error = NULL;
+GMimeParser *parser = NULL;
+GMimeApplicationPkcs7Mime *body = NULL;
+GMimeSignatureList *sig_list = NULL;
+GMimeSignature *sig = NULL;
+GMimeCertificate *cert = NULL;
+GMimeObject *output = NULL;
+GMimeValidity validity = GMIME_VALIDITY_UNKNOWN;
+int len;
+
+g_mime_init ();
+parser = g_mime_parser_new ();
+g_mime_parser_init_with_stream (parser, 
g_mime_stream_file_open("test/corpora/pkcs7/smime-onepart-signed.eml", "r", 
));
+if (error) return !! fprintf (stderr, "failed to instantiate parser with 
test/corpora/pkcs7/smime-onepart-signed.eml\n");
+
+body = GMIME_APPLICATION_PKCS7_MIME(g_mime_message_get_mime_part 
(g_mime_parser_construct_message (parser, NULL)));
+if (body == NULL) return !!fprintf (stderr, "did not find a 
application/pkcs7 message\n");
+
+sig_list = g_mime_application_pkcs7_mime_verify (body, GMIME_VERIFY_NONE, 
, );
+if (error || output == NULL) return !! fprintf (stderr, "verify failed\n");
+
+if (sig_list == NULL) return !! fprintf (stderr, "no GMimeSignatureList 
found\n");
+len = g_mime_signature_list_length (sig_list);
+if (len != 1) return !! fprintf (stderr, "expected 1 signature, got %d\n", 
len);
+sig = g_mime_signature_list_get_signature (sig_list, 0);
+if (sig == NULL) return !! fprintf (stderr, "no GMimeSignature found at 
position 0\n");
+cert = g_mime_signature_get_certificate (sig);
+if (cert == NULL) return !! fprintf (stderr, "no GMimeCertificate 
found\n");
+validity = g_mime_certificate_get_id_validity (cert);
+if (validity != GMIME_VALIDITY_FULL) return !! fprintf (stderr, "Got 
validity %d, expected %d\n", validity, GMIME_VALIDITY_FULL);
+
+return 0;
+}
+EOF
+if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XX"); then
+printf 'No.\nCould not make tempdir for testing X.509 certificate 
validity support.\n'
+errors=$((errors + 1))
+elif ${CC} ${CFLAGS} ${gmime_cflags} _check_x509_validity.c 
${gmime_ldflags} -o _check_x509_validity \
+&& echo disable-crl-checks > "$TEMP_GPG/gpgsm.conf" \
+&& echo 
"4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$TEMP_GPG/trustlist.txt" \
+&& GNUPGHOME=${TEMP_GPG} gpgsm --batch --quiet --import < 
"$srcdir"/test/smime/ca.crt
+then
+if GNUPGHOME=${TEMP_GPG} ./_check_x509_validity; then
+gmime_x509_cert_validity=1
+printf "Yes.\n"
+else
+gmime_x509_cert_validity=0
+printf "No.\n"
+if pkg-config --exists "gmime-3.0 >= 3.2.7"; then
+cat <https://github.com/jstedfast/gmime/pull/90 with
+more details.
+EOF
+errors=$((errors + 1))
+fi
+fi
+else
+printf 'No.\nFailed to set up gpgsm for testing X.509 certificate 
validity support.\n'
+  

Re: [PATCH 1/2 v2] test-lib: mark function variables as local

2020-05-12 Thread Daniel Kahn Gillmor
On Sat 2020-05-09 08:47:10 -0300, David Bremner wrote:
> I'm confused about where to apply 2/2. If I apply it on top of (updated)
> master, it causes test failures. If I apply after the rest of the
> patches in this thread then presumably there is some interval where the
> build is broken (if only for certain GMime versions).

fwiw, i believe that the current master
( 627460d76b95a07084c2b6fc7f647a5547e1 ) *is* broken on systems with
older gmime.

In particular, on debian buster, i see:

T356-protected-headers: Testing Message decryption with protected headers
 FAIL   verify signed PKCS#7 subject (multipart-signed)
sig_uid: object not found: 
data[0][0][0]["crypto"]["signed"]["status"][0]["userid"]

My [PATCH v2/2 v2] in this thread (sent shortly) should resolve this
situation.  Then, i'll send a rebased version of the full S/MIME series
in a new thread.

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


test suite: FIXED messages are misordered with tests

2020-05-12 Thread Daniel Kahn Gillmor
I'm debugging/diagnosing/trying to clean up some "FIXED" known-broken
tests right now.

Sometimes, depending on circumstances i can't predict (race
conditions?), I see funny output like:

~~~
Use "make V=1" to see the details for passing and known broken tests.
INFO: using 2m timeout for tests
INFO: running tests with moreutils parallel
 FIXED  verify signed PKCS#7 subject (onepart-signed) signer User ID

T356-protected-headers: Testing Message decryption with protected headers
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc)
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc) signer User ID
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp)
 BROKEN confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp) 
signer User ID
 BROKEN confirm encryption-protected PKCS#7 subject (enc+legacy-disp)
~~~


Clearly, that FIXED should come *after* the "T356-protected-headers:"
separator.

This is a minor bug, i suppose, but i confess i don't understand the
maze of shell functions in test-lib.sh well enough to see why this is
happening, let alone to fix it.

Anyone interested in fixing it should be able to do so by marking a good
test "known broken" and then re-running the test suite.  The above
output is taken from:

make -j4 check NOTMUCH_TESTS=T356-protected-headers.sh

Sorry to send a bug report with no fixes!

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] configure: report GMime minimum version in ./configure output

2020-05-12 Thread Daniel Kahn Gillmor
We already report the minimum version for Glib, zlib, and Xapian
development libraries.  For consistency, report it for GMime as well.

Signed-off-by: Daniel Kahn Gillmor 
---
 configure | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure b/configure
index 92e5bd1b..a67b4384 100755
--- a/configure
+++ b/configure
@@ -468,7 +468,7 @@ fi
 
 GMIME_MINVER=3.0.3
 
-printf "Checking for GMime development files... "
+printf "Checking for GMime development files (>= $GMIME_MINVER)... "
 if pkg-config --exists "gmime-3.0 >= $GMIME_MINVER"; then
 printf "Yes.\n"
 have_gmime=1
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 1/2 v2] test-lib: mark function variables as local

2020-05-10 Thread Daniel Kahn Gillmor
On Sat 2020-05-09 08:47:10 -0300, David Bremner wrote:
> I'm confused about where to apply 2/2. If I apply it on top of (updated)
> master, it causes test failures. If I apply after the rest of the
> patches in this thread then presumably there is some interval where the
> build is broken (if only for certain GMime versions).

yes, that's right.  For certain GMime versions, there's an intermediate
breakage.  I think you're saying you want me to rerun the series, with
these things split out so that even on systems with older GMime, tests
pass at every commit.  I'll try to get that done today.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/2 v2] test-lib: mark function variables as local

2020-05-08 Thread Daniel Kahn Gillmor
Several functions in test/test-lib.sh used variable names that are
also used outside of those functions (e.g. $output and $expected are
used in many of the test scripts), but they are not expected to
communicate via those variables.

We mark those variables "local" within test-lib.sh so that they do not
get clobbered when used outside test-lib.

We also move the local variable declarations to beginning of each
function, to avoid weird gotchas with local variable declarations as
described in https://tldp.org/LDP/abs/html/localvar.html.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 31 ++-
 1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 5c8eab7c..58972339 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -109,7 +109,6 @@ unset ALTERNATE_EDITOR
 
 add_gnupg_home ()
 {
-local output
 [ -e "${GNUPGHOME}/gpg.conf" ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
@@ -345,13 +344,14 @@ trap 'trap_signal' HUP INT TERM
 # to the message and encrypting/signing.
 emacs_deliver_message ()
 {
-local subject="$1"
-local body="$2"
+local subject body smtp_dummy_pid smtp_dummy_port
+subject="$1"
+body="$2"
 shift 2
 # before we can send a message, we have to prepare the FCC maildir
 mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp}
 # eval'ing smtp-dummy --background will set smtp_dummy_pid and -_port
-local smtp_dummy_pid= smtp_dummy_port=
+smtp_dummy_pid= smtp_dummy_port=
 eval `$TEST_DIRECTORY/smtp-dummy --background sent_message`
 test -n "$smtp_dummy_pid" || return 1
 test -n "$smtp_dummy_port" || return 1
@@ -391,13 +391,14 @@ emacs_deliver_message ()
 # new" after message delivery
 emacs_fcc_message ()
 {
-local nmn_args=''
+local nmn_args subject body
+nmn_args=''
 while [[ "$1" =~ ^-- ]]; do
 nmn_args="$nmn_args $1"
 shift
 done
-local subject="$1"
-local body="$2"
+subject="$1"
+body="$2"
 shift 2
 # before we can send a message, we have to prepare the FCC maildir
 mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp}
@@ -427,6 +428,7 @@ emacs_fcc_message ()
 # number of messages.
 add_email_corpus ()
 {
+local corpus
 corpus=${1:-default}
 
 rm -rf ${MAIL_DIR}
@@ -457,6 +459,7 @@ test_begin_subtest ()
 # name.
 test_expect_equal ()
 {
+   local output expected testname
exec 1>&6 2>&7  # Restore stdout and stderr
if [ -z "$inside_subtest" ]; then
error "bug in the test script: test_expect_equal without 
test_begin_subtest"
@@ -483,6 +486,7 @@ test_expect_equal ()
 # Like test_expect_equal, but takes two filenames.
 test_expect_equal_file ()
 {
+   local file1 file2 testname basename1 basename2
exec 1>&6 2>&7  # Restore stdout and stderr
if [ -z "$inside_subtest" ]; then
error "bug in the test script: test_expect_equal_file without 
test_begin_subtest"
@@ -512,10 +516,11 @@ test_expect_equal_file ()
 # canonicalized before diff'ing.  If an argument cannot be parsed, it
 # is used unchanged so that there's something to diff against.
 test_expect_equal_json () {
+local script output expected
 # The test suite forces LC_ALL=C, but this causes Python 3 to
 # decode stdin as ASCII.  We need to read JSON in UTF-8, so
 # override Python's stdio encoding defaults.
-local script='import json, sys; json.dump(json.load(sys.stdin), 
sys.stdout, sort_keys=True, indent=4)'
+script='import json, sys; json.dump(json.load(sys.stdin), sys.stdout, 
sort_keys=True, indent=4)'
 output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "$script" \
 || echo "$1")
 expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "$script" 
\
@@ -540,6 +545,7 @@ test_sort_json () {
 # read the source of test/json_check_nodes.py (or the output when
 # invoking it without arguments) for an explanation of the syntax.
 test_json_nodes () {
+local output
 exec 1>&6 2>&7 # Restore stdout and stderr
if [ -z "$inside_subtest" ]; then
error "bug in the test script: test_json_eval without 
test_begin_subtest"
@@ -561,6 +567,7 @@ test_json_nodes () {
 }
 
 test_emacs_expect_t () {
+   local result
test "$#" = 1 ||
error "bug in the test script: not 1 parameter to test_emacs_expect_t"
if [ -z "$inside_subtest" ]; then
@@ -653,7 +660,8 @@ notmuch_json_show_sanitize ()
 
 notmuch_emacs_error_sanitize ()
 {
-l

Re: [PATCH 1/2] test-lib: mark function variables as local

2020-05-08 Thread Daniel Kahn Gillmor
On Thu 2020-05-07 10:31:38 +0300, Tomi Ollila wrote:
> Good stuff
>
> robustness comment IMO:
>
> There is slight difference when writing
>
> local foo=`false`
>
> and
>
> local foo; foo=`false`
>
>
> former does not "fail"; latter does,

thanks for pointing this out.  On IRC, jindraj pointed me to
http://tldp.org/LDP/abs/html/localvar.html   this is an unusual and
confusing set of behaviors i had not understood about how local
interacts with return codes, etc.

I'll send around a revised set of patches that uses "local foo" instead
of "local foo=".

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] notmuch(1): clarify documentation about --option/value separators

2020-05-08 Thread Daniel Kahn Gillmor
On Thu 2020-05-07 16:40:26 -0700, Carl Worth wrote:
> On Thu, May 07 2020, Daniel Kahn Gillmor wrote:
>> +separator. Except for boolean options (wihch would be ambiguous), a
>
> Misspelling of "which". And while I'm here, strictly speaking Boolean is
> generally capitalized in English, (being one of those adjectives that is
> derived from a proper noun).
>
> Otherwise, this looks like a good improvement to me. Thanks!

Thanks for catching the "which" misspelling, which should obviously be
fixed if/when the patch is applied.  notmuch-search-terms(7) contains
both capitalized and non-capitalized spellings of "boolean".  I welcome
capitalization cleanup for this over the whole codebase, and hope that a
decision about this particular nit-pick isn't cause for delay of an
otherwise uncontroversial improvement.

  --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] notmuch(1): clarify documentation about --option/value separators

2020-05-07 Thread Daniel Kahn Gillmor
id:CA+Tk8fzRiqxWpd=r8=DRvEewNZXUZgD7MKyRLB1A=r-lxxg...@mail.gmail.com
started a thread of discussion that showed that the cli's current
idiosyncrasies around dealing with boolean options were not
understandable.

This attempts to improve the documentation at least (actual changes to
the API might be better, but have not reached consensus).

Note that no one in the discussion thread identified any other
(non-boolean) command-line options that cannot use space as a
separator.  If such an option is identified (or introduced in the
future), it should be added explicitly to this part of the manual.

Signed-off-by: Daniel Kahn Gillmor 
---
 doc/man1/notmuch.rst | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/doc/man1/notmuch.rst b/doc/man1/notmuch.rst
index d2cd8da5..3dcc646c 100644
--- a/doc/man1/notmuch.rst
+++ b/doc/man1/notmuch.rst
@@ -128,9 +128,9 @@ OPTION SYNTAX
 -
 
 All options accepting an argument can be used with '=' or ':' as a
-separator. For the cases where it's not ambiguous (in particular
-excluding boolean options), a space can also be used. The following
-are all equivalent:
+separator. Except for boolean options (wihch would be ambiguous), a
+space can also be used as a separator. The following are all
+equivalent:
 
 ::
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2] smime: tests of X.509 certificate validity are known-broken on GMime < 3.2.7

2020-05-06 Thread Daniel Kahn Gillmor
When checking cryptographic signatures, Notmuch relies on GMime to
tell it whether the certificate that signs a message has a valid User
ID or not.

If the User ID is not valid, then notmuch does not report the signer's
User ID to the user.  This means that the consumer of notmuch's
cryptographic summary of a message (or of its protected headers) can
be confident in relaying the reported identity to the user.

However, some versions of GMime before 3.2.7 cannot report Certificate
validity for X.509 certificates.  This is resolved upstream in GMime
at https://github.com/jstedfast/gmime/pull/90.

We adapt to this by marking tests of reported User IDs for
S/MIME-signed messages as known-broken if GMime is older than 3.2.7
and has not been patched.

If GMime >= 3.2.7 and certificate validity still doesn't work for
X.509 certs, then there has likely been a regression in GMime and we
should fail early, during ./configure.

To break out these specific User ID checks from other checks, i had to
split some tests into two parts, and reuse $output across the two
subtests.

Signed-off-by: Daniel Kahn Gillmor 
---
 configure  | 79 ++
 test/T355-smime.sh | 19 +---
 test/T356-protected-headers.sh | 15 ++-
 3 files changed, 104 insertions(+), 9 deletions(-)

diff --git a/configure b/configure
index 0cfdaa6f..92e5bd1b 100755
--- a/configure
+++ b/configure
@@ -536,6 +536,82 @@ EOF
 if [ -n "$TEMP_GPG" -a -d "$TEMP_GPG" ]; then
 rm -rf "$TEMP_GPG"
 fi
+
+# see https://github.com/jstedfast/gmime/pull/90
+# should be fixed in GMime in 3.2.7, but some distros might patch
+printf "Checking for GMime X.509 certificate validity... "
+
+cat > _check_x509_validity.c <
+#include 
+
+int main () {
+GError *error = NULL;
+GMimeParser *parser = NULL;
+GMimeApplicationPkcs7Mime *body = NULL;
+GMimeSignatureList *sig_list = NULL;
+GMimeSignature *sig = NULL;
+GMimeCertificate *cert = NULL;
+GMimeObject *output = NULL;
+GMimeValidity validity = GMIME_VALIDITY_UNKNOWN;
+int len;
+
+g_mime_init ();
+parser = g_mime_parser_new ();
+g_mime_parser_init_with_stream (parser, 
g_mime_stream_file_open("test/corpora/pkcs7/smime-onepart-signed.eml", "r", 
));
+if (error) return !! fprintf (stderr, "failed to instantiate parser with 
test/corpora/pkcs7/smime-onepart-signed.eml\n");
+
+body = GMIME_APPLICATION_PKCS7_MIME(g_mime_message_get_mime_part 
(g_mime_parser_construct_message (parser, NULL)));
+if (body == NULL) return !!fprintf (stderr, "did not find a 
application/pkcs7 message\n");
+
+sig_list = g_mime_application_pkcs7_mime_verify (body, GMIME_VERIFY_NONE, 
, );
+if (error || output == NULL) return !! fprintf (stderr, "verify failed\n");
+
+if (sig_list == NULL) return !! fprintf (stderr, "no GMimeSignatureList 
found\n");
+len = g_mime_signature_list_length (sig_list);
+if (len != 1) return !! fprintf (stderr, "expected 1 signature, got %d\n", 
len);
+sig = g_mime_signature_list_get_signature (sig_list, 0);
+if (sig == NULL) return !! fprintf (stderr, "no GMimeSignature found at 
position 0\n");
+cert = g_mime_signature_get_certificate (sig);
+if (cert == NULL) return !! fprintf (stderr, "no GMimeCertificate 
found\n");
+validity = g_mime_certificate_get_id_validity (cert);
+if (validity != GMIME_VALIDITY_FULL) return !! fprintf (stderr, "Got 
validity %d, expected %d\n", validity, GMIME_VALIDITY_FULL);
+
+return 0;
+}
+EOF
+if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XX"); then
+printf 'No.\nCould not make tempdir for testing X.509 certificate 
validity support.\n'
+errors=$((errors + 1))
+elif ${CC} ${CFLAGS} ${gmime_cflags} _check_x509_validity.c 
${gmime_ldflags} -o _check_x509_validity \
+&& echo disable-crl-checks > "$TEMP_GPG/gpgsm.conf" \
+&& echo 
"4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$TEMP_GPG/trustlist.txt" \
+&& GNUPGHOME=${TEMP_GPG} gpgsm --batch --quiet --import < 
"$srcdir"/test/smime/ca.crt
+then
+if GNUPGHOME=${TEMP_GPG} ./_check_x509_validity; then
+gmime_x509_cert_validity=1
+printf "Yes.\n"
+else
+gmime_x509_cert_validity=0
+printf "No.\n"
+if pkg-config --exists "gmime-3.0 >= 3.2.7"; then
+cat <https://github.com/jstedfast/gmime/pull/90 with
+more details.
+EOF
+errors=$((errors + 1))
+fi
+fi
+else
+printf 'No.\nFailed to set up gpgsm for testing X.509 certificate 
validity support.\n'
+  

[PATCH 1/2] test-lib: mark function variables as local

2020-05-06 Thread Daniel Kahn Gillmor
Several functions in test/test-lib.sh used variable names that are
also used outside of those functions (e.g. $output and $expected are
used in many of the test scripts), but they are not expected to
communicate via those variables.

We mark those variables "local" within test-lib.sh so that they do not
get clobbered when used outside test-lib.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 44 ++--
 1 file changed, 22 insertions(+), 22 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 5c8eab7c..e8feab3b 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -109,7 +109,6 @@ unset ALTERNATE_EDITOR
 
 add_gnupg_home ()
 {
-local output
 [ -e "${GNUPGHOME}/gpg.conf" ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
@@ -427,7 +426,7 @@ emacs_fcc_message ()
 # number of messages.
 add_email_corpus ()
 {
-corpus=${1:-default}
+local corpus=${1:-default}
 
 rm -rf ${MAIL_DIR}
 cp -a $NOTMUCH_SRCDIR/test/corpora/$corpus ${MAIL_DIR}
@@ -465,14 +464,14 @@ test_expect_equal ()
test "$#" = 2 ||
error "bug in the test script: not 2 parameters to test_expect_equal"
 
-   output="$1"
-   expected="$2"
+   local output="$1"
+   local expected="$2"
if ! test_skip "$test_subtest_name"
then
if [ "$output" = "$expected" ]; then
test_ok_
else
-   testname=$this_test.$test_count
+   local testname=$this_test.$test_count
echo "$expected" > $testname.expected
echo "$output" > $testname.output
test_failure_ "$(diff -u $testname.expected 
$testname.output)"
@@ -491,16 +490,16 @@ test_expect_equal_file ()
test "$#" = 2 ||
error "bug in the test script: not 2 parameters to 
test_expect_equal_file"
 
-   file1="$1"
-   file2="$2"
+   local file1="$1"
+   local file2="$2"
if ! test_skip "$test_subtest_name"
then
if diff -q "$file1" "$file2" >/dev/null ; then
test_ok_
else
-   testname=$this_test.$test_count
-   basename1=`basename "$file1"`
-   basename2=`basename "$file2"`
+   local testname=$this_test.$test_count
+   local basename1=`basename "$file1"`
+   local basename2=`basename "$file2"`
cp "$file1" "$testname.$basename1"
cp "$file2" "$testname.$basename2"
test_failure_ "$(diff -u "$testname.$basename1" 
"$testname.$basename2")"
@@ -516,9 +515,9 @@ test_expect_equal_json () {
 # decode stdin as ASCII.  We need to read JSON in UTF-8, so
 # override Python's stdio encoding defaults.
 local script='import json, sys; json.dump(json.load(sys.stdin), 
sys.stdout, sort_keys=True, indent=4)'
-output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "$script" \
+local output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c 
"$script" \
 || echo "$1")
-expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "$script" 
\
+local expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c 
"$script" \
 || echo "$2")
 shift 2
 test_expect_equal "$output" "$expected" "$@"
@@ -540,6 +539,7 @@ test_sort_json () {
 # read the source of test/json_check_nodes.py (or the output when
 # invoking it without arguments) for an explanation of the syntax.
 test_json_nodes () {
+local output
 exec 1>&6 2>&7 # Restore stdout and stderr
if [ -z "$inside_subtest" ]; then
error "bug in the test script: test_json_eval without 
test_begin_subtest"
@@ -577,7 +577,7 @@ test_emacs_expect_t () {
inside_subtest=
 
# Report success/failure.
-   result=$(cat OUTPUT)
+   local result=$(cat OUTPUT)
if [ "$result" = t ]
then
test_ok_
@@ -717,7 +717,7 @@ declare -A test_subtest_missing_external_prereq_
 
 # declare prerequisite for the given external binary
 test_declare_external_prereq () {
-   binary="$1"
+   local binary=

Re: Handle PKCS#7 S/MIME messages

2020-05-04 Thread Daniel Kahn Gillmor
Hi Tomi--

On Sat 2020-05-02 00:15:57 +0300, Tomi Ollila wrote:
> I did not see anything suspicious in code, but
>
> I got these test failures:
>
> in ubuntu 19.10 native environment, and
>
> in debian 10 (podman) container running in fedora 31 system
>
>
> T355-smime: Testing S/MIME signature verification and decryption
>  FAIL   Verify signature on PKCS#7 SignedData message
>  crypto: value not equal: data[0][0][0]["crypto"]["signed"]["status"][0] =
>  {'status': 'good', 
>   'fingerprint': '702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB', 
>   'created': 1574813489,
>   'expires': 2611032858} != 
>  {'created': 1574813489, 
>   'expires': 2611032858,
>   'fingerprint': '702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB', 
>   'userid': 'CN=Alice Lovelace', 
>   'status': 'good'}
>
> T356-protected-headers: Testing Message decryption with protected headers
>  FAIL   verify signed PKCS#7 subject (multipart-signed)
>  sig_uid: object not found:  
> data[0][0][0]["crypto"]["signed"]["status"][0]["userid"]
>  FAIL   verify signed PKCS#7 subject (onepart-signed)
>  sig_uid: object not found: 
> data[0][0][0]["crypto"]["signed"]["status"][0]["userid"]
>  FAIL   confirm signed and encrypted PKCS#7 subject (sign+enc)
>  sig_uid: object not found: 
> data[0][0][0]["crypto"]["signed"]["status"][0]["userid"]
>  FAIL   confirm signed and encrypted PKCS#7 subject (sign+enc+legacy-disp)
>  sig_uid: object not found: 
> data[0][0][0]["crypto"]["signed"]["status"][0]["userid"]

Thanks for identifying these.  These are problems related to a bug in
the released version of GMime on those platforms.  Unfixed versions of
gmime cannot report *any* certificate validity for X.509 certificates:

   https://github.com/jstedfast/gmime/pull/90

The fix for gmime is pretty simple, but it's not something we can
address directly in notmuch.

The fix was first released in GMime version 3.2.7, but it was first in
debian in gmime 3.2.6-2, and should be relatively easy to backport for
any distro that wants it (i suppose i could probably get it into the
next point release for debian 10 as well, since it is a bugfix for an
already-exposed API).

So, how should we deal with this in notmuch?  It seems a bit silly to
bump our required version of gmime to the (relatively new) version
3.2.7, for a fix for a cornercase of a novel use case.

Maybe the test suite should change based on version of GMime?  That
would cause problems for distros that backport the GMime fix, though.

I guess i could write a reproducer for the gmime issue and we could
include it in ./configure, and modify the test suite on that basis.

Any other suggestions?

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 7/9] crypto: handle PKCS#7 envelopedData in _notmuch_crypto_decrypt

2020-04-30 Thread Daniel Kahn Gillmor
In the two places where _notmuch_crypto_decrypt handles
multipart/encrypted messages (PGP/MIME), we should also handle PKCS#7
envelopedData (S/MIME).

This is insufficient for fully handling S/MIME encrypted data because
_notmuch_crypto_decrypt isn't yet actually invoked for envelopedData
parts, but that will happen in the following changes.

Signed-off-by: Daniel Kahn Gillmor 
---
 util/crypto.c | 32 ++--
 1 file changed, 26 insertions(+), 6 deletions(-)

diff --git a/util/crypto.c b/util/crypto.c
index fbd5f011..c09f467b 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -55,10 +55,21 @@ _notmuch_crypto_decrypt (bool *attempted,
}
if (attempted)
*attempted = true;
-   ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part),
- GMIME_DECRYPT_NONE,
- 
notmuch_message_properties_value (list),
- decrypt_result, err);
+   if (GMIME_IS_MULTIPART_ENCRYPTED (part)) {
+   ret = g_mime_multipart_encrypted_decrypt 
(GMIME_MULTIPART_ENCRYPTED (part),
+ GMIME_DECRYPT_NONE,
+ 
notmuch_message_properties_value (list),
+ decrypt_result, err);
+   } else if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) {
+   GMimeApplicationPkcs7Mime *pkcs7 = GMIME_APPLICATION_PKCS7_MIME 
(part);
+   GMimeSecureMimeType type = 
g_mime_application_pkcs7_mime_get_smime_type (pkcs7);
+   if (type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
+   ret = g_mime_application_pkcs7_mime_decrypt (pkcs7,
+
GMIME_DECRYPT_NONE,
+
notmuch_message_properties_value (list),
+
decrypt_result, err);
+   }
+   }
if (ret)
break;
}
@@ -81,8 +92,17 @@ _notmuch_crypto_decrypt (bool *attempted,
 GMimeDecryptFlags flags = GMIME_DECRYPT_NONE;
 if (decrypt == NOTMUCH_DECRYPT_TRUE && decrypt_result)
flags |= GMIME_DECRYPT_EXPORT_SESSION_KEY;
-ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part), flags, NULL,
- decrypt_result, err);
+if (GMIME_IS_MULTIPART_ENCRYPTED (part)) {
+   ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part), flags, NULL,
+ decrypt_result, err);
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) {
+   GMimeApplicationPkcs7Mime *pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part);
+   GMimeSecureMimeType p7type = 
g_mime_application_pkcs7_mime_get_smime_type (pkcs7);
+   if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
+   ret = g_mime_application_pkcs7_mime_decrypt (pkcs7, flags, NULL,
+decrypt_result, err);
+   }
+}
 return ret;
 }
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 3/9] cli: include wrapped part of PKCS#7 SignedData in the MIME tree

2020-04-30 Thread Daniel Kahn Gillmor
Unwrap a PKCS#7 SignedData part unconditionally when the cli is
traversing the MIME tree, and return it as a "child" of what would
otherwise be a leaf in the tree.

Unfortunately, this also breaks the JSON output.  We will fix that
next.

Signed-off-by: Daniel Kahn Gillmor 
---
 mime-node.c| 23 +--
 test/T355-smime.sh |  2 +-
 2 files changed, 22 insertions(+), 3 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index ff6805bf..b6431e3b 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -220,8 +220,17 @@ node_verify (mime_node_t *node, GMimeObject *part)
 notmuch_status_t status;
 
 node->verify_attempted = true;
-node->sig_list = g_mime_multipart_signed_verify (
-   GMIME_MULTIPART_SIGNED (part), GMIME_VERIFY_NONE, );
+if (GMIME_IS_APPLICATION_PKCS7_MIME (part))
+   node->sig_list = g_mime_application_pkcs7_mime_verify (
+   GMIME_APPLICATION_PKCS7_MIME (part), GMIME_VERIFY_NONE, 
>unwrapped_child, );
+else
+   node->sig_list = g_mime_multipart_signed_verify (
+   GMIME_MULTIPART_SIGNED (part), GMIME_VERIFY_NONE, );
+
+if (node->unwrapped_child) {
+   node->nchildren = 1;
+   set_unwrapped_child_destructor (node);
+}
 
 if (node->sig_list)
set_signature_list_destructor (node);
@@ -376,6 +385,12 @@ _mime_node_set_up_part (mime_node_t *node, GMimeObject 
*part, int numchild)
} else {
node_verify (node, part);
}
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (part) &&
+  GMIME_SECURE_MIME_TYPE_SIGNED_DATA == 
g_mime_application_pkcs7_mime_get_smime_type (GMIME_APPLICATION_PKCS7_MIME 
(part))) {
+   /* If node->ctx->crypto->verify is false, it would be better
+* to just unwrap (instead of verifying), but
+* https://github.com/jstedfast/gmime/issues/67 */
+   node_verify (node, part);
 } else {
if (_notmuch_message_crypto_potential_payload (node->ctx->msg_crypto, 
part, node->parent ? node->parent->part : NULL, numchild) &&
node->ctx->msg_crypto->decryption_status == 
NOTMUCH_MESSAGE_DECRYPTED_FULL) {
@@ -409,6 +424,10 @@ mime_node_child (mime_node_t *parent, int child)
GMIME_MULTIPART (parent->part), child);
 } else if (GMIME_IS_MESSAGE (parent->part)) {
sub = g_mime_message_get_mime_part (GMIME_MESSAGE (parent->part));
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (parent->part) &&
+  parent->unwrapped_child &&
+  child == 0) {
+   sub = parent->unwrapped_child;
 } else {
/* This should have been caught by _mime_node_set_up_part */
INTERNAL_ERROR ("Unexpected GMimeObject type: %s",
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 0d78f262..710e51ec 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -142,7 +142,6 @@ expected='#notmuch-dump batch-tag:3 config,properties,tags
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "show contents of PKCS#7 SignedData message"
-test_subtest_known_broken
 output=$(notmuch show --format=raw --part=2 
id:smime-onepart-signed@protected-headers.example)
 whitespace=' '
 expected="Bob, we need to cancel this contract.
@@ -178,6 +177,7 @@ On Tue, 26 Nov 2019 20:11:29 -0400, Alice Lovelace 
 wrote:
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "show PKCS#7 SignedData outputs valid JSON"
+test_subtest_known_broken
 output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
 test_valid_json "$output"
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 6/9] crypto: Make _notmuch_crypto_decrypt take a GMimeObject

2020-04-30 Thread Daniel Kahn Gillmor
As we prepare to handle S/MIME-encrypted PKCS#7 EnvelopedData (which
is not multipart), we don't want to be limited to passing only
GMimeMultipartEncrypted MIME parts to _notmuch_crypto_decrypt.

There is no functional change here, just a matter of adjusting how we
pass arguments internally.

Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc  | 8 
 mime-node.c   | 3 +--
 util/crypto.c | 6 +++---
 util/crypto.h | 2 +-
 4 files changed, 9 insertions(+), 10 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index f029b334..da9a3abe 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -369,7 +369,7 @@ _index_content_type (notmuch_message_t *message, 
GMimeObject *part)
 
 static void
 _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t 
*indexopts,
-   GMimeMultipartEncrypted *part,
+   GMimeObject *part,
_notmuch_message_crypto_t *msg_crypto);
 
 static void
@@ -439,7 +439,7 @@ _index_mime_part (notmuch_message_t *message,
 g_mime_multipart_get_part (multipart, i));
if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) {
_index_encrypted_mime_part (message, indexopts,
-   GMIME_MULTIPART_ENCRYPTED 
(part),
+   part,
msg_crypto);
} else {
if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) {
@@ -551,7 +551,7 @@ _index_mime_part (notmuch_message_t *message,
 static void
 _index_encrypted_mime_part (notmuch_message_t *message,
notmuch_indexopts_t *indexopts,
-   GMimeMultipartEncrypted *encrypted_data,
+   GMimeObject *encrypted_data,
_notmuch_message_crypto_t *msg_crypto)
 {
 notmuch_status_t status;
@@ -603,7 +603,7 @@ _index_encrypted_mime_part (notmuch_message_t *message,
g_object_unref (decrypt_result);
 }
 GMimeObject *toindex = clear;
-if (_notmuch_message_crypto_potential_payload (msg_crypto, clear, 
GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT) &&
+if (_notmuch_message_crypto_potential_payload (msg_crypto, clear, 
encrypted_data, GMIME_MULTIPART_ENCRYPTED_CONTENT) &&
msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) {
toindex = _notmuch_repair_crypto_payload_skip_legacy_display (clear);
if (toindex != clear)
diff --git a/mime-node.c b/mime-node.c
index b6431e3b..c2ee858d 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -253,7 +253,6 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part)
 GError *err = NULL;
 GMimeDecryptResult *decrypt_result = NULL;
 notmuch_status_t status;
-GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
 notmuch_message_t *message = NULL;
 
 if (! node->unwrapped_child) {
@@ -266,7 +265,7 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part)
node->unwrapped_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
 
node->ctx->crypto->decrypt,
 message,
-encrypteddata, 
_result, );
+part, _result, 
);
if (node->unwrapped_child)
set_unwrapped_child_destructor (node);
 }
diff --git a/util/crypto.c b/util/crypto.c
index 0bb6f526..fbd5f011 100644
--- a/util/crypto.c
+++ b/util/crypto.c
@@ -34,7 +34,7 @@ GMimeObject *
 _notmuch_crypto_decrypt (bool *attempted,
 notmuch_decryption_policy_t decrypt,
 notmuch_message_t *message,
-GMimeMultipartEncrypted *part,
+GMimeObject *part,
 GMimeDecryptResult **decrypt_result,
 GError **err)
 {
@@ -55,7 +55,7 @@ _notmuch_crypto_decrypt (bool *attempted,
}
if (attempted)
*attempted = true;
-   ret = g_mime_multipart_encrypted_decrypt (part,
+   ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED 
(part),
  GMIME_DECRYPT_NONE,
  
notmuch_message_properties_value (list),
  decrypt_result, err);
@@ -81,7 +81,7 @@ _notmuch_crypto_decrypt (bool *attempted,
 GMimeDecryptFlags flags = GMIME_DECRYPT_NONE;
 if (decrypt == NOTMUCH_DECRYPT_TRUE && decrypt_result)
flags |= GMIME_DECRYPT_EXPORT_SESSION_KEY;
-ret = g_mime_multipart_encrypted_decrypt (part, flags, NULL,
+ret = g_mime_multip

[PATCH 2/9] smime: Identify encrypted S/MIME parts during indexing

2020-04-30 Thread Daniel Kahn Gillmor
We don't handle them correctly yet, but we can at least mark them as
being encrypted.

Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc   | 4 
 test/T355-smime.sh | 1 -
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/lib/index.cc b/lib/index.cc
index bbf13dc5..f029b334 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -654,6 +654,10 @@ _index_pkcs7_part (notmuch_message_t *message,
notmuch_message_add_property (message, "index.repaired", 
"skip-protected-headers-legacy-display");
}
_index_mime_part (message, indexopts, toindex, msg_crypto);
+} else if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
+   _notmuch_message_add_term (message, "tag", "encrypted");
+   if (notmuch_indexopts_get_decrypt_policy (indexopts) != 
NOTMUCH_DECRYPT_FALSE)
+   _notmuch_database_log (notmuch, "Cannot decrypt PKCS#7 
envelopedData (S/MIME encrypted messages)\n");
 } else {
_notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 
smime-type '%s'\n",
   g_mime_object_get_content_type_parameter (part, 
"smime-type"));
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 01e53e33..0d78f262 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -98,7 +98,6 @@ test_json_nodes <<<"$output" \
 
'crypto_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Notmuch 
Test Suite"'
 
 test_begin_subtest "encrypted+signed message is known to be encrypted, but 
signature is unknown"
-test_subtest_known_broken
 output=$(notmuch search subject:"test encrypted message 001")
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox)"
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 4/9] cli/show: If a leaf part has children, show them instead of omitting

2020-04-30 Thread Daniel Kahn Gillmor
Until we did PKCS#7 unwrapping, no leaf MIME part could have a child.

Now, we treat the unwrapped MIME part as the child of the PKCS#7
SignedData object.  So in that case, we want to show it instead of
deliberately omitting the content.

This fixes the test of the protected subject in
id:smime-onepart-signed@protected-headers.example.

Signed-off-by: Daniel Kahn Gillmor 
---
 notmuch-show.c | 11 ++-
 test/T355-smime.sh |  2 --
 test/T356-protected-headers.sh |  1 -
 3 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index ab1cd144..36265043 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -759,7 +759,16 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, 
mime_node_t *node,
sp->string_len (sp, (char *) part_content->data, part_content->len);
g_object_unref (stream_memory);
} else {
-   format_omitted_part_meta_sprinter (sp, meta, GMIME_PART 
(node->part));
+   /* if we have a child part despite being a standard
+* (non-multipart) MIME part, that means there is
+* something to unwrap, which we will present in
+* content: */
+   if (node->nchildren) {
+   sp->map_key (sp, "content");
+   sp->begin_list (sp);
+   nclose = 1;
+   } else
+   format_omitted_part_meta_sprinter (sp, meta, GMIME_PART 
(node->part));
}
 } else if (GMIME_IS_MULTIPART (node->part)) {
sp->map_key (sp, "content");
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 710e51ec..099a3df7 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -177,12 +177,10 @@ On Tue, 26 Nov 2019 20:11:29 -0400, Alice Lovelace 
 wrote:
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "show PKCS#7 SignedData outputs valid JSON"
-test_subtest_known_broken
 output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
 test_valid_json "$output"
 
 test_begin_subtest "Verify signature on PKCS#7 SignedData message"
-test_subtest_known_broken
 output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
 test_json_nodes <<<"$output" \
 'crypto:[0][0][0]["crypto"]["signed"]["status"][0]={
diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh
index 520cb71c..bca50e29 100755
--- a/test/T356-protected-headers.sh
+++ b/test/T356-protected-headers.sh
@@ -157,7 +157,6 @@ test_expect_equal "$output" 
id:protected-with-legacy-display@crypto.notmuchmail.
 
 for variant in multipart-signed onepart-signed; do
 test_begin_subtest "verify signed PKCS#7 subject ($variant)"
-[ "$variant" = multipart-signed ] || test_subtest_known_broken
 output=$(notmuch show --verify --format=json 
"id:smime-${variant}@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 9/9] smime: Index cleartext of envelopedData when requested

2020-04-30 Thread Daniel Kahn Gillmor
Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc   | 5 +++--
 test/T355-smime.sh | 2 --
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index da9a3abe..826aa341 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -656,8 +656,9 @@ _index_pkcs7_part (notmuch_message_t *message,
_index_mime_part (message, indexopts, toindex, msg_crypto);
 } else if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) {
_notmuch_message_add_term (message, "tag", "encrypted");
-   if (notmuch_indexopts_get_decrypt_policy (indexopts) != 
NOTMUCH_DECRYPT_FALSE)
-   _notmuch_database_log (notmuch, "Cannot decrypt PKCS#7 
envelopedData (S/MIME encrypted messages)\n");
+   _index_encrypted_mime_part (message, indexopts,
+   part,
+   msg_crypto);
 } else {
_notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 
smime-type '%s'\n",
   g_mime_object_get_content_type_parameter (part, 
"smime-type"));
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 0d32a7d5..3573b5ee 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -107,12 +107,10 @@ test_begin_subtest "Reindex cleartext"
 test_expect_success "notmuch reindex --decrypt=true subject:'test encrypted 
message 001'"
 
 test_begin_subtest "signature is now known"
-test_subtest_known_broken
 output=$(notmuch search subject:"test encrypted message 001")
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
 
 test_begin_subtest "Encrypted body is indexed"
-test_subtest_known_broken
 output=$(notmuch search 'this is a test encrypted message')
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Handle PKCS#7 S/MIME messages

2020-04-30 Thread Daniel Kahn Gillmor
This series applies after the "Add tests for S/MIME PKCS#7 messages"
series, which was introduced in
id:20200428185723.660184-1-...@fifthhorseman.net

With this series applied, notmuch handles standard PKCS#7 S/MIME
messages (using GnuPG's gpgsm as a backend, as mediated by GMime's use
of GPGME) as well as it handles PGP/MIME messages.

In addition to showing and replying to these messages, the series
covers indexing the cleartext of encrypted messages, and understanding
protected headers.

  --dkg


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/9] lib: index PKCS7 SignedData parts

2020-04-30 Thread Daniel Kahn Gillmor
When we are indexing, we should treat SignedData parts the same way
that we treat a multipart object, indexing the wrapped part as a
distinct MIME object.

Unfortunately, this means doing some sort of cryptographic
verification whose results we throw away, because GMime doesn't offer
us any way to unwrap without doing signature verification.

I've opened https://github.com/jstedfast/gmime/issues/67 to request
the capability from GMime but for now, we'll just accept the
additional performance hit.

As we do this indexing, we also apply the "signed" tag, by analogy
with how we handle multipart/signed messages.  These days, that kind
of change should probably be done with a property instead, but that's
a different set of changes.  This one is just for consistency.

Note that we are currently *only* handling signedData parts, which are
basically clearsigned messages.  PKCS#7 parts can also be
envelopedData and authEnvelopedData (which are effectively encryption
layers), and compressedData (which afaict isn't implemented anywhere,
i've never encountered it).  We're laying the groundwork for indexing
these other S/MIME types here, but we're only dealing with signedData
for now.

Signed-off-by: Daniel Kahn Gillmor 
---
 lib/index.cc   | 57 ++
 test/T355-smime.sh |  2 --
 2 files changed, 57 insertions(+), 2 deletions(-)

diff --git a/lib/index.cc b/lib/index.cc
index 158ba5cf..bbf13dc5 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -372,6 +372,12 @@ _index_encrypted_mime_part (notmuch_message_t *message, 
notmuch_indexopts_t *ind
GMimeMultipartEncrypted *part,
_notmuch_message_crypto_t *msg_crypto);
 
+static void
+_index_pkcs7_part (notmuch_message_t *message,
+  notmuch_indexopts_t *indexopts,
+  GMimeObject *part,
+  _notmuch_message_crypto_t *msg_crypto);
+
 /* Callback to generate terms for each mime part of a message. */
 static void
 _index_mime_part (notmuch_message_t *message,
@@ -466,6 +472,11 @@ _index_mime_part (notmuch_message_t *message,
goto DONE;
 }
 
+if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) {
+   _index_pkcs7_part (message, indexopts, part, msg_crypto);
+   goto DONE;
+}
+
 if (! (GMIME_IS_PART (part))) {
_notmuch_database_log (notmuch_message_get_database (message),
   "Warning: Not indexing unknown mime part: %s.\n",
@@ -608,6 +619,52 @@ _index_encrypted_mime_part (notmuch_message_t *message,
 
 }
 
+static void
+_index_pkcs7_part (notmuch_message_t *message,
+  notmuch_indexopts_t *indexopts,
+  GMimeObject *part,
+  _notmuch_message_crypto_t *msg_crypto)
+{
+GMimeApplicationPkcs7Mime *pkcs7;
+GMimeSecureMimeType p7type;
+GMimeObject *mimeobj = NULL;
+GMimeSignatureList *sigs = NULL;
+GError *err = NULL;
+notmuch_database_t *notmuch = NULL;
+
+pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part);
+p7type = g_mime_application_pkcs7_mime_get_smime_type (pkcs7);
+notmuch = notmuch_message_get_database (message);
+_index_content_type (message, part);
+
+if (p7type == GMIME_SECURE_MIME_TYPE_SIGNED_DATA) {
+   sigs = g_mime_application_pkcs7_mime_verify (pkcs7, GMIME_VERIFY_NONE, 
, );
+   if (sigs == NULL) {
+   _notmuch_database_log (notmuch, "Failed to verify PKCS#7 SignedData 
during indexing. (%d:%d) [%s]\n",
+  err->domain, err->code, err->message);
+   g_error_free (err);
+   goto DONE;
+   }
+   _notmuch_message_add_term (message, "tag", "signed");
+   GMimeObject *toindex = mimeobj;
+   if (_notmuch_message_crypto_potential_payload (msg_crypto, mimeobj, 
part, 0) &&
+   msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) {
+   toindex = _notmuch_repair_crypto_payload_skip_legacy_display 
(mimeobj);
+   if (toindex != mimeobj)
+   notmuch_message_add_property (message, "index.repaired", 
"skip-protected-headers-legacy-display");
+   }
+   _index_mime_part (message, indexopts, toindex, msg_crypto);
+} else {
+   _notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 
smime-type '%s'\n",
+  g_mime_object_get_content_type_parameter (part, 
"smime-type"));
+}
+ DONE:
+if (mimeobj)
+   g_object_unref (mimeobj);
+if (sigs)
+   g_object_unref (sigs);
+}
+
 static notmuch_status_t
 _notmuch_message_index_user_headers (notmuch_message_t *message, GMimeMessage 
*mime_message)
 {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 117fa2b9..01e53e33 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -132,13 +132,11 @@ expected=''
 test_expect_equal "$expected" &q

[PATCH 5/9] cli/reply: Ignore PKCS#7 wrapper parts when replying

2020-04-30 Thread Daniel Kahn Gillmor
When composing a reply, no one wants to see this line in the proposed
message:

Non-text part: application/pkcs7-mime

So we hide it, the same way we hide PGP/MIME cruft.

Signed-off-by: Daniel Kahn Gillmor 
---
 notmuch-reply.c| 5 +++--
 test/T355-smime.sh | 1 -
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/notmuch-reply.c b/notmuch-reply.c
index 2c30f6f9..ceb4f39b 100644
--- a/notmuch-reply.c
+++ b/notmuch-reply.c
@@ -65,8 +65,9 @@ format_part_reply (GMimeStream *stream, mime_node_t *node)
GMimeContentDisposition *disposition = 
g_mime_object_get_content_disposition (node->part);
 
if (g_mime_content_type_is_type (content_type, "application", 
"pgp-encrypted") ||
-   g_mime_content_type_is_type (content_type, "application", 
"pgp-signature")) {
-   /* Ignore PGP/MIME cruft parts */
+   g_mime_content_type_is_type (content_type, "application", 
"pgp-signature") ||
+   g_mime_content_type_is_type (content_type, "application", 
"pkcs7-mime")) {
+   /* Ignore PGP/MIME and S/MIME cruft parts */
} else if (g_mime_content_type_is_type (content_type, "text", "*") &&
   ! g_mime_content_type_is_type (content_type, "text", 
"html")) {
show_text_part_content (node->part, stream, 
NOTMUCH_SHOW_TEXT_PART_REPLY);
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 099a3df7..4b67a559 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -156,7 +156,6 @@ OpenPGP Example Corp"
 test_expect_equal "$expected" "$output"
 
 test_begin_subtest "reply to PKCS#7 SignedData message with proper quoting and 
attribution"
-test_subtest_known_broken
 output=$(notmuch reply id:smime-onepart-signed@protected-headers.example)
 expected="From: Notmuch Test Suite 
 Subject: Re: The FooCorp contract
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 8/9] smime: Pass PKCS#7 envelopedData to node_decrypt_and_verify

2020-04-30 Thread Daniel Kahn Gillmor
This change means we can support "notmuch show --decrypt=true" for
S/MIME encrypted messages, resolving several outstanding broken tests,
including all the remaining S/MIME protected header examples.

We do not yet handle indexing the cleartext of S/MIME encrypted
messages, though.

Signed-off-by: Daniel Kahn Gillmor 
---
 mime-node.c| 6 ++
 test/T355-smime.sh | 2 --
 test/T356-protected-headers.sh | 2 --
 3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index c2ee858d..f552e03a 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -390,6 +390,12 @@ _mime_node_set_up_part (mime_node_t *node, GMimeObject 
*part, int numchild)
 * to just unwrap (instead of verifying), but
 * https://github.com/jstedfast/gmime/issues/67 */
node_verify (node, part);
+} else if (GMIME_IS_APPLICATION_PKCS7_MIME (part) &&
+  GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA == 
g_mime_application_pkcs7_mime_get_smime_type (GMIME_APPLICATION_PKCS7_MIME 
(part)) &&
+  (node->ctx->crypto->decrypt != NOTMUCH_DECRYPT_FALSE)) {
+   node_decrypt_and_verify (node, part);
+   if (node->unwrapped_child && node->nchildren == 0)
+   node->nchildren = 1;
 } else {
if (_notmuch_message_crypto_potential_payload (node->ctx->msg_crypto, 
part, node->parent ? node->parent->part : NULL, numchild) &&
node->ctx->msg_crypto->decryption_status == 
NOTMUCH_MESSAGE_DECRYPTED_FULL) {
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 4b67a559..0d32a7d5 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -80,7 +80,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Decryption (notmuch CLI)"
-test_subtest_known_broken
 notmuch show --decrypt=true subject:"test encrypted message 001" |\
 grep "^This is a" > OUTPUT
 cat < EXPECTED
@@ -89,7 +88,6 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "Cryptographic message status (encrypted+signed)"
-test_subtest_known_broken
 output=$(notmuch show --format=json --decrypt=true subject:"test encrypted 
message 001")
 test_json_nodes <<<"$output" \
 
'crypto_encrypted:[0][0][0]["crypto"]["decrypted"]["status"]="full"' \
diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh
index bca50e29..547b0c9a 100755
--- a/test/T356-protected-headers.sh
+++ b/test/T356-protected-headers.sh
@@ -168,7 +168,6 @@ done
 
 for variant in sign+enc sign+enc+legacy-disp; do
 test_begin_subtest "confirm signed and encrypted PKCS#7 subject ($variant)"
-test_subtest_known_broken
 output=$(notmuch show --decrypt=true --format=json 
"id:smime-${variant}@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
@@ -179,7 +178,6 @@ for variant in sign+enc sign+enc+legacy-disp; do
 done
 
 test_begin_subtest "confirm encryption-protected PKCS#7 subject 
(enc+legacy-disp)"
-test_subtest_known_broken
 output=$(notmuch show --decrypt=true --format=json 
"id:smime-enc+legacy-disp@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}'
 \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 07/15 v2] test: Allow tests to have both gpg and gpgsm active at once

2020-04-30 Thread Daniel Kahn Gillmor
Without this fix, we couldn't run both add_gnupg_home and
add_gpgsm_home in the same test script.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index ac1b9315..1baa2d20 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -110,10 +110,10 @@ unset ALTERNATE_EDITOR
 add_gnupg_home ()
 {
 local output
-[ -d ${GNUPGHOME} ] && return
+[ -e "${GNUPGHOME}/gpg.conf" ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
-mkdir -m 0700 "$GNUPGHOME"
+mkdir -p -m 0700 "$GNUPGHOME"
 gpg --no-tty --import <$NOTMUCH_SRCDIR/test/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
 test_debug "cat $GNUPGHOME/import.log"
 if (gpg --quick-random --version >/dev/null 2>&1) ; then
@@ -132,10 +132,10 @@ add_gnupg_home ()
 add_gpgsm_home ()
 {
 local fpr
-[ -d "$GNUPGHOME" ] && return
+[ -e "$GNUPGHOME/gpgsm.conf" ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
-mkdir -m 0700 "$GNUPGHOME"
+mkdir -p -m 0700 "$GNUPGHOME"
 openssl pkcs12 -export -passout pass: -inkey 
"$NOTMUCH_SRCDIR/test/smime/key+cert.pem" \
 < "$NOTMUCH_SRCDIR/test/smime/test.crt" | \
 gpgsm --batch --no-tty --no-common-certs-import 
--pinentry-mode=loopback --passphrase-fd 3 \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 08/15 v2] tests/smime: include secret key material for Bob

2020-04-30 Thread Daniel Kahn Gillmor
This is taken from the same Internet Draft that test/smime/ca.crt
comes from.  See that draft for more details.
https://www.ietf.org/id/draft-dkg-lamps-samples-02.html#name-pkcs12-object-for-bob

We don't use it yet, but it will be used to decrypt other messages in
the test suite.

Note that we include it here with an empty passphrase, rather than
with the passphrase "bob" that it is supplied with in the I-D.  The
underlying cryptographic material is the same, but this way we can
import cleanly into gpgsm without having a passphrase set on it (gpgsm
converts an empty-string passphrase into no passphrase at all on
import).

Signed-off-by: Daniel Kahn Gillmor 
---
 test/smime/bob.p12 | 58 ++
 test/test-lib.sh   |  2 ++
 2 files changed, 60 insertions(+)
 create mode 100644 test/smime/bob.p12

diff --git a/test/smime/bob.p12 b/test/smime/bob.p12
new file mode 100644
index ..774c77d0
--- /dev/null
+++ b/test/smime/bob.p12
@@ -0,0 +1,58 @@
+-BEGIN PKCS12-
+MIIKWAIBAzCCCh4GCSqGSIb3DQEHAaCCCg8EggoLMIIKBzCCBGcGCSqGSIb3DQEH
+BqCCBFgwggRUAgEAMIIETQYJKoZIhvcNAQcBMBwGCiqGSIb3DQEMAQYwDgQICE8J
+3kMad9UCAggAgIIEIPvHjK0eRQrnowMUsz1z1x/IxslNvG6DjPZjNHCkNYYmiRsg
+Leu5nqKf4emWVvYpnlh+4Gql7pyJm3G3zSNhobPkW+P1Eh80tTBoUk7TIvvvmtrE
+YEc/nRR1p1MgjISq4Q/CM6ccCCw6YEiQcj/0mSS7gmHUegD5glcWbVuqAT8M/p1z
+98OP3z37G8ARRLNj1yyp0SVlt59Sx3WNbmYBqkQ96iukjMJvmjV7o6BFYUx46Llb
+tphhdRgKXbK2r1R0TUlvE659TUwlrpGlaFpaGj1kLdzVAnjh1ZWnWO2a2BSj0LzG
+qRyiLwqDFPLJLQEckfV+RPWiRrSewME8URNKdk6eewtHdhrehMo4ZJnOIum8qcSz
+giW61SSyZJsFvILpmMYghIxWmPd/8cNIHBrdFEa7z3QKh5jcJNTCxz6yO9f8F830
+d+WDK7DbGkUW4mVTGg/lEYnCFZDF6S1mr0hx+cew1FbKjLpxfQllIIrLf5d2BF8H
+0STpuylQDVVBFdTRHyeS6td5nulANgOProrRzy3aAKQmZ6iullKl+i2t/2TwfVP/
+gG+yszpOEf8U9txuvbiZ7j4XV158zdaaGiduDqMKLOvbdctwHAsR9ecx5C3NTRDl
+ZlttNoXN9zhT4CkWk1w4sFk2KUurjVraIcjWVT7yOreaaK+6N09M0tnLPDJDTrow
+8WwP/rZhA+t+CMrhqkFBxXsyo5VTM0jWJGO/NLpYXPhDPBsRq8rs1OCrUoVr34aR
+cpUTNhyXkvJUarWDHs88lg0ps0G9/1dXI1AbEsQQg8u+QT2ztGYrg2OQxQyi1Mo4
+u/FkAcEbtlYYLmJjj/S2qVRPJgBALVjw9k5hnYRdAXWVDCJ96PMn1SKORvlMxnZ7
+djlhaztOhTLsiDzywVDYWLvQElunWcAGeDZykWNytwcEagc0VjWKHMibc0JOZQ1T
+crGyOzTlt09xHj1NrItYefIwdtKuJfkAh03B5xI6rJ9ZbK9xidcVxyeRX0lEqdo9
+WHQrhHefAmeyo0TlfsN67kFDp5FLpwEtNaN0lyzpkl30aWZdtP5vkvtfmy5ugYIO
+bXoVa+tO6k5V/VfUFUKdaY7xAX7XRzUUg4jB0D0CuaX+YS+GL+5wuQwIY1y2ihBb
+CuCxlcP1lVEU4CVQba60VTudJtWyE7QpPhf+y81f1wRjwIihFvwzpUFWf8JVEppe
+v3Yot3OWGBmhEqLkC9LELth8o5gLfyYHaXTYNd9aRTiI+0ZC5U3O4wUwYLTG3exM
+rIDTzEMk/p4DYIHkNKVUiRJfGYdAwuRxf3IMcYWARTXlSzl1C3hWmZfvTPlKs1bB
+OHTHP/P+qdOFjxOh+fbyqXPJauBAhHvHgrp3iI6t834wJou26oWNihM7OnWuyQRt
+9DVxG4l+1VjtbQZfTDCCBZgGCSqGSIb3DQEHAaCCBYkEggWFMIIFgTCCBX0GCyqG
+SIb3DQEMCgECoIIE7jCCBOowHAYKKoZIhvcNAQwBAzAOBAjqo0x2p5SqLAICCAAE
+ggTIe6Ws+lu0CoNlCXGM2BEPV09wuRHTJe+KnesrmRbXPF9linG3d6G++tTkBHz/
+yr77/DV5aDYciV1pGAbLuX2lMwuqdxzJ4OBPBAjuX5H+IPRaTbxfHYYIwhG8oZzy
+aHyVhHr9j0h7lzW7xSTYJuBNEJ58L42dfzpNRw9dyRPmcuhZqW14Z3xyDm8yjHfB
+2p99y9/A4qSyJJSUM3O3nLdtIar3ktSTRAijgqq+s9wnsfozQRzWpYaqiRrdzwfO
+HqXk54l3/lMSyLpfPl9LW7er6JbGI4jEyQ3x8WijATM5h/lkZKejh/mOaWCvs6G6
+fGzV4P35EsToYbOk9GX4jl4SyDBt3iEHYm5teDUhJmTcR39lAQuAfxN6rOn/TkoO
+YLxtdD5DLiTfYZPCFyavLEsamr8A4p93torF6Rs7GsaHE6PmCcprzqx71KV0DZKv
+tMY86RoiWPKLFxZcYt1yz9/95c1SO1s4i1GvLpJTEgQxLM2OhfEwDNKd2rMJoq1I
+YIRPSP204dIVwwNdXN1vB2slhN2+/QMOqsEkWtTOpW2QoTGSze49hfmJGdu+91jd
+XZBBMJQfY4q066/eE4IOW7ZZId5uMYxDRnGdEQjJsxyW8YHWLRGQvBC8gMkdbj8e
+0wkXbe+jML7vG7t3hDhLEbj5sTquIMTWrTirPw4SxLCuGZAyJHFN3/nCaOSMFlCG
+wEZHrAozgQXPBYU7p+uIkJ4lDc2ZtW8NM8U15gKZLDFfAE6Vg0jAtfFMqvNnX630
+xfo1z4jBd7VXbBFrPzrmvlTnb1XxNFcPycowzW9tgtN4YnNroCq98VpMC914tdpJ
++C/PI0eJ7M2ir3ajN0RabSm02JO9Hdwoa5OgqLwPYDwiFyQvKFGKqAF8Ph6pSEiZ
+10OnH+DVgEY70A+Le+ZSDosMdrhZfHbCcIFitZJ3sYV/7Q118QckW3szcjmLHS5g
+M6Whl2HhjLsAfsmCnoRlIwjx4g0TiuZcb4hGysq8QjD3Z8qqFK28m6OMHbASQfWg
+U+Qg3vmEvVsnBxStFEIImS3QYQoaT0pk6zKUYsI/fOBnEgxsY0XwTfXzVw7hZDct
+yhNIQVWmfgVZwUw0wLoNu3A5hupjUwQzQr4TPnKkFPI8qHmRrJgP8EA0U0019y3W
+MlK0h/LAJEaUBS0goLJCJ8+1EWr6femjnyuU5hMizOm+3j0JexjWz5TQttioS7Q/
+vcxt5pA9yAWQdH9j72saKEoKmDi+kIPr4mimKJz99LhKp9A6Hj0f1P2V3As8JWyW
+ZKmJKW7qMMCFADlALolobqzA60j6Zeo5jiEj/j2lVlUPPz47WO+uKeb+rx+hgTUc
+Xrhq0+an5tvEXt/8wy3PJFqP+qqHGhOIuPLuhqPyzNowuXirIXsiWnI44/X48W91
+HPEoL3xaebQ6oyTP8dI4CCkkHgiLWL5mskjHMEXvcdR6k0ygmu8DGQCPfUweUZqZ
+wfkhD/jwbVpLR5Y3chpatW0cJ2bsAWdxwtuxF05+fVEePUsR0x+2/v/8eDEHKYwt
+aYlAhI48nyrKKVMmqvqcXnzmJlUaq05GnEcglFbv4MUExL7CxClls6QnVNiZFPrV
+ffVsYT2A300xrm4pan89n3nuavjJn7L1JJdmMXwwVQYJKoZIhvcNAQkUMUgeRgBH
+AG4AdQBQAEcAIABlAHgAcABvAHIAdABlAGQAIABjAGUAcgB0AGkAZgBpAGMAYQB0
+AGUAIAA0ADIAYgBiADIANAAwADYwIwYJKoZIhvcNAQkVMRYEFGaI9k+ZdE9/rxBZ
+4rSdH1BCuyQGMDEwITAJBgUrDgMCGgUABBRJfL4XyIHpXmjbziCGCbSAOK9jKgQI
+drOMeIgXcCYCAggA
+-END PKCS12-
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 1baa2d20..cb5bb894 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -145,6 +145,8 @@ add_gpgsm_home ()
 gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.

[PATCH 03/15 v2] tests/smime: Include the Sample LAMPS Certificate Authority

2020-04-30 Thread Daniel Kahn Gillmor
This CA is useful for test suites and the like, but is not an
actually-secure CA, because its secret key material is also published.

I plan to use it for its intended purpose in the notmuch test suite.

It was copied from this Internet Draft:

https://tools.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi

Signed-off-by: Daniel Kahn Gillmor 
---
 test/smime/README |  2 ++
 test/smime/ca.crt | 20 
 test/test-lib.sh  |  2 ++
 3 files changed, 24 insertions(+)
 create mode 100644 test/smime/ca.crt

diff --git a/test/smime/README b/test/smime/README
index 46211922..6f276398 100644
--- a/test/smime/README
+++ b/test/smime/README
@@ -5,3 +5,5 @@ key+cert.pem: cert + unencryped private
 % gpgsm --import test.crt
 % gpgsm --export-private-key-p12 -out foo.p12  (no passphrase)
 % openssl pkcs12 -in ns.p12 -clcerts -nodes > key+cert.pem
+
+ca.crt: from 
https://tools.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi
diff --git a/test/smime/ca.crt b/test/smime/ca.crt
new file mode 100644
index ..b33d087f
--- /dev/null
+++ b/test/smime/ca.crt
@@ -0,0 +1,20 @@
+-BEGIN CERTIFICATE-
+MIIDLTCCAhWgAwIBAgIULXcNXGI2bZp38sV7cF6VcQfnKDwwDQYJKoZIhvcNAQEN
+BQAwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0
+eTAgFw0xOTExMjAwNjU0MThaGA8yMDUyMDkyNzA2NTQxOFowLTErMCkGA1UEAxMi
+U2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0eTCCASIwDQYJKoZIhvcN
+AQEBBQADggEPADCCAQoCggEBAMUfZ8+NYSh6h36zQcXBo5B6ficAcBJ1f3aLxyN8
+QXB83XuP8aDRWQ9uJvJpQkWVH4zx96/E/zI0t0lDMYtZNqra16h+gxbHJgoq2pRw
+RCOiyYu/p2vzvvZ1dtFTMc/mIigjA/73kokui62j1EFy//fNVIihkVS3rAweq+fI
+8qJHSMhdc2aYa9wOP0eGe/HTiDYgT4L4f2HTGMGGwQgj1vub0gpR4YHmNqr0GyEA
+63mHUQUZpnmN1FEl+nVFA5Ntu4uF++qf/tkTji89/eXYBdKX2yUdTeTIKoCI65IL
+EXxezjTc8aFjf/8E0aWGVZR/DtCsjWOh/s/mV7n/YPyb4+ECAwEAAaNDMEEwDwYD
+VR0TAQH/BAUwAwEB/zAPBgNVHQ8BAf8EBQMDBwYAMB0GA1UdDgQWBBS3Uk1zwIg9
+ssN6WgzzlPf3gKJ32zANBgkqhkiG9w0BAQ0FAAOCAQEALsU91Bmhc6EgCNr7inY2
+2gYPnosJ+kZ1eC0hvHIK9e0Tx74RmhTOe8M2C9YXQKehHpRaX+DLcjup6scoH/bT
+u0THbmzeOy29TTiFcyV9BK+SEKQWW4s98Fwdk9fPWcflHtYvqxjooAV3vHbt6Xmp
+KrKDz/jdg7t0ptI4zSqAf3wNppiJoswlOHBUnH2W1MIYkWQ4jYj5socblVlklHOr
+ykKUiEZAbjU+C1+0FhT4HgLjBB9R4H1H0JRKsggWiZBBJ6UpN0dTN4iD0mDVa0jy
+sJqqWnIViy/xaSDcNaWJmU3o2KmkMkdpinoJ5uLkAHQqXjFaujdU1PkufeA7v3uG
+Rw==
+-END CERTIFICATE-
diff --git a/test/test-lib.sh b/test/test-lib.sh
index d4fcea5a..1ffedb25 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -139,6 +139,8 @@ add_gpgsm_home ()
 gpgsm --batch --no-tty --no-common-certs-import --disable-dirmngr --import 
< $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
 fpr=$(gpgsm --batch --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
 echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
+gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.crt
+echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$GNUPGHOME/trustlist.txt"
 test_debug "cat $GNUPGHOME/import.log"
 }
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 15/15] tests: disable CRL checks from gpgsm

2020-04-30 Thread Daniel Kahn Gillmor
On Wed 2020-04-29 23:12:33 +0300, Tomi Ollila wrote:
> Rest of the series look tolerable to me. That one missing 
> "inconsistent quotes" is inconsistent with added quotes
> in one of the changes in previous email (which just did that)
>
> Otherwise OK (provided that tests pass)
> (except that https://www.ietf.org/id/draft-dkg-lamps-samples-01.html
> if not found (by me either, like David)

I've updated the series to address Tomi's quote inconsistency points,
and to repoint the URL in the commit message to the tools.ietf.org
address (which will continue to host expired drafts, unlike
www.ietf.org).

It's on the "smime-tests" branch on https://gitlab.com/dkg/notmuch.git

I'll send the three updated patches to the list here as well.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 07/15] test: Allow tests to have both gpg and gpgsm active at once

2020-04-30 Thread Daniel Kahn Gillmor
On Wed 2020-04-29 23:02:19 +0300, Tomi Ollila wrote:
> On Tue, Apr 28 2020, Daniel Kahn Gillmor wrote:
>
>> Without this fix, we couldn't run both add_gnupg_home and
>> add_gpgsm_home in the same test script.
>>
>> Signed-off-by: Daniel Kahn Gillmor 
>> ---
>>  test/test-lib.sh | 8 
>>  1 file changed, 4 insertions(+), 4 deletions(-)
>>
>> diff --git a/test/test-lib.sh b/test/test-lib.sh
>> index ac1b9315..d9997b27 100644
>> --- a/test/test-lib.sh
>> +++ b/test/test-lib.sh
>> @@ -110,10 +110,10 @@ unset ALTERNATE_EDITOR
>>  add_gnupg_home ()
>>  {
>>  local output
>> -[ -d ${GNUPGHOME} ] && return
>> +[ -e ${GNUPGHOME}/gpg.conf ] && return
>
> So far so good (except perhaps David's comment not "url" not found)
>
> But here this change could include "consistently quoted" variable
> (or/and have it done in that one commit earlier...)

agreed, quoting this would be good.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Inconsistencies in handling command flags: `--flag=value` different than `--flag value`

2020-04-30 Thread Daniel Kahn Gillmor
On Wed 2020-04-29 08:33:24 -0700, Jameson Graef Rollins wrote:
> On Tue, Apr 28 2020, Daniel Kahn Gillmor  wrote:
>> One final way we could normalize everything and make it less
>> idiosyncratic, with shorter, simpler man pages: deprecate and then drop
>> the --booloption/--no-booloption mechanisms, requiring --booloption=true
>> or --booloption=false instead.  Once they're dropped, allow whitespace
>> between "--booloption true" and "--booloption false" just like every
>> other type of option.
>
> Or we could just use only --booloption/--no-booloption...

I'd be sad about that, because it seems like a pretty idiosyncratic
thing to have options of a certain type behave so differently from
others.

we've seen boolean options turn into non-boolean options in the past,
and changing the syntax of how they're invoked when that happens feels
ugly (though i guess this is at least in part an aesthetic preference).

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 08/15] tests/smime: include secret key material for Bob

2020-04-30 Thread Daniel Kahn Gillmor
On Wed 2020-04-29 23:05:03 +0300, Tomi Ollila wrote:
> Now that I started w/ consistenly quotes -- "$NOTMUCH_SRCDIR/..."
>
> Or maybe not, is this variable consistently unquoted -- or something ;)

there are lots of places where NOTMUCH_SRCDIR is unquoted, and some
where it is.  I guess i should probably not introduce any more unquoted
instances, though.  I'll try to fix that.

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 03/15] tests/smime: Include the Sample LAMPS Certificate Authority

2020-04-30 Thread Daniel Kahn Gillmor
On Tue 2020-04-28 22:43:10 -0300, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>> This CA is useful for test suites and the like, but is not an
>> actually-secure CA, because its secret key material is also published.
>>
>> I plan to use it for its intended purpose in the notmuch test suite.
>>
>> It was copied from this Internet Draft:
>>
>> https://www.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi
>
> This page is not found for me.

hm, it has been superceded by
https://www.ietf.org/id/draft-dkg-lamps-samples-02.html#name-certificate-authority-certi
(which has the same content for the relevant section).

the IETF tools interface also has a non-expiring version of the drafts:

 
https://tools.ietf.org/id/draft-dkg-lamps-samples-02.html#name-certificate-authority-certi


feel free to amend the commit message if that would help.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Inconsistencies in handling command flags: `--flag=value` different than `--flag value`

2020-04-28 Thread Daniel Kahn Gillmor
On Mon 2020-04-27 22:21:36 +0300, Ciprian Dorin Craciun wrote:
> On Mon, Apr 27, 2020 at 9:21 PM Tomi Ollila  wrote:
>>> [dkg wrote:]
>>> release, remove the suggestion to use a whitespace separator from the
>>> documentation, and eventually phase it out entirely in some future
>>> release.
>>
>> Alternatively we could check that next arg is (case-insensitively)
>> (subset of) 'true', 'false', 'yes', 'no', '0', '1', 't', 'nil'
>> (but not tpyoes of these ;) and in that case have that as an option
>> value...
>
> This would be perhaps the best approach.  However I don't think it
> would solve the issues for integrators that would not see these
> warnings in the logs, until it is too late.

this looks really ugly to me, in that some legitimate queries
(e.g. those that include terms like "true" or "1") might not be
accessible, unless the user supplies --booloption=true instead of
--booloption.

I mean, these are all slightly idiosyncratic corner cases, but this
particular corner case looks super ugly and hard to explain to me.  i'm
trying to imagine writing some example text that explains it for the man
page, and it comes out horribly complex!  If we can't explain it
succinctly in the manpage, should we be implementing it?

> Perhaps there could be an additional option (either on the command
> line or in the configuration) that would apply "strict" checking, and
> not letting any other form except `--argument=value`, including the
> boolean flags, and failing loudly.
>
> I think this third option would enable much safer integrations.
>
> (BTW, this "strict" option could also apply to the parsing of the
> search terms, which most of the time are under the control of the end
> user.)

I've spent many years helping to maintain GnuPG now, and i'm pretty wary
of having contextually different modes of argument parsing and
interacting/intersecting arguments.  It also leads to some weird
ambiguities: if --strict is supplied on the command line, then does it
need to be first on the command line?  or could parsing the command line
turn out different if you tack on --strict at the end?  Seems like we'd
be injecting additional idiosyncracies to chase after the first.

One final way we could normalize everything and make it less
idiosyncratic, with shorter, simpler man pages: deprecate and then drop
the --booloption/--no-booloption mechanisms, requiring --booloption=true
or --booloption=false instead.  Once they're dropped, allow whitespace
between "--booloption true" and "--booloption false" just like every
other type of option.



in case it's not clear: I believe that "we have succinct and yet
complete man pages" is a convenient shorthand for "have we made this
command-line program behave in an understandable/usable way?"

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 05/15] tests/smime: Use gpgsm instead of openssl for mml creation of S/MIME msgs

2020-04-28 Thread Daniel Kahn Gillmor
The documentation for message mode clearly states that EasyPG (which
uses GnuPG) is the default and recommended way to use S/MIME with
mml-secure:

[0] 
https://www.gnu.org/software/emacs/manual/html_node/message/Using-S_002fMIME.html

To ensure that this mode works, we just need to import the secret key
in question into gpgsm in addition to the public key.  gpgsm should be
able pick the right keys+certificates to use based on To/From headers,
so we don't have to specify anything manually in the #secure mml tag.

The import process from the OpenSSL-preferred form (cert+secretkey) is
rather ugly, because gpgsm wants to see a PKCS#12 object when
importing secret keys.

Note that EasyPG generates the more modern Content-Type:
application/pkcs7-signature instead of application/x-pkcs7-signature
for the detached signature.

We are also obliged to manually set gpgsm's include-certs setting to 1
because gpgsm defaults to send "everything but the root cert".  In our
weird test case, the certificate we're using is self-signed, so it
*is* the root cert, which means that gpgsm doesn't include it by
default.  Setting it to 1 forces inclusion of the signer's cert, which
satisfies openssl's smime subcommand. See https://dev.gnupg.org/T4878
for more details.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T355-smime.sh |  4 ++--
 test/test-lib.el   | 10 --
 test/test-lib.sh   |  6 +-
 3 files changed, 7 insertions(+), 13 deletions(-)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 84be515a..9debf2da 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -24,7 +24,7 @@ test_begin_subtest "emacs delivery of S/MIME encrypted + 
signed message"
 test_expect_success \
 'emacs_fcc_message \
 "test encrypted message 001" \
-"<#secure method=smime mode=signencrypt keyfile=\\\"test_suite.pem\\\" 
certfile=\\\"test_suite.pem\\\">\nThis is a test encrypted message.\n"'
+"<#secure method=smime mode=signencrypt>\nThis is a test encrypted 
message.\n"'
 
 test_begin_subtest "Signature verification (openssl)"
 notmuch show --format=raw subject:"test signed message 001" |\
@@ -65,7 +65,7 @@ expected='[[[{"id": "X",
   "content-disposition": "attachment",
   "content-length": "NONZERO",
   "content-transfer-encoding": "base64",
-  "content-type": "application/x-pkcs7-signature",
+  "content-type": "application/pkcs7-signature",
   "filename": "smime.p7s"}]}]},
  ['
 test_expect_equal_json \
diff --git a/test/test-lib.el b/test/test-lib.el
index 3ae7a090..b47b388e 100644
--- a/test/test-lib.el
+++ b/test/test-lib.el
@@ -193,13 +193,3 @@ nothing."
 ;; environments
 
 (setq mm-text-html-renderer 'html2text)
-
-;; Set some variables for S/MIME tests.
-
-(setq smime-keys '(("" "test_suite.pem" nil)))
-
-(setq mml-smime-use 'openssl)
-
-;; all test keys are without passphrase
-(eval-after-load 'smime
-  '(defun smime-ask-passphrase (cache)  nil))
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 31f37ed7..ac1b9315 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -136,11 +136,15 @@ add_gpgsm_home ()
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
 mkdir -m 0700 "$GNUPGHOME"
-gpgsm --batch --no-tty --no-common-certs-import --disable-dirmngr --import 
< $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
+openssl pkcs12 -export -passout pass: -inkey 
"$NOTMUCH_SRCDIR/test/smime/key+cert.pem" \
+< "$NOTMUCH_SRCDIR/test/smime/test.crt" | \
+gpgsm --batch --no-tty --no-common-certs-import 
--pinentry-mode=loopback --passphrase-fd 3 \
+  --disable-dirmngr --import  >"$GNUPGHOME"/import.log 2>&1 3<<<''
 fpr=$(gpgsm --batch --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
 echo "$fpr S relax" >> "$GNUPGHOME/trustlist.txt"
 gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.crt
 echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$GNUPGHOME/trustlist.txt"
+echo include-certs::1 | gpgconf --output /dev/null --change-options gpgsm
 test_debug "cat $GNUPGHOME/import.log"
 }
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 03/15] tests/smime: Include the Sample LAMPS Certificate Authority

2020-04-28 Thread Daniel Kahn Gillmor
This CA is useful for test suites and the like, but is not an
actually-secure CA, because its secret key material is also published.

I plan to use it for its intended purpose in the notmuch test suite.

It was copied from this Internet Draft:

https://www.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi

Signed-off-by: Daniel Kahn Gillmor 
---
 test/smime/README |  2 ++
 test/smime/ca.crt | 20 
 test/test-lib.sh  |  2 ++
 3 files changed, 24 insertions(+)
 create mode 100644 test/smime/ca.crt

diff --git a/test/smime/README b/test/smime/README
index 46211922..88633bcc 100644
--- a/test/smime/README
+++ b/test/smime/README
@@ -5,3 +5,5 @@ key+cert.pem: cert + unencryped private
 % gpgsm --import test.crt
 % gpgsm --export-private-key-p12 -out foo.p12  (no passphrase)
 % openssl pkcs12 -in ns.p12 -clcerts -nodes > key+cert.pem
+
+ca.crt: from 
https://www.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi
diff --git a/test/smime/ca.crt b/test/smime/ca.crt
new file mode 100644
index ..b33d087f
--- /dev/null
+++ b/test/smime/ca.crt
@@ -0,0 +1,20 @@
+-BEGIN CERTIFICATE-
+MIIDLTCCAhWgAwIBAgIULXcNXGI2bZp38sV7cF6VcQfnKDwwDQYJKoZIhvcNAQEN
+BQAwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0
+eTAgFw0xOTExMjAwNjU0MThaGA8yMDUyMDkyNzA2NTQxOFowLTErMCkGA1UEAxMi
+U2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0eTCCASIwDQYJKoZIhvcN
+AQEBBQADggEPADCCAQoCggEBAMUfZ8+NYSh6h36zQcXBo5B6ficAcBJ1f3aLxyN8
+QXB83XuP8aDRWQ9uJvJpQkWVH4zx96/E/zI0t0lDMYtZNqra16h+gxbHJgoq2pRw
+RCOiyYu/p2vzvvZ1dtFTMc/mIigjA/73kokui62j1EFy//fNVIihkVS3rAweq+fI
+8qJHSMhdc2aYa9wOP0eGe/HTiDYgT4L4f2HTGMGGwQgj1vub0gpR4YHmNqr0GyEA
+63mHUQUZpnmN1FEl+nVFA5Ntu4uF++qf/tkTji89/eXYBdKX2yUdTeTIKoCI65IL
+EXxezjTc8aFjf/8E0aWGVZR/DtCsjWOh/s/mV7n/YPyb4+ECAwEAAaNDMEEwDwYD
+VR0TAQH/BAUwAwEB/zAPBgNVHQ8BAf8EBQMDBwYAMB0GA1UdDgQWBBS3Uk1zwIg9
+ssN6WgzzlPf3gKJ32zANBgkqhkiG9w0BAQ0FAAOCAQEALsU91Bmhc6EgCNr7inY2
+2gYPnosJ+kZ1eC0hvHIK9e0Tx74RmhTOe8M2C9YXQKehHpRaX+DLcjup6scoH/bT
+u0THbmzeOy29TTiFcyV9BK+SEKQWW4s98Fwdk9fPWcflHtYvqxjooAV3vHbt6Xmp
+KrKDz/jdg7t0ptI4zSqAf3wNppiJoswlOHBUnH2W1MIYkWQ4jYj5socblVlklHOr
+ykKUiEZAbjU+C1+0FhT4HgLjBB9R4H1H0JRKsggWiZBBJ6UpN0dTN4iD0mDVa0jy
+sJqqWnIViy/xaSDcNaWJmU3o2KmkMkdpinoJ5uLkAHQqXjFaujdU1PkufeA7v3uG
+Rw==
+-END CERTIFICATE-
diff --git a/test/test-lib.sh b/test/test-lib.sh
index d4fcea5a..1ffedb25 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -139,6 +139,8 @@ add_gpgsm_home ()
 gpgsm --batch --no-tty --no-common-certs-import --disable-dirmngr --import 
< $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
 fpr=$(gpgsm --batch --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
 echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
+gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.crt
+echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$GNUPGHOME/trustlist.txt"
 test_debug "cat $GNUPGHOME/import.log"
 }
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 04/15] tests/smime: consistently quote $GNUPGHOME

2020-04-28 Thread Daniel Kahn Gillmor
Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 1ffedb25..31f37ed7 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -132,13 +132,13 @@ add_gnupg_home ()
 add_gpgsm_home ()
 {
 local fpr
-[ -d ${GNUPGHOME} ] && return
+[ -d "$GNUPGHOME" ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
 mkdir -m 0700 "$GNUPGHOME"
 gpgsm --batch --no-tty --no-common-certs-import --disable-dirmngr --import 
< $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
 fpr=$(gpgsm --batch --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
-echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
+echo "$fpr S relax" >> "$GNUPGHOME/trustlist.txt"
 gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.crt
 echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$GNUPGHOME/trustlist.txt"
 test_debug "cat $GNUPGHOME/import.log"
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 14/15] test/protected-headers: Add tests for S/MIME protected headers

2020-04-28 Thread Daniel Kahn Gillmor
Recognize the protected subject for S/MIME example protected header
messages.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T356-protected-headers.sh | 38 +++---
 1 file changed, 35 insertions(+), 3 deletions(-)

diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh
index 925805df..b7a83715 100755
--- a/test/T356-protected-headers.sh
+++ b/test/T356-protected-headers.sh
@@ -1,14 +1,14 @@
 #!/usr/bin/env bash
 
-# TODO:
-#  * check S/MIME as well as PGP/MIME
-
 test_description='Message decryption with protected headers'
 . $(dirname "$0")/test-lib.sh || exit 1
 
 ##
 
+test_require_external_prereq gpgsm
+
 add_gnupg_home
+add_gpgsm_home
 
 add_email_corpus protected-headers
 
@@ -155,6 +155,38 @@ test_begin_subtest "identify message that had a legacy 
display part skipped duri
 output=$(notmuch search --output=messages 
property:index.repaired=skip-protected-headers-legacy-display)
 test_expect_equal "$output" 
id:protected-with-legacy-disp...@crypto.notmuchmail.org
 
+for variant in multipart-signed onepart-signed; do
+test_begin_subtest "verify signed PKCS#7 subject ($variant)"
+test_subtest_known_broken
+output=$(notmuch show --verify --format=json 
"id:smime-${variant}@protected-headers.example")
+test_json_nodes <<<"$output" \
+
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
+
'sig_good:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' \
+
'sig_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"'
 \
+
'sig_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice 
Lovelace"' \
+'not_encrypted:[0][0][0]["crypto"]!"decrypted"'
+done
+
+for variant in sign+enc sign+enc+legacy-disp; do
+test_begin_subtest "confirm signed and encrypted PKCS#7 subject ($variant)"
+test_subtest_known_broken
+output=$(notmuch show --decrypt=true --format=json 
"id:smime-${variant}@protected-headers.example")
+test_json_nodes <<<"$output" \
+
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
+
'sig_good:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' \
+
'sig_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"'
 \
+
'sig_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice 
Lovelace"' \
+
'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}'
+done
+
+test_begin_subtest "confirm encryption-protected PKCS#7 subject 
(enc+legacy-disp)"
+test_subtest_known_broken
+output=$(notmuch show --decrypt=true --format=json 
"id:smime-enc+legacy-disp@protected-headers.example")
+test_json_nodes <<<"$output" \
+
'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}'
 \
+'no_sig:[0][0][0]["crypto"]!"signed"'
+
+
 # TODO: test that a part that looks like a legacy-display in
 # multipart/signed, but not encrypted, is indexed and not stripped.
 
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 15/15] tests: disable CRL checks from gpgsm

2020-04-28 Thread Daniel Kahn Gillmor
GPGME has a strange failure mode when it is in offline mode, and/or
when certificates don't have any CRLs: in particular, it refuses to
accept the validity of any certificate other than a "root" cert.

This can be worked around by setting the `disable-crl-checks`
configuration variable for gpgsm.

I've reported this to the GPGME upstream at
https://dev.gnupg.org/T4883, but I have no idea how it will be
resolved.  In the meantime, we'll just work around it.

Note that this fixes the test for verification of
id:smime-multipart-signed@protected-headers.example, because
multipart/signed messages are already handled correctly (one-part
PKCS#7 messages will get fixed later).

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T356-protected-headers.sh | 2 +-
 test/test-lib.sh   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh
index b7a83715..520cb71c 100755
--- a/test/T356-protected-headers.sh
+++ b/test/T356-protected-headers.sh
@@ -157,7 +157,7 @@ test_expect_equal "$output" 
id:protected-with-legacy-display@crypto.notmuchmail.
 
 for variant in multipart-signed onepart-signed; do
 test_begin_subtest "verify signed PKCS#7 subject ($variant)"
-test_subtest_known_broken
+[ "$variant" = multipart-signed ] || test_subtest_known_broken
 output=$(notmuch show --verify --format=json 
"id:smime-${variant}@protected-headers.example")
 test_json_nodes <<<"$output" \
 
'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6f47994e..2a7cbbb1 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -144,7 +144,7 @@ add_gpgsm_home ()
 echo "$fpr S relax" >> "$GNUPGHOME/trustlist.txt"
 gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.crt
 echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> 
"$GNUPGHOME/trustlist.txt"
-echo include-certs::1 | gpgconf --output /dev/null --change-options gpgsm
+printf '%s::1\n' include-certs disable-crl-checks | gpgconf --output 
/dev/null --change-options gpgsm
 gpgsm --batch --no-tty --no-common-certs-import --pinentry-mode=loopback 
--passphrase-fd 3 \
   --disable-dirmngr --import $NOTMUCH_SRCDIR/test/smime/bob.p12 
>>"$GNUPGHOME"/import.log 2>&1 3<<<''
 test_debug "cat $GNUPGHOME/import.log"
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 11/15] tests/smime: Test indexing cleartext of envelopedData

2020-04-28 Thread Daniel Kahn Gillmor
These tests describe some simple behavior we would expect to work if
we were to correctly index the cleartext of encrypted S/MIME messages
(PKCS#7 envelopedData).

Of course, they don't currently pass, so we mark them known-broken.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T355-smime.sh | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index f3956a34..14e4531d 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -97,4 +97,26 @@ test_json_nodes <<<"$output" \
 
'crypto_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="616F46CD73834C63847756AF0DFB64A6E0972A47"'
 \
 
'crypto_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Notmuch 
Test Suite"'
 
+test_begin_subtest "encrypted+signed message is known to be encrypted, but 
signature is unknown"
+test_subtest_known_broken
+output=$(notmuch search subject:"test encrypted message 001")
+test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox)"
+
+test_begin_subtest "Encrypted body is not indexed"
+output=$(notmuch search 'this is a test encrypted message')
+test_expect_equal "$output" ""
+
+test_begin_subtest "Reindex cleartext"
+test_expect_success "notmuch reindex --decrypt=true subject:'test encrypted 
message 001'"
+
+test_begin_subtest "signature is now known"
+test_subtest_known_broken
+output=$(notmuch search subject:"test encrypted message 001")
+test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
+
+test_begin_subtest "Encrypted body is indexed"
+test_subtest_known_broken
+output=$(notmuch search 'this is a test encrypted message')
+test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
+
 test_done
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 07/15] test: Allow tests to have both gpg and gpgsm active at once

2020-04-28 Thread Daniel Kahn Gillmor
Without this fix, we couldn't run both add_gnupg_home and
add_gpgsm_home in the same test script.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index ac1b9315..d9997b27 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -110,10 +110,10 @@ unset ALTERNATE_EDITOR
 add_gnupg_home ()
 {
 local output
-[ -d ${GNUPGHOME} ] && return
+[ -e ${GNUPGHOME}/gpg.conf ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
-mkdir -m 0700 "$GNUPGHOME"
+mkdir -p -m 0700 "$GNUPGHOME"
 gpg --no-tty --import <$NOTMUCH_SRCDIR/test/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
 test_debug "cat $GNUPGHOME/import.log"
 if (gpg --quick-random --version >/dev/null 2>&1) ; then
@@ -132,10 +132,10 @@ add_gnupg_home ()
 add_gpgsm_home ()
 {
 local fpr
-[ -d "$GNUPGHOME" ] && return
+[ -e "$GNUPGHOME/gpgsm.conf" ] && return
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
-mkdir -m 0700 "$GNUPGHOME"
+mkdir -p -m 0700 "$GNUPGHOME"
 openssl pkcs12 -export -passout pass: -inkey 
"$NOTMUCH_SRCDIR/test/smime/key+cert.pem" \
 < "$NOTMUCH_SRCDIR/test/smime/test.crt" | \
 gpgsm --batch --no-tty --no-common-certs-import 
--pinentry-mode=loopback --passphrase-fd 3 \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 13/15] tests/smime: add tests for S/MIME SignedData

2020-04-28 Thread Daniel Kahn Gillmor
Add a simple S/MIME SignedData message, taken from an upcoming draft
of
https://datatracker.ietf.org/doc/draft-autocrypt-lamps-protected-headers/

RFC 8551 describes a SignedData, a one-part clearsigned object that is
more resistant to common patterns of MTA message munging than
multipart/signed (but has the downside that it is only readable by
clients that implement S/MIME).

To make sure sure notmuch can handle this kind of object, we want to
know a few things:

Already working:

 - Is the content of the SignedData object indexed?  It actually is
   right now because of dumb luck -- i think we're indexing the raw
   CMS object and it happens to contain the cleartext of the message
   in a way that we can consume it before passing it on to Xapian.
 - Are we accidentally indexing the embedded PKCS#7 certificates? We
   don't want to, and for some reason I don't understand, our indexing
   is actually skipping the embedded certificates already.  That's
   good!

Still need fixing:
 - do we know the MIME type of the embedded part?
 - do we know that the message is signed?
 - can notmuch-show read its content?
 - can notmuch-show indicate the signature validity?
 - can notmuch-reply properly quote and attribute content?

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T355-smime.sh  | 77 +
 test/corpora/pkcs7/smime-onepart-signed.eml | 51 ++
 2 files changed, 128 insertions(+)
 create mode 100644 test/corpora/pkcs7/smime-onepart-signed.eml

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 14e4531d..117fa2b9 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -119,4 +119,81 @@ test_subtest_known_broken
 output=$(notmuch search 'this is a test encrypted message')
 test_expect_equal "$output" "thread:0002   2000-01-01 [1/1] 
Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)"
 
+add_email_corpus pkcs7
+
+test_begin_subtest "index PKCS#7 SignedData message"
+output=$(notmuch search --output=messages Thanks)
+expected=id:smime-onepart-signed@protected-headers.example
+test_expect_equal "$expected" "$output"
+
+test_begin_subtest "do not index embedded certificates from PKCS#7 SignedData"
+output=$(notmuch search --output=messages 'LAMPS Certificate')
+expected=''
+test_expect_equal "$expected" "$output"
+
+test_begin_subtest "know the MIME type of the embedded part in PKCS#7 
SignedData"
+test_subtest_known_broken
+output=$(notmuch search --output=messages 'mimetype:text/plain')
+expected=id:smime-onepart-signed@protected-headers.example
+test_expect_equal "$expected" "$output"
+
+test_begin_subtest "PKCS#7 SignedData message is tagged 'signed'"
+test_subtest_known_broken
+output=$(notmuch dump id:smime-onepart-signed@protected-headers.example)
+expected='#notmuch-dump batch-tag:3 config,properties,tags
++inbox +signed +unread -- id:smime-onepart-signed@protected-headers.example'
+test_expect_equal "$expected" "$output"
+
+test_begin_subtest "show contents of PKCS#7 SignedData message"
+test_subtest_known_broken
+output=$(notmuch show --format=raw --part=2 
id:smime-onepart-signed@protected-headers.example)
+whitespace=' '
+expected="Bob, we need to cancel this contract.
+
+Please start the necessary processes to make that happen today.
+
+Thanks, Alice
+--${whitespace}
+Alice Lovelace
+President
+OpenPGP Example Corp"
+test_expect_equal "$expected" "$output"
+
+test_begin_subtest "reply to PKCS#7 SignedData message with proper quoting and 
attribution"
+test_subtest_known_broken
+output=$(notmuch reply id:smime-onepart-signed@protected-headers.example)
+expected="From: Notmuch Test Suite 
+Subject: Re: The FooCorp contract
+To: Alice Lovelace , Bob Babbage 
+In-Reply-To: 
+References: 
+
+On Tue, 26 Nov 2019 20:11:29 -0400, Alice Lovelace  wrote:
+> Bob, we need to cancel this contract.
+>${whitespace}
+> Please start the necessary processes to make that happen today.
+>${whitespace}
+> Thanks, Alice
+> --${whitespace}
+> Alice Lovelace
+> President
+> OpenPGP Example Corp"
+test_expect_equal "$expected" "$output"
+
+test_begin_subtest "show PKCS#7 SignedData outputs valid JSON"
+output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
+test_valid_json "$output"
+
+test_begin_subtest "Verify signature on PKCS#7 SignedData message"
+test_subtest_known_broken
+output=$(notmuch show --format=json 
id:smime-onepart-signed@protected-headers.example)
+test_json_nodes <<<"$output" \
+'crypto:[0][0][0]["crypto"]["signed"]["status"][0]={
+"created" : 1574813489,
+"expires" : 2611032

[PATCH 12/15] test-lib.sh: add test_valid_json

2020-04-28 Thread Daniel Kahn Gillmor
This test does exactly what it says on the tin.  It expects JSON data
to be parseable by Python, at least.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index dd7fdfaa..6f47994e 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -524,6 +524,12 @@ test_expect_equal_json () {
 test_expect_equal "$output" "$expected" "$@"
 }
 
+# Ensure that the argument is valid JSON data.
+test_valid_json () {
+PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "import sys, json; 
json.load(sys.stdin)" <<<"$1"
+test_expect_equal "$?" 0
+}
+
 # Sort the top-level list of JSON data from stdin.
 test_sort_json () {
 PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c \
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 06/15] tests/smime: avoid copying the key+cert.pem around

2020-04-28 Thread Daniel Kahn Gillmor
No functional change.

We no longer need to identify the key and cert to mml-mode when
sending an S/MIME message, so making a copy of key+cert.pem to
test_suite.pem is superfluous.  Get rid of the extra file.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T355-smime.sh | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 9debf2da..e92a7b16 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -6,9 +6,7 @@ test_description='S/MIME signature verification and decryption'
 test_require_external_prereq openssl
 test_require_external_prereq gpgsm
 
-cp $NOTMUCH_SRCDIR/test/smime/key+cert.pem test_suite.pem
-
-FINGERPRINT=$(openssl x509 -fingerprint -in test_suite.pem -noout | sed -e 
's/^.*=//' -e s/://g)
+FINGERPRINT=$(openssl x509 -fingerprint -in 
"$NOTMUCH_SRCDIR/test/smime/key+cert.pem" -noout | sed -e 's/^.*=//' -e s/://g)
 
 add_gpgsm_home
 
@@ -74,7 +72,7 @@ test_expect_equal_json \
 
 test_begin_subtest "Decryption and signature verification (openssl)"
 notmuch show --format=raw subject:"test encrypted message 001" |\
-openssl smime -decrypt -recip test_suite.pem |\
+openssl smime -decrypt -recip $NOTMUCH_SRCDIR/test/smime/key+cert.pem |\
 openssl smime -verify -CAfile $NOTMUCH_SRCDIR/test/smime/test.crt 2>OUTPUT
 cat < EXPECTED
 Verification successful
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 09/15] tests: Add S/MIME messages to protected-headers corpus

2020-04-28 Thread Daniel Kahn Gillmor
These sample messages are taken directly from the Protected Headers
draft:

https://www.ietf.org/id/draft-autocrypt-lamps-protected-headers-02.html

Note that this commit doesn't strictly pass the common git pre-commit
hook due to introducing some trailing whitespace.  That's just the
nature of the corpus, though.  We should have that trailing
whitespace, so I've made this commit with --no-verify.

Signed-off-by: Daniel Kahn Gillmor 
---
 .../smime-enc+legacy-disp.eml |  50 +
 .../smime-multipart-signed.eml|  68 
 .../smime-onepart-signed.eml  |  54 ++
 .../smime-sign+enc+legacy-disp.eml| 102 ++
 .../protected-headers/smime-sign+enc.eml  |  95 
 5 files changed, 369 insertions(+)
 create mode 100644 test/corpora/protected-headers/smime-enc+legacy-disp.eml
 create mode 100644 test/corpora/protected-headers/smime-multipart-signed.eml
 create mode 100644 test/corpora/protected-headers/smime-onepart-signed.eml
 create mode 100644 
test/corpora/protected-headers/smime-sign+enc+legacy-disp.eml
 create mode 100644 test/corpora/protected-headers/smime-sign+enc.eml

diff --git a/test/corpora/protected-headers/smime-enc+legacy-disp.eml 
b/test/corpora/protected-headers/smime-enc+legacy-disp.eml
new file mode 100644
index ..6f5c9417
--- /dev/null
+++ b/test/corpora/protected-headers/smime-enc+legacy-disp.eml
@@ -0,0 +1,50 @@
+Received: from localhost (localhost [127.0.0.1]); Wed, 27 Nov 2019
+ 01:27:28 -0700 (UTC-07:00)
+MIME-Version: 1.0
+Content-Transfer-Encoding: base64
+Content-Type: application/pkcs7-mime; name="smime.p7m";
+ smime-type="enveloped-data"
+From: Alice Lovelace 
+To: Bob Babbage 
+Date: Wed, 27 Nov 2019 01:27:00 -0700
+Message-ID: 
+Subject: ...
+
+MIIG5QYJKoZIhvcNAQcDoIIG1jCCBtICAQAxggLCMIIBXQIBADBFMC0xKzApBgNV
+BAMTIlNhbXBsZSBMQU1QUyBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkCFCJT7jBtAgsf
+As31ycE+Ot95phvCMA0GCSqGSIb3DQEBAQUABIIBADEhlzhFzYj6tUAdsRCrSiLl
+d9cgKtlAesJ4cDY4szFWAbnwrCmEcFxjFDUOjbfQCYCG80Sxd+xntni73I7PI2rR
+QLjk3w9VhLwFRyzy7qyJi2CavjKTxysX9f36+FXA+THfVQRM5ypiyYJg91X51PNX
+hJj3DHrnxqKeSl/z1hdt9r+s6XAUCBSvL99BGnODWhNIZtPDzt8fMNcgarfw+D5F
+IZJb6+wX30tkztHkpHHKrrDPveyfnlS/p06Gi3ekrrhBtMQMRb9PA/E+ivDPktsm
+aKg0Oauw4oZSKW3f4ukYhbnndbbagNsnTfs/QFy/p+hhKTrfCd0h1N8mTzedVX0w
+ggFdAgEAMEUwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1
+dGhvcml0eQIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQEBBQAEggEA
+FaK5QaPXJ133D2uybQt//oeDm6PkCAFW9YVOgjnLLz6FD54Dt2i1KCQu1Xlg9W3P
+1zJdYXOftDgilylNfmt/muEsvbRfFtMWUq0VGirHz//BWmY2cW/ocinFO514iviL
+MLE1umsXRNwVIVIk/uh7AmqXjPkRZgRgIMUbSbtmW4DDja+ZM0vmqFQ1iUIlApth
+FpjFfPDHHD8isLTbGi2iK6dEN3DIJFGbg5o3nK6yAhVZ7x3LfFNSNVDDSY5mPFG9
+Vm6uRgEE3Y5P6DbXXo6MHTgg0XY2f4y6MEWhOg37NT9aFAfzBBxJ1oSBWpOOfZnV
+K1DvAwPaemSRz9oWDcBM8DCCBAUGCSqGSIb3DQEHATAUBggqhkiG9w0DBwQIsFkN
+8DEx8muAggPgWGF2WsPq3/a9jUa5GA0YFPiINuETCGTNaEXiVxnT0h0CF+EhZ0T2
+HFCiZEM0dzO05zt9WdVvAREaCSh7ZWG9D9wJF9x+tqQbzMuJ2AdKuoOH73kClvkx
+pHxANLhkY7hzIqRb/eLG5D7Xh8iCDiFecXDh7EHqD/R+sfLN9aHKOcKyY36kesBQ
+R8aHZbbFnnD+oXSDNIPcntGG3BSGMxsWuOp+rpTKeIHWFIungDNKsLIy3kWleENw
+FVIcjUF6QhI1HYW6BeXuVq40GV2OOkmB24rYEW1Jg0hAtY+5rn2mRoyxvUC87bjQ
+hLu6xgPmhun9J324eM5aYVwkmVBnRW9hyxClZ7Sv0zlL7lGQ0VQG+zWHeJ+h/M2j
+mQpLgAUEGxxNCm5ASHuXPIN6pSvrOVplrT8kKLPpmMYEwmTX2/rBO4P8I8uNrqYD
+AyX8p0/l2ArczkWzGTz2luBahrD+cTZPApe5SeyXOxWBl1Lmb0G8o4twBeeBLiHP
+XwYvttx0JYG/hc/lmMpEemJqwj9uZ3wGD03dIhhDX2Oj4ek/7jT6yqJh8C1H+PqA
++HNfNXsFQDrRORoqJS8YVEiYRDQNyePy2ugzLTh88nPtJp92hY7bk9zl3AYaiVFH
++szlLoyzfM9D+geZemR8XfI2ijGnrWMlnyPah/zA6J6RwemhuiMklZGYG85hMU9H
+K4CFVM+m7xYxKpwFVnmkVZjzWInirJhehElhtCXpx/IFGxH9CPbCyEZV1WVStrl/
+0fWTGicMXez6hVQCadWCXy96/eLIXOrC54gSoIJX2TD6jdVEu1YptutyGI6KdQ2p
+yXwhs98Uj7DM3nmFeAcjjN3e8pPoX7aG8eP+MfmHlWN6jA44jMaJmIdp9J20g74J
+MdjvnHa/cGibW/RamPiFObN0F94A83vcpUfU/zZ8cFHi/3/lN6Rm9+3/giGRZa9E
+Y6e2/CEq1cUbPQ09fPwRJmjZCfDce71DKe+ZFGdYtFR7JwDEeZ6BB4Ff4rXctcWD
+PgUJqUGv/SXBcFn4cNUK9MYYqVu1ovd/T7FMf+i3c5MH6BRCvft/i5aeBR+A26Gk
+2awtBPYdHW6+AslrFjncBbtPDlU6vX9AWuC0k0MQYnNkTWS8gTvsriXJZ6Zu5iFE
+ExNuFz7YcnMKnguOn2ph5azzeMm83AYzWXzZPu3mdr5Siuu/Ke38oADKP+BZ08Za
+XVvKvvfnRPXO9kG9hgvEMRU9KOcxn82XoGPNZib+9SPa2zYx5P6HX1Bqe/cmKAen
+FKEiJLSTP2/pc6AWAICqJl978HaUHfMFiN7jEUppAifpAWqNcIGSW5w=
+
diff --git a/test/corpora/protected-headers/smime-multipart-signed.eml 
b/test/corpora/protected-headers/smime-multipart-signed.eml
new file mode 100644
index ..f05d2d98
--- /dev/null
+++ b/test/corpora/protected-headers/smime-multipart-signed.eml
@@ -0,0 +1,68 @@
+Received: from localhost (localhost [127.0.0.1]); Tue, 26 Nov 2019
+ 20:03:17 -0400 (UTC-04:00)
+MIME-Version: 1.0
+Content-Type: multipart/signed; boundary="179";
+ protocol="application/pkcs7-signature"; micalg="sha-256"
+From: Alice Lovelace 
+To: Bob Babbage 
+Date: Tue, 26 Nov 2019 20:03:00 -0400
+Subject: The FooCorp contract
+Message-ID: 
+
+--179
+Content-Type: text/plain; charset="us-ascii"; pr

[PATCH 02/15] tests/smime: Always use --batch with gpgsm

2020-04-28 Thread Daniel Kahn Gillmor
GnuPG's gpgsm, like gpg, should always be used with --batch when it is
invoked in a non-interactive environment.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/test-lib.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/test/test-lib.sh b/test/test-lib.sh
index 6a62b5c1..d4fcea5a 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -136,8 +136,8 @@ add_gpgsm_home ()
 _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
 at_exit_function _gnupg_exit
 mkdir -m 0700 "$GNUPGHOME"
-gpgsm --no-tty --no-common-certs-import --disable-dirmngr --import < 
$NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
-fpr=$(gpgsm  --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
+gpgsm --batch --no-tty --no-common-certs-import --disable-dirmngr --import 
< $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
+fpr=$(gpgsm --batch --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
 echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
 test_debug "cat $GNUPGHOME/import.log"
 }
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 08/15] tests/smime: include secret key material for Bob

2020-04-28 Thread Daniel Kahn Gillmor
This is taken from the same Internet Draft that test/smime/ca.crt
comes from.  See that draft for more details.
https://www.ietf.org/id/draft-dkg-lamps-samples-02.html#name-pkcs12-object-for-bob

We don't use it yet, but it will be used to decrypt other messages in
the test suite.

Note that we include it here with an empty passphrase, rather than
with the passphrase "bob" that it is supplied with in the I-D.  The
underlying cryptographic material is the same, but this way we can
import cleanly into gpgsm without having a passphrase set on it (gpgsm
converts an empty-string passphrase into no passphrase at all on
import).

Signed-off-by: Daniel Kahn Gillmor 
---
 test/smime/bob.p12 | 58 ++
 test/test-lib.sh   |  2 ++
 2 files changed, 60 insertions(+)
 create mode 100644 test/smime/bob.p12

diff --git a/test/smime/bob.p12 b/test/smime/bob.p12
new file mode 100644
index ..774c77d0
--- /dev/null
+++ b/test/smime/bob.p12
@@ -0,0 +1,58 @@
+-BEGIN PKCS12-
+MIIKWAIBAzCCCh4GCSqGSIb3DQEHAaCCCg8EggoLMIIKBzCCBGcGCSqGSIb3DQEH
+BqCCBFgwggRUAgEAMIIETQYJKoZIhvcNAQcBMBwGCiqGSIb3DQEMAQYwDgQICE8J
+3kMad9UCAggAgIIEIPvHjK0eRQrnowMUsz1z1x/IxslNvG6DjPZjNHCkNYYmiRsg
+Leu5nqKf4emWVvYpnlh+4Gql7pyJm3G3zSNhobPkW+P1Eh80tTBoUk7TIvvvmtrE
+YEc/nRR1p1MgjISq4Q/CM6ccCCw6YEiQcj/0mSS7gmHUegD5glcWbVuqAT8M/p1z
+98OP3z37G8ARRLNj1yyp0SVlt59Sx3WNbmYBqkQ96iukjMJvmjV7o6BFYUx46Llb
+tphhdRgKXbK2r1R0TUlvE659TUwlrpGlaFpaGj1kLdzVAnjh1ZWnWO2a2BSj0LzG
+qRyiLwqDFPLJLQEckfV+RPWiRrSewME8URNKdk6eewtHdhrehMo4ZJnOIum8qcSz
+giW61SSyZJsFvILpmMYghIxWmPd/8cNIHBrdFEa7z3QKh5jcJNTCxz6yO9f8F830
+d+WDK7DbGkUW4mVTGg/lEYnCFZDF6S1mr0hx+cew1FbKjLpxfQllIIrLf5d2BF8H
+0STpuylQDVVBFdTRHyeS6td5nulANgOProrRzy3aAKQmZ6iullKl+i2t/2TwfVP/
+gG+yszpOEf8U9txuvbiZ7j4XV158zdaaGiduDqMKLOvbdctwHAsR9ecx5C3NTRDl
+ZlttNoXN9zhT4CkWk1w4sFk2KUurjVraIcjWVT7yOreaaK+6N09M0tnLPDJDTrow
+8WwP/rZhA+t+CMrhqkFBxXsyo5VTM0jWJGO/NLpYXPhDPBsRq8rs1OCrUoVr34aR
+cpUTNhyXkvJUarWDHs88lg0ps0G9/1dXI1AbEsQQg8u+QT2ztGYrg2OQxQyi1Mo4
+u/FkAcEbtlYYLmJjj/S2qVRPJgBALVjw9k5hnYRdAXWVDCJ96PMn1SKORvlMxnZ7
+djlhaztOhTLsiDzywVDYWLvQElunWcAGeDZykWNytwcEagc0VjWKHMibc0JOZQ1T
+crGyOzTlt09xHj1NrItYefIwdtKuJfkAh03B5xI6rJ9ZbK9xidcVxyeRX0lEqdo9
+WHQrhHefAmeyo0TlfsN67kFDp5FLpwEtNaN0lyzpkl30aWZdtP5vkvtfmy5ugYIO
+bXoVa+tO6k5V/VfUFUKdaY7xAX7XRzUUg4jB0D0CuaX+YS+GL+5wuQwIY1y2ihBb
+CuCxlcP1lVEU4CVQba60VTudJtWyE7QpPhf+y81f1wRjwIihFvwzpUFWf8JVEppe
+v3Yot3OWGBmhEqLkC9LELth8o5gLfyYHaXTYNd9aRTiI+0ZC5U3O4wUwYLTG3exM
+rIDTzEMk/p4DYIHkNKVUiRJfGYdAwuRxf3IMcYWARTXlSzl1C3hWmZfvTPlKs1bB
+OHTHP/P+qdOFjxOh+fbyqXPJauBAhHvHgrp3iI6t834wJou26oWNihM7OnWuyQRt
+9DVxG4l+1VjtbQZfTDCCBZgGCSqGSIb3DQEHAaCCBYkEggWFMIIFgTCCBX0GCyqG
+SIb3DQEMCgECoIIE7jCCBOowHAYKKoZIhvcNAQwBAzAOBAjqo0x2p5SqLAICCAAE
+ggTIe6Ws+lu0CoNlCXGM2BEPV09wuRHTJe+KnesrmRbXPF9linG3d6G++tTkBHz/
+yr77/DV5aDYciV1pGAbLuX2lMwuqdxzJ4OBPBAjuX5H+IPRaTbxfHYYIwhG8oZzy
+aHyVhHr9j0h7lzW7xSTYJuBNEJ58L42dfzpNRw9dyRPmcuhZqW14Z3xyDm8yjHfB
+2p99y9/A4qSyJJSUM3O3nLdtIar3ktSTRAijgqq+s9wnsfozQRzWpYaqiRrdzwfO
+HqXk54l3/lMSyLpfPl9LW7er6JbGI4jEyQ3x8WijATM5h/lkZKejh/mOaWCvs6G6
+fGzV4P35EsToYbOk9GX4jl4SyDBt3iEHYm5teDUhJmTcR39lAQuAfxN6rOn/TkoO
+YLxtdD5DLiTfYZPCFyavLEsamr8A4p93torF6Rs7GsaHE6PmCcprzqx71KV0DZKv
+tMY86RoiWPKLFxZcYt1yz9/95c1SO1s4i1GvLpJTEgQxLM2OhfEwDNKd2rMJoq1I
+YIRPSP204dIVwwNdXN1vB2slhN2+/QMOqsEkWtTOpW2QoTGSze49hfmJGdu+91jd
+XZBBMJQfY4q066/eE4IOW7ZZId5uMYxDRnGdEQjJsxyW8YHWLRGQvBC8gMkdbj8e
+0wkXbe+jML7vG7t3hDhLEbj5sTquIMTWrTirPw4SxLCuGZAyJHFN3/nCaOSMFlCG
+wEZHrAozgQXPBYU7p+uIkJ4lDc2ZtW8NM8U15gKZLDFfAE6Vg0jAtfFMqvNnX630
+xfo1z4jBd7VXbBFrPzrmvlTnb1XxNFcPycowzW9tgtN4YnNroCq98VpMC914tdpJ
++C/PI0eJ7M2ir3ajN0RabSm02JO9Hdwoa5OgqLwPYDwiFyQvKFGKqAF8Ph6pSEiZ
+10OnH+DVgEY70A+Le+ZSDosMdrhZfHbCcIFitZJ3sYV/7Q118QckW3szcjmLHS5g
+M6Whl2HhjLsAfsmCnoRlIwjx4g0TiuZcb4hGysq8QjD3Z8qqFK28m6OMHbASQfWg
+U+Qg3vmEvVsnBxStFEIImS3QYQoaT0pk6zKUYsI/fOBnEgxsY0XwTfXzVw7hZDct
+yhNIQVWmfgVZwUw0wLoNu3A5hupjUwQzQr4TPnKkFPI8qHmRrJgP8EA0U0019y3W
+MlK0h/LAJEaUBS0goLJCJ8+1EWr6femjnyuU5hMizOm+3j0JexjWz5TQttioS7Q/
+vcxt5pA9yAWQdH9j72saKEoKmDi+kIPr4mimKJz99LhKp9A6Hj0f1P2V3As8JWyW
+ZKmJKW7qMMCFADlALolobqzA60j6Zeo5jiEj/j2lVlUPPz47WO+uKeb+rx+hgTUc
+Xrhq0+an5tvEXt/8wy3PJFqP+qqHGhOIuPLuhqPyzNowuXirIXsiWnI44/X48W91
+HPEoL3xaebQ6oyTP8dI4CCkkHgiLWL5mskjHMEXvcdR6k0ygmu8DGQCPfUweUZqZ
+wfkhD/jwbVpLR5Y3chpatW0cJ2bsAWdxwtuxF05+fVEePUsR0x+2/v/8eDEHKYwt
+aYlAhI48nyrKKVMmqvqcXnzmJlUaq05GnEcglFbv4MUExL7CxClls6QnVNiZFPrV
+ffVsYT2A300xrm4pan89n3nuavjJn7L1JJdmMXwwVQYJKoZIhvcNAQkUMUgeRgBH
+AG4AdQBQAEcAIABlAHgAcABvAHIAdABlAGQAIABjAGUAcgB0AGkAZgBpAGMAYQB0
+AGUAIAA0ADIAYgBiADIANAAwADYwIwYJKoZIhvcNAQkVMRYEFGaI9k+ZdE9/rxBZ
+4rSdH1BCuyQGMDEwITAJBgUrDgMCGgUABBRJfL4XyIHpXmjbziCGCbSAOK9jKgQI
+drOMeIgXcCYCAggA
+-END PKCS12-
diff --git a/test/test-lib.sh b/test/test-lib.sh
index d9997b27..dd7fdfaa 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -145,6 +145,8 @@ add_gpgsm_home ()
 gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr 
--import < $NOTMUCH_SRCDIR/test/smime/ca.

Add tests for S/MIME PKCS#7 messages

2020-04-28 Thread Daniel Kahn Gillmor
S/MIME messages that use PKCS#7 are not currently well-handled by
notmuch.

This series introduces a set of tests that cover such messages, most
of which are initially broken.  A future (shorter) series will resolve
these tests.

Some S/MIME messages *are* handled correctly by notmuch already: in
particular, multipart/signed messages where the detached signature is
a PKCS#7 signature object (as opposed to one-part S/MIME SignedData
messages) are already working.

There should be no functional change to notmuch from applying this
series, just a more complete test suite.

Comments and feedback welcome, as always!

Regards,

--dkg





___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 01/15] tests: move add_gpgsm_home to test-lib.sh

2020-04-28 Thread Daniel Kahn Gillmor
This allows us to test S/MIME messages in other tests.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T355-smime.sh | 13 -
 test/test-lib.sh   | 13 +
 2 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 11a4d6cd..84be515a 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -3,19 +3,6 @@
 test_description='S/MIME signature verification and decryption'
 . $(dirname "$0")/test-lib.sh || exit 1
 
-add_gpgsm_home ()
-{
-local fpr
-[ -d ${GNUPGHOME} ] && return
-_gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
-at_exit_function _gnupg_exit
-mkdir -m 0700 "$GNUPGHOME"
-gpgsm --no-tty --no-common-certs-import --disable-dirmngr --import < 
$NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
-fpr=$(gpgsm  --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
-echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
-test_debug "cat $GNUPGHOME/import.log"
-}
-
 test_require_external_prereq openssl
 test_require_external_prereq gpgsm
 
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 7f8a3a4d..6a62b5c1 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -129,6 +129,19 @@ add_gnupg_home ()
 printf '%s:6:\n' "$FINGERPRINT" | gpg --quiet --batch --no-tty 
--import-ownertrust
 }
 
+add_gpgsm_home ()
+{
+local fpr
+[ -d ${GNUPGHOME} ] && return
+_gnupg_exit () { gpgconf --kill all 2>/dev/null || true; }
+at_exit_function _gnupg_exit
+mkdir -m 0700 "$GNUPGHOME"
+gpgsm --no-tty --no-common-certs-import --disable-dirmngr --import < 
$NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1
+fpr=$(gpgsm  --list-key test_su...@notmuchmail.org | sed -n 
's/.*fingerprint: //p')
+echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt
+test_debug "cat $GNUPGHOME/import.log"
+}
+
 # Each test should start with something like this, after copyright notices:
 #
 # test_description='Description of this test...
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 10/15] tests/smime: Verify cryptographic message status

2020-04-28 Thread Daniel Kahn Gillmor
When consuming a signed+encrypted S/MIME message generated by emacs,
we expect to see the same cryptographic properties for the message as
a whole.  This is not done correctly yet, so the test is marked as
known broken.

Signed-off-by: Daniel Kahn Gillmor 
---
 test/T355-smime.sh | 9 +
 1 file changed, 9 insertions(+)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index e92a7b16..f3956a34 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -88,4 +88,13 @@ This is a test encrypted message.
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "Cryptographic message status (encrypted+signed)"
+test_subtest_known_broken
+output=$(notmuch show --format=json --decrypt=true subject:"test encrypted 
message 001")
+test_json_nodes <<<"$output" \
+
'crypto_encrypted:[0][0][0]["crypto"]["decrypted"]["status"]="full"' \
+
'crypto_sigok:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' \
+
'crypto_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="616F46CD73834C63847756AF0DFB64A6E0972A47"'
 \
+
'crypto_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Notmuch 
Test Suite"'
+
 test_done
-- 
2.26.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Inconsistencies in handling command flags: `--flag=value` different than `--flag value`

2020-04-27 Thread Daniel Kahn Gillmor
On Mon 2020-04-27 14:53:07 -0300, David Bremner wrote:
> Quoting notmuch(1)
>
>OPTION SYNTAX
>All options accepting an argument can be used with '='
>or ':' as a separator. For the cases where it's not ambiguous
>(in particular excluding boolean options), a space can also be
>used.

This is a pretty twisty way to say what we mean.  Are there other cases
besides boolean options?  If there are, perhaps it'd be clearer to say
something like this for the last sentence:

Except for boolean options and other potential ambiguous cases, a
space can also be used as a separator.

If there aren't, we could say:

Except for boolean options (which would be ambiguous), a space can
also be used as a separator.

Alternately, we could deprecate using whitespace for all options,
produce explicit warnings to stderr when whitespace appears on the next
release, remove the suggestion to use a whitespace separator from the
documentation, and eventually phase it out entirely in some future
release.


--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] util/zlib-extra: de-inline gzerror_str

2020-04-27 Thread Daniel Kahn Gillmor
On Mon 2020-04-27 09:28:08 -0300, David Bremner wrote:
> It turns out the behaviour of inline functions in C header files is
> not a good idea, and can cause linking problems if the compiler
> decides not to inline them.  In principle this is solvable by using a
> "static inline" declaration, but this potentially makes a copy in
> every compilation unit. Since we don't actually care about the
> performance of this function, just use a non-inline function.

LGTM.  No need for premature optimization in the error case anyway.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Weird tagging issue

2020-03-29 Thread Daniel Kahn Gillmor
On Sun 2020-03-29 17:53:01 -0700, Carl Worth wrote:
> On Mon, Mar 30 2020, Brian May wrote:
>> This applies to all 4 of those messages. Yes, they all look like they
>> have the same Message-ID
>
> 4 different messages all with the same message ID definitely violates a
> core assumption of notmuch (that each message as a unique message ID).

Well, not if they're actually all the same message, right?

Brian, can you confirm whether the body of these 4 messages are the same
or not?  (e.g. you might have gotten different copies due to receiving
mail at different e-mail addresses, or through a mailing list, or just
through an SMTP hiccup)

> I think it would be reasonable to state that the behavior of notmuch is
> officially "undefined" in that case.

What about for a message that References: its own Message-ID:?  Do we
expect to handle that case sensibly?  I'm not sure where i'd look in the
test suite to confirm that we intend to make that work OK, and i can
definitely imagine that being a goofy corner case when trying to
manipulate threads.

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Weird tagging issue

2020-03-29 Thread Daniel Kahn Gillmor
Hi Brian--

whew, what a mess!

I'm having a hard time identifying a specific problem, but I did notice
this surprising sequence:

On Mon 2020-03-30 08:03:20 +1100, Brian May wrote:
> subgraph "cluster_id:pr-wspdigital/bupaoshc/8...@bitbucket.org" {
> "pr-wspdigital/bupaoshc/8...@bitbucket.org" [shape=folder];
> "1584434229.8921_0.wspdigital:2,S" [shape=note];
> "1584434231.8924_0.wspdigital:2,S" [shape=note];
> "1584434232.8927_0.wspdigital:2,S" [shape=note];
> "1584434234.8930_0.wspdigital:2,S" [shape=note];
> }

[…]
> "1584434229.8921_0.wspdigital:2,S" -> { 
> "pr-wspdigital/bupaoshc/8...@bitbucket.org" } 
> "1584434231.8924_0.wspdigital:2,S" -> { 
> "pr-wspdigital/bupaoshc/8...@bitbucket.org" }
> "1584434232.8927_0.wspdigital:2,S" -> { 
> "pr-wspdigital/bupaoshc/8...@bitbucket.org" }
> "1584434234.8930_0.wspdigital:2,S" -> { 
> "pr-wspdigital/bupaoshc/8...@bitbucket.org" }

The surprising thing about this output is that there appear to be 4
copies of message that References: itself!

And, many many other messages also reference that message too.

Would you be willing to send me the full headers from one of the above 4
files so i can verify that this is correct?

I have no idea whether this is part of the problem you're seeing, but it
is certainly unusual.

--dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] debian: Add packaging for python3-notmuch2

2020-03-24 Thread Daniel Kahn Gillmor
On Fri 2020-01-10 18:16:37 -0500, Daniel Kahn Gillmor wrote:
> I took one more step at debugging the newly-built modules to try to
> understand why the non-stripped versions might differ, and noticed that
> the debugging info in each module itself is different.  in particular,
> in the 3.7 module, the debugging info contains different paths:
>
>  -./bindings/python-cffi/build/temp.linux-amd64-3.7/notmuch2._capi.c:1272
>  +./bindings/python-cffi/build/temp.linux-amd64-3.8/notmuch2._capi.c:1272
>
> (notmuch2._capi.c is a generated C file here, iiuc)
>
> so maybe the build-id is being generated based on the contents of the
> debug info, in addition to the contents of the stripped-down data?

FWIW, I think the above is indeed what is happening -- we're getting
different build-ids because the paths to the source files (to be stored
in the debug info) are different.

I talked with folks on #debian-python, and they said, basically
(paraphrasing):

  Don't worry about the fact that we're shipping two distinct .so's,
  because debian will only release with one version of python anyway.
  It's an artifact that is only relevant for testing, during the
  transition.

So i hope my public diagnosis of this concern didn't scare people off of
this patch.  We should apply it and ship it in debian.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] nmbug: explicitly prefer python3

2020-03-24 Thread Daniel Kahn Gillmor
On Thu 2020-03-12 18:57:51 +0200, Tomi Ollila wrote:
> I've been running nmbug with a wrapper that runs python 2.7 (since default
> python in that particular machine is python 2.6  -- which doesn't work with
> nmbug (or it may but that is complicated, i'm not sure...).
>
> So, to me just doing 1,s/python/python3/ does not affect me in any way
> for the time being -- and anyone else who is stuck w/ python 2.7 (i doubt
> there are any) could still use python 2.7 the same way...
>
> So, IMO let's get it done =D

Thanks for the explanation and the pointer for other folks who might be
in the same situation, Tomi.  I've cleared notmuch::moreinfo and
notmuch::needs-review from this bug report now.

 --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Weird tagging issue

2020-03-22 Thread Daniel Kahn Gillmor
On Sun 2020-03-22 22:12:14 -0300, David Bremner wrote:
> I think dkg and I agreed a few years ago we should ship [draw-thread]
> as one of our devel tools, but then I didn't follow through. So I'd be
> interested in knowing if it works for you.

fwiw, i still think it's worth shipping it in devel/ sooner rather than
later :)

I wonder whether Sean Whitton would consider something like it in
mailscripts; i suppose we'd need better documentation and a reasonable
test suite for it to do that responsibly.

  --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Weird tagging issue

2020-03-20 Thread Daniel Kahn Gillmor
Hi Brian--

On Fri 2020-03-20 11:06:55 +1100, Brian May wrote:
> Brian May  writes:
>
>> I am having a problem with certain messages, in that I remove the tag
>> and it still shows up in search results.
>
> I just recreated the entire database, and I still get the same problem.

This sounds really confusing, and i'm not sure how to help you debug. :/
when you say "recreated the entire database", do you mean you dropped
all tags and everything, and reindexed the entire datastore?

Does the thread that you were working with contain any sensitive or
private messages?  If not, can you isolate the specific files from the
maildir that compose the thread (e.g. with a manual grep) and try to
recreate the failure with a smaller maildir and dedicated notmuch
database?  I know this is tedious, but if you can do it and share the
maildir, then it would help other folks reproduce the problem and dig
into it further.

if i was debugging this locally and i had a dedicated maildir that
demonstrated the problem, i'd probably try to inspect the xapian
database (e.g. with xapian-delve).  There are several folks on the
#notmuch channel on freenode that can probably give pointers.

Sorry to not be more helpful,

  --dkg



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/2] mime-node: rename decrypted_child to unwrapped_child

2020-03-18 Thread Daniel Kahn Gillmor
When walking the MIME tree, we might need to extract a new MIME
object.  Thus far, we've only done it when decrypting
multipart/encrypted messages, but PKCS#7 (RFC 8551, S/MIME) has
several other transformations that warrant a comparable form of
unwrapping.

Make this member re-usable for PKCS#7 unwrappings as well as
multipart/encrypted decryptions.

This change is just a naming change, it has no effect on function.

Signed-off-by: Daniel Kahn Gillmor 
---
 mime-node.c  | 10 +-
 notmuch-client.h |  6 --
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/mime-node.c b/mime-node.c
index e531078c..2a823dfd 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -227,19 +227,19 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part)
 GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part);
 notmuch_message_t *message = NULL;
 
-if (! node->decrypted_child) {
+if (! node->unwrapped_child) {
for (mime_node_t *parent = node; parent; parent = parent->parent)
if (parent->envelope_file) {
message = parent->envelope_file;
break;
}
 
-   node->decrypted_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
+   node->unwrapped_child = _notmuch_crypto_decrypt 
(>decrypt_attempted,
 
node->ctx->crypto->decrypt,
 message,
 encrypteddata, 
_result, );
 }
-if (! node->decrypted_child) {
+if (! node->unwrapped_child) {
fprintf (stderr, "Failed to decrypt part: %s\n",
 err ? err->message : "no error explanation given");
goto DONE;
@@ -380,8 +380,8 @@ mime_node_child (mime_node_t *parent, int child)
return NULL;
 
 if (GMIME_IS_MULTIPART (parent->part)) {
-   if (child == GMIME_MULTIPART_ENCRYPTED_CONTENT && 
parent->decrypted_child)
-   sub = parent->decrypted_child;
+   if (child == GMIME_MULTIPART_ENCRYPTED_CONTENT && 
parent->unwrapped_child)
+   sub = parent->unwrapped_child;
else
sub = g_mime_multipart_get_part (
GMIME_MULTIPART (parent->part), child);
diff --git a/notmuch-client.h b/notmuch-client.h
index 74690054..89e15ba6 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -395,8 +395,10 @@ struct mime_node {
 struct mime_node_context *ctx;
 
 /* Internal: For successfully decrypted multipart parts, the
- * decrypted part to substitute for the second child. */
-GMimeObject *decrypted_child;
+ * decrypted part to substitute for the second child; or, for
+ * PKCS#7 parts, the part returned after removing/processing the
+ * PKCS#7 transformation */
+GMimeObject *unwrapped_child;
 
 /* Internal: The next child for depth-first traversal and the part
  * number to assign it (or -1 if unknown). */
-- 
2.25.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2] mime-node: Clean up unwrapped MIME parts correctly.

2020-03-18 Thread Daniel Kahn Gillmor
Avoid a memory leak in the notmuch command line.

gmime_multipart_encrypted_decrypt returns a GMimeObject marked by
GMime as "transfer full", so we are supposed to clean up after it.

When parsing a message, notmuch would leak one GMimeObject part per
multipart/encrypted MIME layer.  We clean it up by analogy with
cleaning up the signature list associated with a MIME node.

Signed-off-by: Daniel Kahn Gillmor 
---
 mime-node.c | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/mime-node.c b/mime-node.c
index 2a823dfd..ff6805bf 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -192,6 +192,26 @@ set_signature_list_destructor (mime_node_t *node)
 }
 }
 
+/* Unwrapped MIME part destructor */
+static int
+_unwrapped_child_free (GMimeObject **proxy)
+{
+g_object_unref (*proxy);
+return 0;
+}
+
+/* Set up unwrapped MIME part destructor */
+static void
+set_unwrapped_child_destructor (mime_node_t *node)
+{
+GMimeObject **proxy = talloc (node, GMimeObject *);
+
+if (proxy) {
+   *proxy = node->unwrapped_child;
+   talloc_set_destructor (proxy, _unwrapped_child_free);
+}
+}
+
 /* Verify a signed mime node */
 static void
 node_verify (mime_node_t *node, GMimeObject *part)
@@ -238,6 +258,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject 
*part)
 
node->ctx->crypto->decrypt,
 message,
 encrypteddata, 
_result, );
+   if (node->unwrapped_child)
+   set_unwrapped_child_destructor (node);
 }
 if (! node->unwrapped_child) {
fprintf (stderr, "Failed to decrypt part: %s\n",
-- 
2.25.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


  1   2   3   4   5   6   7   8   9   10   >