[PATCH 4/8] test: initial tests for S/MIME and notmuch-emacs

2015-08-16 Thread David Bremner
Test the ability of notmuch-mua-mail to send S/MIME signed (and
encrypted) messages; this really relies on existing functionality in
message-mode.

The dependency on openssl to generate keys seems acceptable since
that's the method I got to work for smime signing in emacs.

The generated keys and messages will later be useful for testing the
notmuch CLI.
---
 test/T355-smime.sh | 53 ++
 test/smime/openssl-ca-req.conf | 13 +++
 test/smime/openssl-req.conf| 13 +++
 test/test-lib.el   | 10 
 test/test-lib.sh   |  2 +-
 5 files changed, 90 insertions(+), 1 deletion(-)
 create mode 100755 test/T355-smime.sh
 create mode 100644 test/smime/openssl-ca-req.conf
 create mode 100644 test/smime/openssl-req.conf

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
new file mode 100755
index 000..5f3ff12
--- /dev/null
+++ b/test/T355-smime.sh
@@ -0,0 +1,53 @@
+#!/usr/bin/env bash
+
+test_description='S/MIME signature verification and decryption'
+. ./test-lib.sh
+
+test_require_external_prereq openssl
+
+test_begin_subtest "Generate CA Cert"
+openssl genpkey -algorithm RSA -out ca.key -pass pass:test -des3 1024
+openssl req -new -x509 -key ca.key -passin pass:test \
+   -config $TEST_DIRECTORY/smime/openssl-ca-req.conf -out ca.crt
+test_expect_equal "$(openssl verify ca.crt | tail -1)" "OK"
+
+test_begin_subtest "Generate User Cert"
+openssl genpkey -algorithm RSA  -out smime.key 1024
+openssl req -config $TEST_DIRECTORY/smime/openssl-req.conf \
+   -new -key smime.key -passin pass:test -nodes \
+   -out smime.csr
+openssl x509 -req -in smime.csr -passin pass:test -CA ca.crt -CAkey ca.key 
-set_serial 1 -out test_suite.crt -setalias "Self Signed SMIME" -addtrust 
emailProtection -addreject clientAuth -addreject serverAuth -trustout
+# we need one file with the cert and private key
+cat test_suite.crt smime.key > test_suite.pem
+test_expect_equal "$(openssl verify -purpose smimesign -CAfile ca.crt 
test_suite.pem)" "test_suite.pem: OK"
+
+test_expect_success 'emacs delivery of S/MIME signed message' \
+ 'emacs_fcc_message \
+ "test signed message 001" \
+ "This is a test signed message." \
+ "(mml-secure-message-sign \"smime\")"'
+
+# Hard code the MML to avoid several interactive questions
+test_expect_success 'emacs delivery of S/MIME encrypted + signed message' \
+'emacs_fcc_message \
+"test encrypted message 001" \
+"<#secure method=smime mode=signencrypt keyfile=\\\"test_suite.pem\\\" 
certfile=\\\"test_suite.pem\\\">\nThis is a test encrypted message.\n"'
+
+test_begin_subtest "Signature verification (openssl)"
+notmuch show --format=raw subject:"test signed message 001" |\
+openssl smime -verify -CAfile ca.crt 2>OUTPUT
+cat < EXPECTED
+Verification successful
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "Decryption and signature verification (openssl)"
+notmuch show --format=raw subject:"test encrypted message 001" |\
+openssl smime -decrypt -recip test_suite.pem |\
+openssl smime -verify -CAfile ca.crt 2>OUTPUT
+cat < EXPECTED
+Verification successful
+EOF
+test_expect_equal_file OUTPUT EXPECTED
+
+test_done
diff --git a/test/smime/openssl-ca-req.conf b/test/smime/openssl-ca-req.conf
new file mode 100644
index 000..49572ee
--- /dev/null
+++ b/test/smime/openssl-ca-req.conf
@@ -0,0 +1,13 @@
+ [ req ]
+ distinguished_name = req_distinguished_name
+ prompt = no
+
+
+ [ req_distinguished_name ]
+ C  = OZ
+ ST = Munchkinlandia
+ L  = Emerald City
+ O  = Organization Name
+ OU = Dept. of Fake Certs
+ CN = Fast Eddies Certs and Chips
+ emailAddress   = fake-ca at example.com
diff --git a/test/smime/openssl-req.conf b/test/smime/openssl-req.conf
new file mode 100644
index 000..c6b9de7
--- /dev/null
+++ b/test/smime/openssl-req.conf
@@ -0,0 +1,13 @@
+ [ req ]
+ distinguished_name = req_distinguished_name
+ prompt = no
+
+
+ [ req_distinguished_name ]
+ C  = OZ
+ ST = Munchkinlandia
+ L  = Emerald City
+ O  = Not much organization
+ OU = Dept. of Testing
+ CN = Notmuch Test Suite
+ emailAddress   = test_suite at notmuchmail.org
diff --git a/test/test-lib.el b/test/test-lib.el
index 04c8d63..596a705 100644
--- a/test/test-lib.el
+++ b/test/test-lib.el
@@ -188,3 +188,13 @@ nothing."
 ;; environments

 (setq mm-text-html-renderer 'html2text)
+
+;; Set some variables for S/MIME tests.
+
+(setq smime-keys '(("" "test_suite.pem" nil)))
+
+(setq mml-smime-use 'openssl)
+
+;; all test keys are without passphrase
+(eval-after-load 'smime
+  '(defun smime-ask-passphrase (cache)  nil))
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 0bf7163..198d2bb 100

ANNOUNCE: muchsync 2 released

2015-08-16 Thread David Mazieres
I'm pleased to announce the release of muchsync version 2.  Muchsync is
a tool for replicating and synchronizing your notmuch databases across
machines.

The new version is reported to build out of the box on Mac OS X.

There's one new feature, which is a new notmuch config option,
"muchsync.and_tags".  When a message experiences an update conflict,
muchsync deletes any tag in muchsync.and_tags unless that tag is present
in both replicas (a logical and).  For all other tags, muchsync adds the
tag if it exists on either replica (a logical or).  The default for
muchsync.and_tags is to use the value of new.tags, as in previous
versions of muchsync.

As usual, the distribution is available from the muchsync home page:

http://www.muchsync.org/

Enjoy.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] cli/lib: remove support for GMime 2.4

2015-08-16 Thread Tomi Ollila
On Sun, Aug 16 2015, David Bremner  wrote:

> It's becoming a maintenance burden to do anything things with the
> crypto glue code twice, once for 2.4 and once for 2.6. I don't have
> any 2.4 version available to test on my development machine anymore,
> so the 2.4 specific code paths are likely not very well tested.
> ---
>
> I started to rebase the SMIME signature verification patches and got
> aggravated at solving the same conflicts twice in every file.
>
> 2.6.7 is from 2012, so that's a bit newer than some of our
> requirements, but I think not so bad. YMMV.

Perhaps it is time for me to update to gmime 2.6 in this machine. PITA ;/

Tomi

>
>  INSTALL|  6 ++--
>  configure  | 49 ++---
>  crypto.c   | 25 -
>  mime-node.c| 83 ---
>  notmuch-show.c | 87 
> --
>  5 files changed, 17 insertions(+), 233 deletions(-)
>
> diff --git a/INSTALL b/INSTALL
> index eaccd93..b1b9cd5 100644
> --- a/INSTALL
> +++ b/INSTALL
> @@ -20,7 +20,7 @@ configure stage.
>  
>  Dependencies
>  
> -Notmuch depends on four libraries: Xapian, GMime 2.4 or 2.6,
> +Notmuch depends on four libraries: Xapian, GMime 2.6,
>  Talloc, and zlib which are each described below:
>  
>   Xapian
> @@ -39,8 +39,8 @@ Talloc, and zlib which are each described below:
>   reading mail while notmuch would wait for Xapian when removing
>   the "inbox" and "unread" tags from messages in a thread.
>  
> - GMime 2.4 or 2.6
> - 
> + GMime  2.6
> + --
>   GMime provides decoding of MIME email messages for Notmuch.
>  
>   Without GMime, Notmuch would not be able to extract and index
> diff --git a/configure b/configure
> index d289880..440d678 100755
> --- a/configure
> +++ b/configure
> @@ -71,16 +71,6 @@ WITH_BASH=1
>  WITH_RUBY=1
>  WITH_ZSH=1
>  
> -# Compatible GMime versions (with constraints).
> -# If using GMime 2.6, we need to have a version >= 2.6.5 to avoid a
> -# crypto bug. We need 2.6.7 for permissive "From " header handling.
> -GMIME_24_VERSION_CTR=''
> -GMIME_24_VERSION="gmime-2.4 $GMIME_24_VERSION_CTR"
> -GMIME_26_VERSION_CTR='>= 2.6.7'
> -GMIME_26_VERSION="gmime-2.6 $GMIME_26_VERSION_CTR"
> -
> -WITH_GMIME_VERSIONS="$GMIME_26_VERSION;$GMIME_24_VERSION"
> -
>  usage ()
>  {
>  cat < @@ -140,10 +130,6 @@ Fine tuning of some installation directories is 
> available:
>   --bashcompletiondir=DIR Bash completions files 
> [SYSCONFDIR/bash_completion.d]
>   --zshcompletiondir=DIR  Zsh completions files 
> [PREFIX/share/zsh/functions/Completion/Unix]
>  
> -Some specific library versions can be specified (auto-detected otherwise):
> -
> - --with-gmime-version=VERS   Specify GMIME version (2.4 or 2.6)
> -
>  Some features can be disabled (--with-feature=no is equivalent to
>  --without-feature) :
>  
> @@ -231,12 +217,6 @@ for option; do
>   fi
>  elif [ "${option}" = '--without-zsh-completion' ] ; then
>   WITH_ZSH=0
> -elif [ "${option%%=*}" = '--with-gmime-version' ] ; then
> - if [ "${option#*=}" = '2.4' ]; then
> -  WITH_GMIME_VERSIONS=$GMIME_24_VERSION
> -  elif [ "${option#*=}" = '2.6' ]; then
> -  WITH_GMIME_VERSIONS=$GMIME_26_VERSION
> - fi
>  elif [ "${option%%=*}" = '--build' ] ; then
>   true
>  elif [ "${option%%=*}" = '--host' ] ; then
> @@ -391,20 +371,19 @@ if [ ${have_xapian} = "1" ]; then
>  esac
>  fi
>  
> +
> +# we need to have a version >= 2.6.5 to avoid a crypto bug. We need
> +# 2.6.7 for permissive "From " header handling.
> +GMIME_MINVER=2.6.7
> +
>  printf "Checking for GMime development files... "
> -have_gmime=0
> -IFS=';'
> -for gmimepc in $WITH_GMIME_VERSIONS; do
> -if pkg-config --exists $gmimepc; then
> - printf "Yes ($gmimepc).\n"
> - have_gmime=1
> - gmime_cflags=$(pkg-config --cflags $gmimepc)
> - gmime_ldflags=$(pkg-config --libs $gmimepc)
> - break
> -fi
> -done
> -IFS=$DEFAULT_IFS
> -if [ "$have_gmime" = "0" ]; then
> +if pkg-config --exists "gmime-2.6 >= $GMIME_MINVER"; then
> +printf "Yes.\n"
> +have_gmime=1
> +gmime_cflags=$(pkg-config --cflags gmime-2.6)
> +gmime_ldflags=$(pkg-config --libs gmime-2.6)
> +else
> +have_gmime=0
>  printf "No.\n"
>  errors=$((errors + 1))
>  fi
> @@ -640,7 +619,7 @@ EOF
>   echo
>  fi
>  if [ $have_gmime -eq 0 ]; then
> - echo "  Either GMime 2.4 library" $GMIME_24_VERSION_CTR "or GMime 2.6 
> library" $GMIME_26_VERSION_CTR
> + echo "  GMime 2.6 library >= $GMIME_MINVER"
>   echo "  (including development files such as headers)"
>   echo "  http://spruce.sourceforge.net/gmime/";
>   echo
> @@ -1026,7 +1005,7 @@ LINKER_RESOLVES_LIBRARY_DEPENDENCIES = 
> ${linker_resolves_library_dependencies}
>  XAPIAN_CXXFLAGS = ${xapian_cxxflags}
>  XAPIAN_LDFLAGS = ${xapian_ldflags}
>  

Re: [DRAFT PATCH] emacs: show local date next to Date: in case value differs

2015-08-16 Thread Tomi Ollila
On Sun, Aug 16 2015, David Bremner  wrote:

> Tomi Ollila  writes:
>
>> Now I took a little better look why the 'relative' date is not enough for
>> me is that it doesn't always show the local date (it shows like 
>> (28 mins. ago) or (March 30)). If these were (28 mins. ago (07:22)) and
>> (March 30 16:20) then it would be better (w/ +0300 that might be marginally
>> more useful in general but perhaps something that can be lived without).
>
> Going back to this thread after a long time, what about an option to
> show dates always in local absolute time only? This would be compact,
> and somehow easy to decode. Or more generally, choose a TZ to display
> all dates in?

Somebody might like that, but *I* want to be able to see the sender date
(being it right or wrong, machine cannot be sure but human reader can
guess better) and local time (a version that is always absolute). In my
emacs buffer the Date: header in your email currently looked to me like:

Date: Sun, 16 Aug 2015 12:15:14 +0200  (Sun, 16 Aug 2015 13:15:14 +0300) 

In this case the dates did not make much of a difference (and in this time
(22:39 +0300)) the message header bar there was (Today 13:15)). Anyway
sometimes IT matters and in my 80-character terminal window this fits fine.

I don't know better solution and don't know whether there is anyone else
interested in such a feature so I just keep using this for myself...

Tomi

> d

PS: last moment revelation: perhaps I could use just (13:15:14 +0300) there;
still if I'd be the single user of that...
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Enabling and disabling maildir.synchronize_flags

2015-08-16 Thread dm-list-email-notmuch
David Bremner  writes:

> David Mazieres  writes:
>> So my question remains, what's the easiest safe way to re-enable
[ 2 more citation lines. Click/Enter to show. ]
>> synchronize_flags after disabling it?  (Safe meaning it won't change any
>> tags.)  It could be that there's a very simple answer, in which case
>> sticking it in the man page might be nice.
>
> I can't think of a simple, safe, and fast answer.

Okay, thanks.  At least I wasn't missing something obvious.

> 2) when the lastmod changes go in, it seems like you could run the first
>notmuch new after enabling tag synchonizing, and dump only the tag
>changes since a checkpoint lastmod value. This would allow rolling
>back the unwanted tag changes.

Indeed, one of many reasons I'm eagerly awaiting lastmod changes.

> [1]: see this potential test, if for some reason we wanted to
>  guarantee this behaviours.

If we did want this, I'm assuming it would take the form of a new option
to notmuch new (--override-flags) which says to do the synchronization
in the other direction (Xapian -> Maildir)?  There would be benefit to
having such a flag, but I don't know how hard it would be to implement,
so I can't do the cost/benefit analysis.

As a kind of aside, one reason people might want to synchronize flags is
for mobile device support.  I don't regularly access my email from my
mobile phone, but on those rare occasions when I might need to, I set up
an IMAP server and use an imap client on the phone.  I wonder if anyone
has thought about implementing an IMAP-ish server directly on top of
libnotmuch.  (I say IMAP-ish because the obvious SEARCH command
implementation wouldn't be RFC3501-compliant, but who cares when notmuch
has something better.)

Does anyone else use both notmuch but also access email from a mobile
device?  If so what do you do?

David
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[DRAFT PATCH] emacs: show local date next to Date: in case value differs

2015-08-16 Thread David Bremner
Tomi Ollila  writes:

> Now I took a little better look why the 'relative' date is not enough for
> me is that it doesn't always show the local date (it shows like 
> (28 mins. ago) or (March 30)). If these were (28 mins. ago (07:22)) and
> (March 30 16:20) then it would be better (w/ +0300 that might be marginally
> more useful in general but perhaps something that can be lived without).

Going back to this thread after a long time, what about an option to
show dates always in local absolute time only? This would be compact,
and somehow easy to decode. Or more generally, choose a TZ to display
all dates in?

d


Enabling and disabling maildir.synchronize_flags

2015-08-16 Thread David Bremner
dm-list-email-notmuch at scs.stanford.edu writes:

> It seems that disabling it should simply be safe.  But re-enabling, one
> risks losing tags, as the next notmuch new will cause old maildir flags
> to override the xapian database.  So that suggests something like:
>
>notmuch dump > backup
>notmuch config set maildir.synchronize_flags false
># Do I need to run notmuch new here?
>notmuch restore < backup

Hi David;

Sorry about the long delay. I'm not sure I follow the connection between
your paragraph above and the the example. The example seems safe, but as
you say, disabling synching should bs safe anyway.

d


[PATCH] emacs: tree bugfix

2015-08-16 Thread David Bremner
Mark Walters  writes:

> Formerly replying to an encrypted message in tree-view did not
> work. This is a first attempt to make it work.
> ---
>
> notmuch-mua-reply decides whether to process crypto based on the
> buffer-local variable notmuch-show-process-crypto. This sets to its
> default value (which is notmuch-crypto-process-mime) in tree-mode
> buffers.

Thanks for working on this Mark. Indeed it seems to fix the problem in
my tests.

Maybe just move some of the discussion into the commit message and we'll
merge it?

d


[PATCH 3/8] cli: let the user know which protocol is unknown or unsupported

2015-08-16 Thread David Bremner
From: Jani Nikula 

The current error message is not helpful.
---
 crypto.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/crypto.c b/crypto.c
index f415abd..11c167e 100644
--- a/crypto.c
+++ b/crypto.c
@@ -67,7 +67,8 @@ notmuch_crypto_get_context (notmuch_crypto_t *crypto, const 
char *protocol)
return protocols[i].get_context (crypto);
 }
 
-fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
+fprintf (stderr, "Unknown or unsupported cryptographic protocol %s.\n",
+protocol);
 
 return NULL;
 }
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/8] crypto: make crypto ctx initialization an array

2015-08-16 Thread David Bremner
From: Jani Nikula 

Make it trivial to add handlers for new protocols without duplicating
code. No functional changes.
---
 crypto.c | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/crypto.c b/crypto.c
index 1187ad7..f415abd 100644
--- a/crypto.c
+++ b/crypto.c
@@ -49,6 +49,7 @@ notmuch_crypto_context_t *
 notmuch_crypto_get_context (notmuch_crypto_t *crypto, const char *protocol)
 {
 notmuch_crypto_context_t *cryptoctx = NULL;
+size_t i;
 
 if (! protocol) {
fprintf (stderr, "Cryptographic protocol is empty.\n");
@@ -61,14 +62,14 @@ notmuch_crypto_get_context (notmuch_crypto_t *crypto, const 
char *protocol)
  * parameter names as defined in this document are
  * case-insensitive."  Thus, we use strcasecmp for the protocol.
  */
-if (strcasecmp (protocol, "application/pgp-signature") == 0 ||
-   strcasecmp (protocol, "application/pgp-encrypted") == 0) {
-   cryptoctx = create_gpg_context (crypto);
-} else {
-   fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
+for (i = 0; i < ARRAY_SIZE (protocols); i++) {
+   if (strcasecmp (protocol, protocols[i].protocol) == 0)
+   return protocols[i].get_context (crypto);
 }
 
-return cryptoctx;
+fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
+
+return NULL;
 }
 
 int
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 6/8] cli: crypto: S/MIME verification support

2015-08-16 Thread David Bremner
From: Jani Nikula 

notmuch-show --verify will now also process S/MIME multiparts if
encountered. Requires gmime-2.6 and gpgsm.

Based on work by Jameson Graef Rollins .
---
 crypto.c   | 50 ++
 notmuch-client.h   |  7 +--
 test/T355-smime.sh |  1 -
 3 files changed, 55 insertions(+), 3 deletions(-)

diff --git a/crypto.c b/crypto.c
index 11c167e..ce683d2 100644
--- a/crypto.c
+++ b/crypto.c
@@ -43,6 +43,51 @@ create_gpg_context (notmuch_crypto_t *crypto)
 return gpgctx;
 }
 
+/* Create a PKCS7 context (GMime 2.6) */
+static notmuch_crypto_context_t *
+create_pkcs7_context (notmuch_crypto_t *crypto)
+{
+notmuch_crypto_context_t *pkcs7ctx;
+
+if (crypto->pkcs7ctx)
+   return crypto->pkcs7ctx;
+
+/* TODO: GMimePasswordRequestFunc */
+pkcs7ctx = g_mime_pkcs7_context_new (NULL);
+if (! pkcs7ctx) {
+   fprintf (stderr, "Failed to construct pkcs7 context.\n");
+   return NULL;
+}
+crypto->pkcs7ctx = pkcs7ctx;
+
+g_mime_pkcs7_context_set_always_trust ((GMimePkcs7Context *) pkcs7ctx,
+  FALSE);
+
+return pkcs7ctx;
+}
+
+static const struct {
+const char *protocol;
+notmuch_crypto_context_t *(*get_context) (notmuch_crypto_t *crypto);
+} protocols[] = {
+{
+   .protocol = "application/pgp-signature",
+   .get_context = create_gpg_context,
+},
+{
+   .protocol = "application/pgp-encrypted",
+   .get_context = create_gpg_context,
+},
+{
+   .protocol = "application/pkcs7-signature",
+   .get_context = create_pkcs7_context,
+},
+{
+   .protocol = "application/x-pkcs7-signature",
+   .get_context = create_pkcs7_context,
+},
+};
+
 /* for the specified protocol return the context pointer (initializing
  * if needed) */
 notmuch_crypto_context_t *
@@ -81,5 +126,10 @@ notmuch_crypto_cleanup (notmuch_crypto_t *crypto)
crypto->gpgctx = NULL;
 }
 
+if (crypto->pkcs7ctx) {
+   g_object_unref (crypto->pkcs7ctx);
+   crypto->pkcs7ctx = NULL;
+}
+
 return 0;
 }
diff --git a/notmuch-client.h b/notmuch-client.h
index 1f82656..774b620 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -31,6 +31,8 @@
 #include 
 
 typedef GMimeCryptoContext notmuch_crypto_context_t;
+/* This is automatically included only since gmime 2.6.10 */
+#include 
 
 #include "notmuch.h"
 
@@ -69,6 +71,7 @@ typedef struct notmuch_show_format {
 
 typedef struct notmuch_crypto {
 notmuch_crypto_context_t* gpgctx;
+notmuch_crypto_context_t* pkcs7ctx;
 notmuch_bool_t verify;
 notmuch_bool_t decrypt;
 const char *gpgpath;
@@ -406,8 +409,8 @@ struct mime_node {
 /* Construct a new MIME node pointing to the root message part of
  * message. If crypto->verify is true, signed child parts will be
  * verified. If crypto->decrypt is true, encrypted child parts will be
- * decrypted.  If crypto->gpgctx is NULL, it will be lazily
- * initialized.
+ * decrypted.  If the crypto contexts (crypto->gpgctx or
+ * crypto->pkcs7) are NULL, they will be lazily initialized.
  *
  * Return value:
  *
diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index b3cc76e..caedf5e 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -56,7 +56,6 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "signature verification (notmuch CLI)"
-test_subtest_known_broken
 output=$(notmuch show --format=json --verify subject:"test signed message 001" 
\
 | notmuch_json_show_sanitize \
 | sed -e 's|"created": [1234567890]*|"created": 946728000|' \
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/8] crypto: refactor context creation to facilitate further work

2015-08-16 Thread David Bremner
From: Jani Nikula 

Let the context creation functions decide how to handle multiple calls
and cache the crypto context. No functional changes.
---
 crypto.c | 19 ++-
 1 file changed, 10 insertions(+), 9 deletions(-)

diff --git a/crypto.c b/crypto.c
index a6eb27d..1187ad7 100644
--- a/crypto.c
+++ b/crypto.c
@@ -22,14 +22,20 @@
 
 /* Create a GPG context (GMime 2.6) */
 static notmuch_crypto_context_t *
-create_gpg_context (const char *gpgpath)
+create_gpg_context (notmuch_crypto_t *crypto)
 {
 notmuch_crypto_context_t *gpgctx;
 
+if (crypto->gpgctx)
+   return crypto->gpgctx;
+
 /* TODO: GMimePasswordRequestFunc */
-gpgctx = g_mime_gpg_context_new (NULL, gpgpath ? gpgpath : "gpg");
-if (! gpgctx)
+gpgctx = g_mime_gpg_context_new (NULL, crypto->gpgpath ? crypto->gpgpath : 
"gpg");
+if (! gpgctx) {
+   fprintf (stderr, "Failed to construct gpg context.\n");
return NULL;
+}
+crypto->gpgctx = gpgctx;
 
 g_mime_gpg_context_set_use_agent ((GMimeGpgContext *) gpgctx, TRUE);
 g_mime_gpg_context_set_always_trust ((GMimeGpgContext *) gpgctx, FALSE);
@@ -57,12 +63,7 @@ notmuch_crypto_get_context (notmuch_crypto_t *crypto, const 
char *protocol)
  */
 if (strcasecmp (protocol, "application/pgp-signature") == 0 ||
strcasecmp (protocol, "application/pgp-encrypted") == 0) {
-   if (! crypto->gpgctx) {
-   crypto->gpgctx = create_gpg_context (crypto->gpgpath);
-   if (! crypto->gpgctx)
-   fprintf (stderr, "Failed to construct gpg context.\n");
-   }
-   cryptoctx = crypto->gpgctx;
+   cryptoctx = create_gpg_context (crypto);
 } else {
fprintf (stderr, "Unknown or unsupported cryptographic protocol.\n");
 }
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 7/8] debian: Recommend gpgsm for S/MIME support

2015-08-16 Thread David Bremner
From: Jameson Graef Rollins 

---
 debian/control | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index 4bc4cd9..05cd04f 100644
--- a/debian/control
+++ b/debian/control
@@ -31,7 +31,7 @@ Vcs-Browser: http://git.notmuchmail.org/git/notmuch
 Package: notmuch
 Architecture: any
 Depends: libnotmuch4 (= ${binary:Version}), ${shlibs:Depends}, ${misc:Depends}
-Recommends: notmuch-emacs | notmuch-vim | notmuch-mutt | alot,  gnupg-agent
+Recommends: notmuch-emacs | notmuch-vim | notmuch-mutt | alot,  gnupg-agent, 
gpgsm
 Description: thread-based email index, search and tagging
  Notmuch is a system for indexing, searching, reading, and tagging
  large collections of email messages in maildir or mh format. It uses
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 5/8] test: add broken S/MIME signature verification test for notmuch CLI

2015-08-16 Thread David Bremner
The test is pretty much cut and paste from the PGP/MIME version, with
obvious updates taken from notmuch output.  This also requires setting
up gpgsm infrastucture.
---
 test/T355-smime.sh | 50 ++
 test/test-lib.sh   |  1 +
 2 files changed, 51 insertions(+)

diff --git a/test/T355-smime.sh b/test/T355-smime.sh
index 5f3ff12..b3cc76e 100755
--- a/test/T355-smime.sh
+++ b/test/T355-smime.sh
@@ -3,7 +3,17 @@
 test_description='S/MIME signature verification and decryption'
 . ./test-lib.sh
 
+add_gpgsm_home ()
+{
+local output
+[ -d ${GNUPGHOME} ] && return
+mkdir -m 0700 "$GNUPGHOME"
+gpgsm --no-tty --import < test_suite.pem >"$GNUPGHOME"/import.log 2>&1
+test_debug "cat $GNUPGHOME/import.log"
+}
+
 test_require_external_prereq openssl
+test_require_external_prereq gpgsm
 
 test_begin_subtest "Generate CA Cert"
 openssl genpkey -algorithm RSA -out ca.key -pass pass:test -des3 1024
@@ -21,6 +31,10 @@ openssl x509 -req -in smime.csr -passin pass:test -CA ca.crt 
-CAkey ca.key -set_
 cat test_suite.crt smime.key > test_suite.pem
 test_expect_equal "$(openssl verify -purpose smimesign -CAfile ca.crt 
test_suite.pem)" "test_suite.pem: OK"
 
+add_gpgsm_home
+
+FINGERPRINT=$(openssl x509 -fingerprint -in test_suite.crt -noout | sed -e 
's/^.*=//' -e s/://g)
+
 test_expect_success 'emacs delivery of S/MIME signed message' \
  'emacs_fcc_message \
  "test signed message 001" \
@@ -41,6 +55,42 @@ Verification successful
 EOF
 test_expect_equal_file OUTPUT EXPECTED
 
+test_begin_subtest "signature verification (notmuch CLI)"
+test_subtest_known_broken
+output=$(notmuch show --format=json --verify subject:"test signed message 001" 
\
+| notmuch_json_show_sanitize \
+| sed -e 's|"created": [1234567890]*|"created": 946728000|' \
+ -e 's|"expires": [1234567890]*|"expires": 424242424|' )
+expected='[[[{"id": "X",
+ "match": true,
+ "excluded": false,
+ "filename": "Y",
+ "timestamp": 946728000,
+ "date_relative": "2000-01-01",
+ "tags": ["inbox","signed"],
+ "headers": {"Subject": "test signed message 001",
+ "From": "Notmuch Test Suite ",
+ "To": "test_su...@notmuchmail.org",
+ "Date": "Sat, 01 Jan 2000 12:00:00 +"},
+ "body": [{"id": 1,
+ "sigstatus": [{"status": "good",
+ "fingerprint": "'$FINGERPRINT'",
+ "expires": 424242424,
+ "created": 946728000}],
+ "content-type": "multipart/signed",
+ "content": [{"id": 2,
+ "content-type": "text/plain",
+ "content": "This is a test signed message.\n"},
+ {"id": 3,
+  "content-length": 1930,
+  "content-transfer-encoding": "base64",
+  "content-type": "application/x-pkcs7-signature",
+  "filename": "smime.p7s"}]}]},
+ ['
+test_expect_equal_json \
+"$output" \
+"$expected"
+
 test_begin_subtest "Decryption and signature verification (openssl)"
 notmuch show --format=raw subject:"test encrypted message 001" |\
 openssl smime -decrypt -recip test_suite.pem |\
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 198d2bb..dff8c7a 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -1320,4 +1320,5 @@ test_declare_external_prereq ${TEST_EMACSCLIENT}
 test_declare_external_prereq gdb
 test_declare_external_prereq gpg
 test_declare_external_prereq openssl
+test_declare_external_prereq gpgsm
 test_declare_external_prereq ${NOTMUCH_PYTHON}
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 8/8] debian: add gpgsm as build dependency

2015-08-16 Thread David Bremner
It's not needed for the actual build, but it is needed to run
the SMIME tests.
---
 debian/control | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/control b/debian/control
index 05cd04f..37ecedd 100644
--- a/debian/control
+++ b/debian/control
@@ -22,6 +22,7 @@ Build-Depends:
  emacs23-nox | emacs23 (>=23~) | emacs23-lucid (>=23~),
  gdb [!s390x !ia64 !armel],
  dtach (>= 0.8),
+ gpgsm, 
  bash-completion (>=1.9.0~)
 Standards-Version: 3.9.6
 Homepage: http://notmuchmail.org/
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[no subject]

2015-08-16 Thread David Bremner
It turns out S/MIME encryption requires non-trivial additional code in 
libgmime. There is no reason to wait for that to support signtures (via 
emacs+openssl|epg), and verification (via notmuch-cli).

Here we also test encryption, relying on emacs message-mode facilities.

(At least) two things could be improved here:

- we are using some unholy mix of openssl and gpgsm, and it would
  probably be best to get everything working with gpgsm, since
  we are already working with gpg and gpg-agent.

- It should be possible to do S/MIME decryption on the emacs level  using e.g. 
smime-decrype-region.

On the other hand, it seems like supporting signature verification
is a step forward.

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] fixup! cli/lib: remove support for GMime 2.4

2015-08-16 Thread David Bremner
---
 notmuch-client.h | 18 +-
 1 file changed, 1 insertion(+), 17 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 78680aa..1f82656 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -30,16 +30,7 @@
 
 #include 
 
-/* GMIME_CHECK_VERSION in gmime 2.4 is not usable from the
- * preprocessor (it calls a runtime function). But since
- * GMIME_MAJOR_VERSION and friends were added in gmime 2.6, we can use
- * these to check the version number. */
-#ifdef GMIME_MAJOR_VERSION
-#define GMIME_ATLEAST_26
 typedef GMimeCryptoContext notmuch_crypto_context_t;
-#else
-typedef GMimeCipherContext notmuch_crypto_context_t;
-#endif
 
 #include "notmuch.h"
 
@@ -394,17 +385,10 @@ struct mime_node {
 
 /* True if signature verification on this part was attempted. */
 notmuch_bool_t verify_attempted;
-#ifdef GMIME_ATLEAST_26
+
 /* The list of signatures for signed or encrypted containers. If
  * there are no signatures, this will be NULL. */
 GMimeSignatureList* sig_list;
-#else
-/* For signed or encrypted containers, the validity of the
- * signature.  May be NULL if signature verification failed.  If
- * there are simply no signatures, this will be non-NULL with an
- * empty signers list. */
-const GMimeSignatureValidity *sig_validity;
-#endif
 
 /* Internal: Context inherited from the root iterator. */
 struct mime_node_context *ctx;
-- 
2.5.0

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] cli/lib: remove support for GMime 2.4

2015-08-16 Thread David Bremner
It's becoming a maintenance burden to do anything things with the
crypto glue code twice, once for 2.4 and once for 2.6. I don't have
any 2.4 version available to test on my development machine anymore,
so the 2.4 specific code paths are likely not very well tested.
---

I started to rebase the SMIME signature verification patches and got
aggravated at solving the same conflicts twice in every file.

2.6.7 is from 2012, so that's a bit newer than some of our
requirements, but I think not so bad. YMMV.

 INSTALL|  6 ++--
 configure  | 49 ++---
 crypto.c   | 25 -
 mime-node.c| 83 ---
 notmuch-show.c | 87 --
 5 files changed, 17 insertions(+), 233 deletions(-)

diff --git a/INSTALL b/INSTALL
index eaccd93..b1b9cd5 100644
--- a/INSTALL
+++ b/INSTALL
@@ -20,7 +20,7 @@ configure stage.
 
 Dependencies
 
-Notmuch depends on four libraries: Xapian, GMime 2.4 or 2.6,
+Notmuch depends on four libraries: Xapian, GMime 2.6,
 Talloc, and zlib which are each described below:
 
Xapian
@@ -39,8 +39,8 @@ Talloc, and zlib which are each described below:
reading mail while notmuch would wait for Xapian when removing
the "inbox" and "unread" tags from messages in a thread.
 
-   GMime 2.4 or 2.6
-   
+   GMime  2.6
+   --
GMime provides decoding of MIME email messages for Notmuch.
 
Without GMime, Notmuch would not be able to extract and index
diff --git a/configure b/configure
index d289880..440d678 100755
--- a/configure
+++ b/configure
@@ -71,16 +71,6 @@ WITH_BASH=1
 WITH_RUBY=1
 WITH_ZSH=1
 
-# Compatible GMime versions (with constraints).
-# If using GMime 2.6, we need to have a version >= 2.6.5 to avoid a
-# crypto bug. We need 2.6.7 for permissive "From " header handling.
-GMIME_24_VERSION_CTR=''
-GMIME_24_VERSION="gmime-2.4 $GMIME_24_VERSION_CTR"
-GMIME_26_VERSION_CTR='>= 2.6.7'
-GMIME_26_VERSION="gmime-2.6 $GMIME_26_VERSION_CTR"
-
-WITH_GMIME_VERSIONS="$GMIME_26_VERSION;$GMIME_24_VERSION"
-
 usage ()
 {
 cat <= 2.6.5 to avoid a crypto bug. We need
+# 2.6.7 for permissive "From " header handling.
+GMIME_MINVER=2.6.7
+
 printf "Checking for GMime development files... "
-have_gmime=0
-IFS=';'
-for gmimepc in $WITH_GMIME_VERSIONS; do
-if pkg-config --exists $gmimepc; then
-   printf "Yes ($gmimepc).\n"
-   have_gmime=1
-   gmime_cflags=$(pkg-config --cflags $gmimepc)
-   gmime_ldflags=$(pkg-config --libs $gmimepc)
-   break
-fi
-done
-IFS=$DEFAULT_IFS
-if [ "$have_gmime" = "0" ]; then
+if pkg-config --exists "gmime-2.6 >= $GMIME_MINVER"; then
+printf "Yes.\n"
+have_gmime=1
+gmime_cflags=$(pkg-config --cflags gmime-2.6)
+gmime_ldflags=$(pkg-config --libs gmime-2.6)
+else
+have_gmime=0
 printf "No.\n"
 errors=$((errors + 1))
 fi
@@ -640,7 +619,7 @@ EOF
echo
 fi
 if [ $have_gmime -eq 0 ]; then
-   echo "  Either GMime 2.4 library" $GMIME_24_VERSION_CTR "or GMime 2.6 
library" $GMIME_26_VERSION_CTR
+   echo "  GMime 2.6 library >= $GMIME_MINVER"
echo "  (including development files such as headers)"
echo "  http://spruce.sourceforge.net/gmime/";
echo
@@ -1026,7 +1005,7 @@ LINKER_RESOLVES_LIBRARY_DEPENDENCIES = 
${linker_resolves_library_dependencies}
 XAPIAN_CXXFLAGS = ${xapian_cxxflags}
 XAPIAN_LDFLAGS = ${xapian_ldflags}
 
-# Flags needed to compile and link against GMime-2.4
+# Flags needed to compile and link against GMime
 GMIME_CFLAGS = ${gmime_cflags}
 GMIME_LDFLAGS = ${gmime_ldflags}
 
diff --git a/crypto.c b/crypto.c
index 026640f..a6eb27d 100644
--- a/crypto.c
+++ b/crypto.c
@@ -20,8 +20,6 @@
 
 #include "notmuch-client.h"
 
-#ifdef GMIME_ATLEAST_26
-
 /* Create a GPG context (GMime 2.6) */
 static notmuch_crypto_context_t *
 create_gpg_context (const char *gpgpath)
@@ -39,29 +37,6 @@ create_gpg_context (const char *gpgpath)
 return gpgctx;
 }
 
-#else /* GMIME_ATLEAST_26 */
-
-/* Create a GPG context (GMime 2.4) */
-static notmuch_crypto_context_t *
-create_gpg_context (const char* gpgpath)
-{
-GMimeSession *session;
-notmuch_crypto_context_t *gpgctx;
-
-session = g_object_new (g_mime_session_get_type (), NULL);
-gpgctx = g_mime_gpg_context_new (session, gpgpath ? gpgpath : "gpg");
-g_object_unref (session);
-
-if (! gpgctx)
-   return NULL;
-
-g_mime_gpg_context_set_always_trust ((GMimeGpgContext *) gpgctx, FALSE);
-
-return gpgctx;
-}
-
-#endif /* GMIME_ATLEAST_26 */
-
 /* for the specified protocol return the context pointer (initializing
  * if needed) */
 notmuch_crypto_context_t *
diff --git a/mime-node.c b/mime-node.c
index fd9e4a4..e96e663 100644
--- a/mime-node.c
+++ b/mime-node.c
@@ -129,8 +129,6 @@ DONE:
 return status;
 }
 
-#ifdef GMIME_ATLEAST_26
-
 /* Signature list destructor (GMime 2.6) */

Re: Enabling and disabling maildir.synchronize_flags

2015-08-16 Thread David Bremner
David Mazieres  writes:


> So my question remains, what's the easiest safe way to re-enable
> synchronize_flags after disabling it?  (Safe meaning it won't change any
> tags.)  It could be that there's a very simple answer, in which case
> sticking it in the man page might be nice.

I can't think of a simple, safe, and fast answer.

As a hack, if you run notmuch new right before you enable
synchronize_flags, then next the next notmuch new will not notice the
fact that flags are out of sync [1]. I don't think this really qualifies
as safe, as the fact that the maildir is out of sync with the database
will eventually bite you. Maybe in some special circumstances this
helps.

Otherwise, two things to that might speed up the process.

1) dump and restore both take queries, so you can only worry about the
maildir relevant flags.

2) when the lastmod changes go in, it seems like you could run the first
   notmuch new after enabling tag synchonizing, and dump only the tag
   changes since a checkpoint lastmod value. This would allow rolling
   back the unwanted tag changes.


[1]: see this potential test, if for some reason we wanted to
 guarantee this behaviours.
 
test_begin_subtest "Turning on maildir.synchronize_flags w/o backup"
OLDCONFIG=$(notmuch config get maildir.synchronize_flags)

notmuch config set maildir.synchronize_flags true
add_message [subject]='"Message to test ignoring flags"' \
[date]='"Sat, 01 Jan 2000 12:00:00 -"' \
[filename]='ignoring-flags' [dir]=cur
${NOTMUCH_NEW}
notmuch dump > EXPECTED

notmuch config set maildir.synchronize_flags false
# make flags out of sync with db
mv ${MAIL_DIR}/cur/ignoring-flags ${MAIL_DIR}/cur/ignoring-flags,S
${NOTMUCH_NEW}

notmuch config set maildir.synchronize_flags true
${NOTMUCH_NEW}
notmuch dump > OUTPUT
notmuch config set maildir.synchronize_flags "${OLDCONFIG}"
test_expect_equal_file EXPECTED OUTPUT
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [DRAFT PATCH] emacs: show local date next to Date: in case value differs

2015-08-16 Thread David Bremner
Tomi Ollila  writes:

> Now I took a little better look why the 'relative' date is not enough for
> me is that it doesn't always show the local date (it shows like 
> (28 mins. ago) or (March 30)). If these were (28 mins. ago (07:22)) and
> (March 30 16:20) then it would be better (w/ +0300 that might be marginally
> more useful in general but perhaps something that can be lived without).

Going back to this thread after a long time, what about an option to
show dates always in local absolute time only? This would be compact,
and somehow easy to decode. Or more generally, choose a TZ to display
all dates in?

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Enabling and disabling maildir.synchronize_flags

2015-08-16 Thread David Mazieres
David Bremner  writes:

> dm-list-email-notm...@scs.stanford.edu writes:
>
>> It seems that disabling it should simply be safe.  But re-enabling, one
>> risks losing tags, as the next notmuch new will cause old maildir flags
>> to override the xapian database.  So that suggests something like:
>>
>>notmuch dump > backup
>>notmuch config set maildir.synchronize_flags false
>># Do I need to run notmuch new here?
>>notmuch restore < backup
>
> Hi David;
>
> Sorry about the long delay. I'm not sure I follow the connection between
> your paragraph above and the the example. The example seems safe, but as
> you say, disabling synching should bs safe anyway.

It's not an example, it's kind of a worst case scenario if there's no
easy and safe way way to enable synchronize_flags.  I want to try
disabling flags, but if I change my mind and the only way to get it back
is to do a full notmuch dump/restore, that's a pretty hefty penalty.
Also, even if I do a full notmuch dump / restore, I'm not sure if the
notmuch new is necessary in the middle.

So my question remains, what's the easiest safe way to re-enable
synchronize_flags after disabling it?  (Safe meaning it won't change any
tags.)  It could be that there's a very simple answer, in which case
sticking it in the man page might be nice.

Thanks,
David
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Enabling and disabling maildir.synchronize_flags

2015-08-16 Thread David Mazieres
David Bremner  writes:

> dm-list-email-notmuch at scs.stanford.edu writes:
>
>> It seems that disabling it should simply be safe.  But re-enabling, one
>> risks losing tags, as the next notmuch new will cause old maildir flags
>> to override the xapian database.  So that suggests something like:
>>
>>notmuch dump > backup
>>notmuch config set maildir.synchronize_flags false
>># Do I need to run notmuch new here?
>>notmuch restore < backup
>
> Hi David;
>
> Sorry about the long delay. I'm not sure I follow the connection between
> your paragraph above and the the example. The example seems safe, but as
> you say, disabling synching should bs safe anyway.

It's not an example, it's kind of a worst case scenario if there's no
easy and safe way way to enable synchronize_flags.  I want to try
disabling flags, but if I change my mind and the only way to get it back
is to do a full notmuch dump/restore, that's a pretty hefty penalty.
Also, even if I do a full notmuch dump / restore, I'm not sure if the
notmuch new is necessary in the middle.

So my question remains, what's the easiest safe way to re-enable
synchronize_flags after disabling it?  (Safe meaning it won't change any
tags.)  It could be that there's a very simple answer, in which case
sticking it in the man page might be nice.

Thanks,
David


Re: Enabling and disabling maildir.synchronize_flags

2015-08-16 Thread David Bremner
dm-list-email-notm...@scs.stanford.edu writes:

> It seems that disabling it should simply be safe.  But re-enabling, one
> risks losing tags, as the next notmuch new will cause old maildir flags
> to override the xapian database.  So that suggests something like:
>
>notmuch dump > backup
>notmuch config set maildir.synchronize_flags false
># Do I need to run notmuch new here?
>notmuch restore < backup

Hi David;

Sorry about the long delay. I'm not sure I follow the connection between
your paragraph above and the the example. The example seems safe, but as
you say, disabling synching should bs safe anyway.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs: tree bugfix

2015-08-16 Thread David Bremner
Mark Walters  writes:

> Formerly replying to an encrypted message in tree-view did not
> work. This is a first attempt to make it work.
> ---
>
> notmuch-mua-reply decides whether to process crypto based on the
> buffer-local variable notmuch-show-process-crypto. This sets to its
> default value (which is notmuch-crypto-process-mime) in tree-mode
> buffers.

Thanks for working on this Mark. Indeed it seems to fix the problem in
my tests.

Maybe just move some of the discussion into the commit message and we'll
merge it?

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch