new "crypto" branch providing full PGP/MIME support

[Jameson Graef Rollins]

Quick!  While cworth is working I want to push this back to the top of
the queue!

crypto branch is still ready for merge!

git://finestructure.net/notmuch

Thanks!

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

Re: new crypto branch providing full PGP/MIME support

[Jameson Graef Rollins]

Quick!  While cworth is working I want to push this back to the top of
the queue!

crypto branch is still ready for merge!

git://finestructure.net/notmuch

Thanks!

jamie.


pgpU4YPWyQSEZ.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

signed/encrypted tagging in crypto branch [was: Re: [Review] Re: new "crypto" branch providing full PGP/MIME support]

[Jameson Rollins]

Hey, folks.  I just pushed a couple of patches to my "crypto" branch [0]
that add support for auto-tagging of multipart/signed and
multipart/encrypted messages with the "signed" and "encrypted" tags
respectively.  Only new messages are thus tagged, so a database rebuild
is required to auto-tag old messages.

To be clear, after the previous discussion, these tags indicate nothing
about the validity of signatures or the decryptability of encrypted
messages.  They only indicate that a message contains a signed or
encrypted part according to the self-declared mime type.

This certainly makes it easier to find crypto messages, and it should
allow people to highlight signed or encrypted messages in the search
output using the "notmuch-search-lines-faces" customization variable
[1].

jamie.

[0] git://finestructure.net/notmuch
[1] The notmuch-search-lines-faces variable needs to be rewritten to
allow for full face customization support, but it also needs to continue
to specify a tag/face mapping.  Any elisp experts out there have any
good suggestions how to fix this?
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

signed/encrypted tagging in crypto branch [was: Re: [Review] Re: new crypto branch providing full PGP/MIME support]

[Jameson Rollins]

Hey, folks.  I just pushed a couple of patches to my crypto branch [0]
that add support for auto-tagging of multipart/signed and
multipart/encrypted messages with the signed and encrypted tags
respectively.  Only new messages are thus tagged, so a database rebuild
is required to auto-tag old messages.

To be clear, after the previous discussion, these tags indicate nothing
about the validity of signatures or the decryptability of encrypted
messages.  They only indicate that a message contains a signed or
encrypted part according to the self-declared mime type.

This certainly makes it easier to find crypto messages, and it should
allow people to highlight signed or encrypted messages in the search
output using the notmuch-search-lines-faces customization variable
[1].

jamie.

[0] git://finestructure.net/notmuch
[1] The notmuch-search-lines-faces variable needs to be rewritten to
allow for full face customization support, but it also needs to continue
to specify a tag/face mapping.  Any elisp experts out there have any
good suggestions how to fix this?


pgpoI349yCViC.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Simon Fondrie-Teitler]

On Mon, 28 Feb 2011 14:24:03 +0100, Sebastian Spaeth  
wrote:
> On Sun, 27 Feb 2011 10:41:48 +, Darren McGuicken  fernseed.info> wrote:
> I also run the crypto branch since it has been published and it is
> working just  fine.

I have just started using the crypto branch, and after a few misteps on
my part it is working. It works great. 

 --Simon
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Rob Browning]

Jameson Rollins  writes:

> If folks have suggestions for disambiguating tag names that don't
> themselves create further confusion on some other front, then I'm
> inclined to just go with the simplest and most straightforward tag name.

Are persistent tags required here?  The original question at least,
seemed to just be asking for a visual indicator that a message has
encrypted or signed bits.  So I wondered if that might be accomplished
without actual tags.

Just curious.
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Mon, 28 Feb 2011 21:16:13 -0600, Rob Browning  
wrote:
> Are persistent tags required here?  The original question at least,
> seemed to just be asking for a visual indicator that a message has
> encrypted or signed bits.  So I wondered if that might be accomplished
> without actual tags.

Hey, Rob.  It probably could, but given that we already have
infrastructure for modifying the face of lines in the search output
based on tags, it therefore seems like the easiest way to achieve the
indicator that Ross was interested in would also be via a tag.  Any
other method would probably require extra hacking of the search
function, and hacking of the emacs interface to parse it and act on it.

To me personally the issue was more about wanting to be able to easily
find signed or encrypted messages.  The easiest way to do that would be
with a tag also, since that's kind of what they're for (again I can
imagine some other sort of internal flag in the database, but that seems
like it would be a lot more work).

Given that it should be fairly easy to tag these messages during notmuch
new, and that tags can be easily leveraged by existing functions, tags
seem to me to be the way to go.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Mon, 28 Feb 2011 15:08:39 -0500, Daniel Kahn Gillmor  wrote:
> The outstanding question in my mind is whether those tags could be
> mistaken by a na?ve user for meaning one of the other concepts.  Is
> there a way to name the tags to minimize that kind of confusion?

I think that would be difficult without using a long and cumbersome tag
name ("signed-but-not-verified"??).  But I think it might be a bit of a
moot point, since I kind of think that any user that actually
understands what a signature is, and what signature verification means,
is sophisticated enough to understand that the mere presence of a
signature does not mean it's been verified.  I could be wrong, though.

If folks have suggestions for disambiguating tag names that don't
themselves create further confusion on some other front, then I'm
inclined to just go with the simplest and most straightforward tag name.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

On 02/28/2011 02:56 PM, Jameson Rollins wrote:
> On Mon, 28 Feb 2011 13:59:54 -0500, Daniel Kahn Gillmor  fifthhorseman.net> wrote:
>> But: what does the "signed" tag mean? i wouldn't want to necessarily
>> conflate these four ideas:
> 
> These are good points, Daniel.  However, I had actually just been
> thinking of something much simpler, along the lines of just tagging
> "signed" any message with a "multipart/signed" part, and "encrypted" any
> message with a "multipart/encrypted" part.

this is a fair answer to my questions, not an evasion -- you're
selecting level 0 in both tracks, which is not a bad thing (it's
certainly simpler to get right!)

The outstanding question in my mind is whether those tags could be
mistaken by a na?ve user for meaning one of the other concepts.  Is
there a way to name the tags to minimize that kind of confusion?

--dkg

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Sebastian Spaeth]

On Sun, 27 Feb 2011 10:41:48 +, Darren McGuicken  wrote:
> If feedback is needed here then likewise, I've been running the crypto
> branch since it was made available.  The only strangeness I've seen was
> that which was reported in id:"87sjw2h6xy.fsf at bookbinder.fernseed.info"
> for expired keys.

I also run the crypto branch since it has been published and it is
working just  fine.

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

On 02/28/2011 01:25 PM, Jameson Rollins wrote:
> On Mon, 28 Feb 2011 08:52:45 -0500, Ross Glover  
> wrote:
>> I too am now running the crypto branch and find it quite amazing.  The
>> one feature I would like added, though, is some face color or
>> auto-tagging in the search buffer for mail with encrypted mime parts.
>> It seems like this could be achieved with notmuch effort (by someone
>> notme) by adding similar functionality to that of attachments in
>> index.cc.
> 
> Yes, this is a good idea, Ross, and one that I've actually been wanting
> to implement.  I was thinking of auto-tagging messages with signed parts
> with something like "signed", and encrypted messages with "encrypted".
> Do people like those tags, or would they prefer to see something
> different?  Or more specific, like "pgp-signed"?

i don't care much about the difference between PGP/MIME and S/MIME
message formats, so i prefer the term "signed" to "pgp-signed" and
"encrypted" to "pgp-encrypted".

  

But: what does the "signed" tag mean? i wouldn't want to necessarily
conflate these four ideas:

 0) "this message claims to be cryptographically-signed"

 1) "we have verified a cryptographic signature over this message"

 2) "we have verified a cryptographic signature over this message from a
known key (that is, we believe we know who the key belongs to)"

 3) "we have verified a cryptographic signature on this message from the
sender claimed in the From: line"

3 implies 2, 2 implies 1, and 1 implies 0, of course.  But which level
would a "signed" tag signify?

I'll also note that signed+encrypted messages would not get tagged with
"signed" unless the recipient has successfully decrypted them.  And
then, it's possible that some sub-parts of a message are signed, and
others are not.  Would the tags indicate the maximum "level" found? or
the minimum?  something else?

  

For that matter, what would an automatically-placed "encrypted" tag
mean?  i can think of a few different approaches:

 0) some part of this message is wrapped in an encrypted MIME block

 1) some part of this message is wrapped in an encrypted MIME block that
claims to be decryptable by a key you control

 2) some part of this message is wrapped in an encrypted MIME block and
you can actually decrypt it (have decrypted it in the past?).

2 in particular couldn't be auto-assigned without having access to the
user's secret key material in the first place, but maybe it could be
assigned after a decryption succeeds?


--dkg

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Mon, 28 Feb 2011 13:59:54 -0500, Daniel Kahn Gillmor  wrote:
> But: what does the "signed" tag mean? i wouldn't want to necessarily
> conflate these four ideas:

These are good points, Daniel.  However, I had actually just been
thinking of something much simpler, along the lines of just tagging
"signed" any message with a "multipart/signed" part, and "encrypted" any
message with a "multipart/encrypted" part.

This simpler approach would certainly satisfy my needs, without having
to get into sorting out all the complicated details in the points you
brought up.

Does that sound like it would work for folks, or would they like to see
a more nuanced approach to handling tagging of signed/encrypted
messages?

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Mon, 28 Feb 2011 08:52:45 -0500, Ross Glover  
wrote:
> I too am now running the crypto branch and find it quite amazing.  The
> one feature I would like added, though, is some face color or
> auto-tagging in the search buffer for mail with encrypted mime parts.
> It seems like this could be achieved with notmuch effort (by someone
> notme) by adding similar functionality to that of attachments in
> index.cc.

Yes, this is a good idea, Ross, and one that I've actually been wanting
to implement.  I was thinking of auto-tagging messages with signed parts
with something like "signed", and encrypted messages with "encrypted".
Do people like those tags, or would they prefer to see something
different?  Or more specific, like "pgp-signed"?

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Ross Glover]

I too am now running the crypto branch and find it quite amazing.  The
one feature I would like added, though, is some face color or
auto-tagging in the search buffer for mail with encrypted mime parts.
It seems like this could be achieved with notmuch effort (by someone
notme) by adding similar functionality to that of attachments in
index.cc.

ross

-- 


Sent from an Emacs buffer.

Re: [Review] Re: new crypto branch providing full PGP/MIME support

[Sebastian Spaeth]

On Sun, 27 Feb 2011 10:41:48 +, Darren McGuicken 
mailing-notm...@fernseed.info wrote:
 If feedback is needed here then likewise, I've been running the crypto
 branch since it was made available.  The only strangeness I've seen was
 that which was reported in id:87sjw2h6xy@bookbinder.fernseed.info
 for expired keys.

I also run the crypto branch since it has been published and it is
working just  fine.

Sebastian


pgpG32EW429Vz.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [Review] Re: new crypto branch providing full PGP/MIME support

[Jameson Rollins]

On Mon, 28 Feb 2011 13:59:54 -0500, Daniel Kahn Gillmor 
d...@fifthhorseman.net wrote:
 But: what does the signed tag mean? i wouldn't want to necessarily
 conflate these four ideas:

These are good points, Daniel.  However, I had actually just been
thinking of something much simpler, along the lines of just tagging
signed any message with a multipart/signed part, and encrypted any
message with a multipart/encrypted part.

This simpler approach would certainly satisfy my needs, without having
to get into sorting out all the complicated details in the points you
brought up.

Does that sound like it would work for folks, or would they like to see
a more nuanced approach to handling tagging of signed/encrypted
messages?

jamie.


pgpZh63tduyBT.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [Review] Re: new crypto branch providing full PGP/MIME support

[Rob Browning]

Jameson Rollins jroll...@finestructure.net writes:

 If folks have suggestions for disambiguating tag names that don't
 themselves create further confusion on some other front, then I'm
 inclined to just go with the simplest and most straightforward tag name.

Are persistent tags required here?  The original question at least,
seemed to just be asking for a visual indicator that a message has
encrypted or signed bits.  So I wondered if that might be accomplished
without actual tags.

Just curious.
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[Review] Re: new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

On Sat, 26 Feb 2011 20:45:15 -0400, David Bremner  wrote:
> Further to our discussion on IRC the other day about providing
> feedback on patches, I have run these patches pretty much all of
> February with no glitches.  I am running them on 3 different machines,
> although they are all Debian AMD64 boxes.

If feedback is needed here then likewise, I've been running the crypto
branch since it was made available.  The only strangeness I've seen was
that which was reported in id:"87sjw2h6xy.fsf at bookbinder.fernseed.info"
for expired keys.

Even with that glitch, I really can't see me going back to vanilla
notmuch until these patches are pulled.  They're just ridiculously
useful.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

[Review] Re: new "crypto" branch providing full PGP/MIME support

[David Bremner]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.  It has been applied
> on top of cworth's current master (21e97c50).  It includes the
> following:
> 

Further to our discussion on IRC the other day about providing feedback
on patches, I have run these patches pretty much all of February with
no glitches.  I am running them on 3 different machines, although they
are all Debian AMD64 boxes.

David

P.S. I also started the (possibly goofy) idea of putting [Review] in the
subject to highlight the fact there is feedback on the patches.

[Review] Re: new crypto branch providing full PGP/MIME support

[David Bremner]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins 
jroll...@finestructure.net wrote:
 Hi, all.  I have pushed a new branch called crypto to my notmuch
 repository [0].  This branch provides full support for PGP/MIME signed
 and encrypted messages, including emacs UI support.  It has been applied
 on top of cworth's current master (21e97c50).  It includes the
 following:
 

Further to our discussion on IRC the other day about providing feedback
on patches, I have run these patches pretty much all of February with
no glitches.  I am running them on 3 different machines, although they
are all Debian AMD64 boxes.

David

P.S. I also started the (possibly goofy) idea of putting [Review] in the
subject to highlight the fact there is feedback on the patches.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

On Fri, 04 Feb 2011 09:32:23 -0800, Jameson Rollins  wrote:
> Yeah, I've found some emails that are doing this as well.  I'm looking
> in to what we can do to mitigate the problem, which I think is
> ultimately a problem in GMime.  The json output should not be
> breaking, though, so that I will definitely fix.

I pulled your latest updates to the crypto branch (I can reply to
encrypted mails without resorting to Gnus!  W00t!) and have noticed that
I'm not seeing the json eof anymore, but if I grab the keys for the
various folks in this thread as a test (Micah seems to be signing with
an expired key?) then emacs just hangs when trying to produce the
notmuch-show buffer.

I can actually see the notmuch show process pegging the cpu in top.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: new crypto branch providing full PGP/MIME support

[Darren McGuicken]

On Fri, 04 Feb 2011 09:32:23 -0800, Jameson Rollins 
jroll...@finestructure.net wrote:
 Yeah, I've found some emails that are doing this as well.  I'm looking
 in to what we can do to mitigate the problem, which I think is
 ultimately a problem in GMime.  The json output should not be
 breaking, though, so that I will definitely fix.

I pulled your latest updates to the crypto branch (I can reply to
encrypted mails without resorting to Gnus!  W00t!) and have noticed that
I'm not seeing the json eof anymore, but if I grab the keys for the
various folks in this thread as a test (Micah seems to be signing with
an expired key?) then emacs just hangs when trying to produce the
notmuch-show buffer.

I can actually see the notmuch show process pegging the cpu in top.


pgp3ZjZRCg0WX.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]

[Sebastian Spaeth]

I just love seeing all those yellow signatures. It reminds me to meet
you all in real life to verify your keys :-).

Thanks for implementing this, the whole lot of you. It works really
well.

Sebastian
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[David Bremner]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  wrote:
> 
> Please test and provide feedback.  I would really like to see this
> series merged into the mainline for the next release, if at all
> possible.

I reported some pretty weird experiences yesterday on IRC with replies
not working. It turns out they were caused by having old versions of
xapian installed in /usr/local (which I think might have caused some
mixup with headers and libraries being different versions).

The decryption is working ok for me now (aside from the issue I
mentioned already with unknown encrypted+signed).

d

tl;dr: yesterdays weirdness was not the fault of J. Rollins.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 315 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

Hey, folks.  I've pushed some improvements to the "crypto" branch that:

* add support for replying to encrypted messages
* don't show sigstatus button for unsigned encrypted messages

As always, please let me know encounter any problems.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

On 02/04/2011 11:59 AM, micah anderson wrote:
> On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor  fifthhorseman.net> wrote:
>> when you say "encrypted by" do you mean "encrypted to"?  do you have
>> access to the corresponding secret key?
> 
> If I open a message that was sent to me and was encrypted by the sender
> using a key that they have since revoked, or has expired, I saw this.

so you hold key A, the sender holds key B, they encrypt the message with
B, and then revoke key B, and then you try to read the message (without
holding key B)?

or are you saying you hold key A, sender holds key B, they encrypt to A,
*sign* the message with B, and then revoke B?  If it's this case, does
the same thing happen even if the message is not encrypted?

Sorry for the nit-picking pedantry here -- I'm trying to get a clear
workflow for a test case.

> Most humans I know reference their key by the keyid, where keyid means
> the 8 character representation of their key. When they need to ensure
> that their key is not being spoofed they either rely on the tools to do
> cryptographic verification, or inspect the fingerprint. I dont think
> moving towards using a longer format of the keyid helps here. The key is
> unknown, and to get it known requires going through a key signing
> process, or fingerprint verification, which isn't aided by having a
> longer form keyid. 
> 
> If I am presented with a short form keyid, and I go and try and receive
> that key from the keyservers and then I am presented with two different
> options, then I'm going to inspect the two and determine then which one
> is the right one. I dont need to front-load that knowledge in the
> interface.

and if you're presented with only one key, you will just accept that
one?  that puts you at the mercy of the network (or the keyserver
operator at least, and we can't all have the luxury of being or knowing
the keyserver operator directly).

If you're arguing "i would like to be able to recognize the key by
32-bit key ID" i don't think it's actually possible to do so because of
the spoofability; a tool that provides cryptographic assurances should
discourage attempts to make the user vulnerable to spoofing.

If you're saying we should remove the 64-bit keyID entirely and just say
"signature from unknown, non-validated key", i think that's a defensible
position, though it's not one i would take.

>> is this really a use case worth bothering with?
> 
> No.

:)

>> do you have a suggestion for how you think it should behave
>> differently?
> 
> I think my suggestion was already made: short form.

and as i said above, i think this is a mistake if we're trying to
provide tools that give cryptographic assurances.

thanks for the testing and the corner cases!

--dkg

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 

new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

On Thu, 03 Feb 2011 13:02:41 -0800, Jameson Rollins  wrote:
> > Can we pass the decrypted text to message mode on a reply?
> Yeah, some folks pointed this out on #notmuch this morning.  The
> decryption is only happening in notmuch-show, but we need to do it in
> notmuch-reply as well for the decrypted text to show up in the reply.
> I'm working on it now.

Great news, thanks!

Also: another confirmation for the revoked/expired key behaviour.  Emacs
UI just reports:

  json-read-string: Wrong type argument: characterp, :json-eof

Let me know if you need any samples of misbehaving mails.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[micah anderson]

On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor  wrote:

> On 02/03/2011 11:25 AM, micah anderson wrote:
> > 1. I personally think notmuch-show-process-pgpmime should default to
> > true
> 
> note that with it set to false, you can still M-RET (instead of RET) on
> an item in the summary window to have it set for that particular view.

Yes, that is true. However, I still think the default should be
on. Especially considering all three people (myself included) who I've
seen try this have been puzzled by why it wasn't working and had to be
told that they needed to turn it on.

> > 3. i'm not sure expired/revoked keys are handled properly - tested on a
> > message that was encrypted by a key that was revoked and got "End of
> > file during parsing"
> 
> when you say "encrypted by" do you mean "encrypted to"?  do you have
> access to the corresponding secret key?

If I open a message that was sent to me and was encrypted by the sender
using a key that they have since revoked, or has expired, I saw this.

> > 5. unknown keys are represented in a long format,
> > eg. '0x5585F58CC827A062' when most tools represent them just with their
> > shortened keyid (in this case this one would be: 0xC827A062), is there a
> > particular reason for this?
> 
> Short keyIDs are easily spoofable; believing anything based on a matched
> short keyID is a mistake. 

Most humans I know reference their key by the keyid, where keyid means
the 8 character representation of their key. When they need to ensure
that their key is not being spoofed they either rely on the tools to do
cryptographic verification, or inspect the fingerprint. I dont think
moving towards using a longer format of the keyid helps here. The key is
unknown, and to get it known requires going through a key signing
process, or fingerprint verification, which isn't aided by having a
longer form keyid. 

If I am presented with a short form keyid, and I go and try and receive
that key from the keyservers and then I am presented with two different
options, then I'm going to inspect the two and determine then which one
is the right one. I dont need to front-load that knowledge in the
interface.

> > I recognize some people's keyids in the
> > short form, but do not in the longform.
> 
> You can derive the short form from the long form by ignoring
> everything but the last 8 hex characters.

Yes, I know that, but my brain parser looks at '0x5585F58CC827A062' and
has to squint to try and ignore the last 8 characters. In fact, its
often quicker for me to use my cursor to go up there and count backwards
8 characters and then hit a space so I can visually separate it. Visual
interface parse failure with long form.

> But if you actually recognize the short form, i'd expect you to have
> the relevant key in your keyring;

Not always true. In fact the parse error I was discussing above was
about a key I used to use, but have removed from my keyring, and my
secret keyring. In the long form, I did *not* recognize it. If it were
simply 1CF2D62A I would have instantly recognized it as my old key that
I've revoked.

> is this really a use case worth bothering with?

No.

> do you have a suggestion for how you think it should behave
> differently?

I think my suggestion was already made: short form.

m
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Fri, 04 Feb 2011 13:12:27 -0400, David Bremner  wrote:
> I reported some pretty weird experiences yesterday on IRC with replies
> not working. It turns out they were caused by having old versions of
> xapian installed in /usr/local (which I think might have caused some
> mixup with headers and libraries being different versions).
> 
> The decryption is working ok for me now (aside from the issue I
> mentioned already with unknown encrypted+signed).
...
> tl;dr: yesterdays weirdness was not the fault of J. Rollins.

phew!  Although there remains the issue of replying to encrypted
messages, since notmuch-reply was not modified to decrypt messages
before replying.  I'm working on that now.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Fri, 04 Feb 2011 12:09:48 +, Darren McGuicken  wrote:
> On Thu, 03 Feb 2011 13:02:41 -0800, Jameson Rollins  finestructure.net> wrote:
> > > Can we pass the decrypted text to message mode on a reply?
> > Yeah, some folks pointed this out on #notmuch this morning.  The
> > decryption is only happening in notmuch-show, but we need to do it in
> > notmuch-reply as well for the decrypted text to show up in the reply.
> > I'm working on it now.
> 
> Great news, thanks!

Thanks for the feedback!

> Also: another confirmation for the revoked/expired key behaviour.  Emacs
> UI just reports:
> 
>   json-read-string: Wrong type argument: characterp, :json-eof
> 
> Let me know if you need any samples of misbehaving mails.

Yeah, I've found some emails that are doing this as well.  I'm looking
in to what we can do to mitigate the problem, which I think is
ultimately a problem in GMime.  The json output should not be breaking,
though, so that I will definitely fix.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]

[Jameson Rollins]

On Fri, 04 Feb 2011 14:04:23 +0100, Sebastian Spaeth  
wrote:
> I just love seeing all those yellow signatures. It reminds me to meet
> you all in real life to verify your keys :-).
> 
> Thanks for implementing this, the whole lot of you. It works really
> well.

Yes!  I'm really glad you're finding it useful.  Hopefully you're seeing
some green fully-valid-userid sigs in there as well...

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Fri, 04 Feb 2011 08:24:38 -0400, David Bremner  wrote:
> On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  finestructure.net> wrote:
> > Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> > repository [0].  This branch provides full support for PGP/MIME signed
> > and encrypted messages, including emacs UI support.  It has been applied
> > on top of cworth's current master (21e97c50).  It includes the
> > following:
> > 
> 
> For a signed and encrypted message, if the signing key can't be found,
> one just gets:
> 
>  [ multipart/encrypted: decryption error ]
> 
> which is a bit mystifying.  At least gpg on the command line will
> decrypt the message and print a warning about the signature.

So I need to look at this closer, but I think this is a GMime problem.
That error message is generated when the GMime decryption code returns
NULL for the decrypted part.  I'll try to make a test case for this, and
file a bug with GMime upstream.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[David Bremner]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.  It has been applied
> on top of cworth's current master (21e97c50).  It includes the
> following:
> 

For a signed and encrypted message, if the signing key can't be found,
one just gets:

 [ multipart/encrypted: decryption error ]

which is a bit mystifying.  At least gpg on the command line will
decrypt the message and print a warning about the signature.

All the best, and thanks for working on this, 

d

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 315 bytes
Desc: not available
URL: 

Re: new crypto branch providing full PGP/MIME support

[micah anderson]

On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor 
d...@fifthhorseman.net wrote:

 On 02/03/2011 11:25 AM, micah anderson wrote:
  1. I personally think notmuch-show-process-pgpmime should default to
  true
 
 note that with it set to false, you can still M-RET (instead of RET) on
 an item in the summary window to have it set for that particular view.

Yes, that is true. However, I still think the default should be
on. Especially considering all three people (myself included) who I've
seen try this have been puzzled by why it wasn't working and had to be
told that they needed to turn it on.

  3. i'm not sure expired/revoked keys are handled properly - tested on a
  message that was encrypted by a key that was revoked and got End of
  file during parsing
 
 when you say encrypted by do you mean encrypted to?  do you have
 access to the corresponding secret key?

If I open a message that was sent to me and was encrypted by the sender
using a key that they have since revoked, or has expired, I saw this.

  5. unknown keys are represented in a long format,
  eg. '0x5585F58CC827A062' when most tools represent them just with their
  shortened keyid (in this case this one would be: 0xC827A062), is there a
  particular reason for this?
 
 Short keyIDs are easily spoofable; believing anything based on a matched
 short keyID is a mistake. 

Most humans I know reference their key by the keyid, where keyid means
the 8 character representation of their key. When they need to ensure
that their key is not being spoofed they either rely on the tools to do
cryptographic verification, or inspect the fingerprint. I dont think
moving towards using a longer format of the keyid helps here. The key is
unknown, and to get it known requires going through a key signing
process, or fingerprint verification, which isn't aided by having a
longer form keyid. 

If I am presented with a short form keyid, and I go and try and receive
that key from the keyservers and then I am presented with two different
options, then I'm going to inspect the two and determine then which one
is the right one. I dont need to front-load that knowledge in the
interface.

  I recognize some people's keyids in the
  short form, but do not in the longform.
 
 You can derive the short form from the long form by ignoring
 everything but the last 8 hex characters.

Yes, I know that, but my brain parser looks at '0x5585F58CC827A062' and
has to squint to try and ignore the last 8 characters. In fact, its
often quicker for me to use my cursor to go up there and count backwards
8 characters and then hit a space so I can visually separate it. Visual
interface parse failure with long form.

 But if you actually recognize the short form, i'd expect you to have
 the relevant key in your keyring;

Not always true. In fact the parse error I was discussing above was
about a key I used to use, but have removed from my keyring, and my
secret keyring. In the long form, I did *not* recognize it. If it were
simply 1CF2D62A I would have instantly recognized it as my old key that
I've revoked.

 is this really a use case worth bothering with?

No.

 do you have a suggestion for how you think it should behave
 differently?

I think my suggestion was already made: short form.

m


pgpH8uCPxrQjL.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[David Bremner]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins 
jroll...@finestructure.net wrote:
 
 Please test and provide feedback.  I would really like to see this
 series merged into the mainline for the next release, if at all
 possible.

I reported some pretty weird experiences yesterday on IRC with replies
not working. It turns out they were caused by having old versions of
xapian installed in /usr/local (which I think might have caused some
mixup with headers and libraries being different versions).

The decryption is working ok for me now (aside from the issue I
mentioned already with unknown encrypted+signed).

d

tl;dr: yesterdays weirdness was not the fault of J. Rollins.


pgpnUrpra2H8q.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Jameson Rollins]

On Fri, 04 Feb 2011 08:24:38 -0400, David Bremner brem...@unb.ca wrote:
 On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins 
 jroll...@finestructure.net wrote:
  Hi, all.  I have pushed a new branch called crypto to my notmuch
  repository [0].  This branch provides full support for PGP/MIME signed
  and encrypted messages, including emacs UI support.  It has been applied
  on top of cworth's current master (21e97c50).  It includes the
  following:
  
 
 For a signed and encrypted message, if the signing key can't be found,
 one just gets:
 
  [ multipart/encrypted: decryption error ]
 
 which is a bit mystifying.  At least gpg on the command line will
 decrypt the message and print a warning about the signature.

So I need to look at this closer, but I think this is a GMime problem.
That error message is generated when the GMime decryption code returns
NULL for the decrypted part.  I'll try to make a test case for this, and
file a bug with GMime upstream.

jamie.


pgp44IkU2p9Dy.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: always encrypting messages to self [was: Re: new crypto branch providing full PGP/MIME support]

[Jameson Rollins]

On Fri, 04 Feb 2011 14:04:23 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 I just love seeing all those yellow signatures. It reminds me to meet
 you all in real life to verify your keys :-).
 
 Thanks for implementing this, the whole lot of you. It works really
 well.

Yes!  I'm really glad you're finding it useful.  Hopefully you're seeing
some green fully-valid-userid sigs in there as well...

jamie.


pgppy6NrmUVFM.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

On 02/04/2011 11:59 AM, micah anderson wrote:
 On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor 
 d...@fifthhorseman.net wrote:
 when you say encrypted by do you mean encrypted to?  do you have
 access to the corresponding secret key?
 
 If I open a message that was sent to me and was encrypted by the sender
 using a key that they have since revoked, or has expired, I saw this.

so you hold key A, the sender holds key B, they encrypt the message with
B, and then revoke key B, and then you try to read the message (without
holding key B)?

or are you saying you hold key A, sender holds key B, they encrypt to A,
*sign* the message with B, and then revoke B?  If it's this case, does
the same thing happen even if the message is not encrypted?

Sorry for the nit-picking pedantry here -- I'm trying to get a clear
workflow for a test case.

 Most humans I know reference their key by the keyid, where keyid means
 the 8 character representation of their key. When they need to ensure
 that their key is not being spoofed they either rely on the tools to do
 cryptographic verification, or inspect the fingerprint. I dont think
 moving towards using a longer format of the keyid helps here. The key is
 unknown, and to get it known requires going through a key signing
 process, or fingerprint verification, which isn't aided by having a
 longer form keyid. 
 
 If I am presented with a short form keyid, and I go and try and receive
 that key from the keyservers and then I am presented with two different
 options, then I'm going to inspect the two and determine then which one
 is the right one. I dont need to front-load that knowledge in the
 interface.

and if you're presented with only one key, you will just accept that
one?  that puts you at the mercy of the network (or the keyserver
operator at least, and we can't all have the luxury of being or knowing
the keyserver operator directly).

If you're arguing i would like to be able to recognize the key by
32-bit key ID i don't think it's actually possible to do so because of
the spoofability; a tool that provides cryptographic assurances should
discourage attempts to make the user vulnerable to spoofing.

If you're saying we should remove the 64-bit keyID entirely and just say
signature from unknown, non-validated key, i think that's a defensible
position, though it's not one i would take.

 is this really a use case worth bothering with?
 
 No.

:)

 do you have a suggestion for how you think it should behave
 differently?
 
 I think my suggestion was already made: short form.

and as i said above, i think this is a mistake if we're trying to
provide tools that give cryptographic assurances.

thanks for the testing and the corner cases!

--dkg



signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Jameson Rollins]

On Fri, 04 Feb 2011 12:09:48 +, Darren McGuicken 
mailing-notm...@fernseed.info wrote:
 On Thu, 03 Feb 2011 13:02:41 -0800, Jameson Rollins 
 jroll...@finestructure.net wrote:
   Can we pass the decrypted text to message mode on a reply?
  Yeah, some folks pointed this out on #notmuch this morning.  The
  decryption is only happening in notmuch-show, but we need to do it in
  notmuch-reply as well for the decrypted text to show up in the reply.
  I'm working on it now.
 
 Great news, thanks!

Thanks for the feedback!

 Also: another confirmation for the revoked/expired key behaviour.  Emacs
 UI just reports:
 
   json-read-string: Wrong type argument: characterp, :json-eof
 
 Let me know if you need any samples of misbehaving mails.

Yeah, I've found some emails that are doing this as well.  I'm looking
in to what we can do to mitigate the problem, which I think is
ultimately a problem in GMime.  The json output should not be breaking,
though, so that I will definitely fix.

jamie.


pgpMreBm1wDBF.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Jameson Rollins]

On Fri, 04 Feb 2011 13:12:27 -0400, David Bremner brem...@unb.ca wrote:
 I reported some pretty weird experiences yesterday on IRC with replies
 not working. It turns out they were caused by having old versions of
 xapian installed in /usr/local (which I think might have caused some
 mixup with headers and libraries being different versions).
 
 The decryption is working ok for me now (aside from the issue I
 mentioned already with unknown encrypted+signed).
...
 tl;dr: yesterdays weirdness was not the fault of J. Rollins.

phew!  Although there remains the issue of replying to encrypted
messages, since notmuch-reply was not modified to decrypt messages
before replying.  I'm working on that now.

jamie.


pgp5qydZCj9Wm.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

new "crypto" branch providing full PGP/MIME support

[Darren McGuicken]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  wrote:
> Please test and provide feedback.  I would really like to see this
> series merged into the mainline for the next release, if at all
> possible.

Fan.  Tastic.  Thanks so much for this, incredible work!

Grabbed and testing now, the only slight niggle I see is that when I
reply to an encrypted mail I still get:

  On Thu, 03 Feb 2011 19:17:45 +, Someone  wrote:
  Non-text part: application/pgp-encrypted
  Non-text part: application/octet-stream

Can we pass the decrypted text to message mode on a reply?
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]

[Daniel Kahn Gillmor]

On 02/03/2011 03:34 PM, Jameson Rollins wrote:
> On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor  fifthhorseman.net> wrote:
>> You either want to fix this in your emacs config by putting your
>> fingerprint into mml2015-signers and setting mml2015-encrypt-to-self
>>
>> Or you want to set gpg's default-recipient-self option  (and
>> default-recipient option if you hold more than one secret key and want
>> to be sure it chooses the right one)
> 
> Actually, I think the gpg option we're looking for here is
> "encrypt-to".  "default-recipient-self" sets the recipient only if none
> other is specified.  I just set "encrypt-to " in my gpg.conf
> and it seems to do as expected (all encrypted messages are also
> encrypted to myself).

Yes, this is correct.  thanks for figuring that out, Jamie.

--dkg

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 

new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

Hi Micah--

just wanted to follow up on your points/questions:

On 02/03/2011 11:25 AM, micah anderson wrote:
> 1. I personally think notmuch-show-process-pgpmime should default to
> true

note that with it set to false, you can still M-RET (instead of RET) on
an item in the summary window to have it set for that particular view.

> 2. in-line pgp messages don't have any processing done on them. getting
> the mime-encoded processing work is a huge step and I'm happy that
> works, in-line can (and IMHO, should) come later

yeah, inline is a whole different thing, and much more difficult to
manage programmatically in the notmuch backend, i think.  I dealing with
inline signatures and encryption should be a job for the emacs (or vim
or whatever) frontend.

> 3. i'm not sure expired/revoked keys are handled properly - tested on a
> message that was encrypted by a key that was revoked and got "End of
> file during parsing"

when you say "encrypted by" do you mean "encrypted to"?  do you have
access to the corresponding secret key?

> 4. messages that I sent encrypted to someone are not also encrypted to
> myself, which means that a thread which contains my replies isn't able
> to decrypt my messages in that thread and results in a purple
> 'decryption error'. Perhaps this is an emacs UI tweak that needs to be
> made to get messages also encrypted to my own key?

this is an issue for the emacs message modes (or maybe for your gpg
configuration), not for notmuch.

You either want to fix this in your emacs config by putting your
fingerprint into mml2015-signers and setting mml2015-encrypt-to-self

Or you want to set gpg's default-recipient-self option  (and
default-recipient option if you hold more than one secret key and want
to be sure it chooses the right one)

> 5. unknown keys are represented in a long format,
> eg. '0x5585F58CC827A062' when most tools represent them just with their
> shortened keyid (in this case this one would be: 0xC827A062), is there a
> particular reason for this?

Short keyIDs are easily spoofable; believing anything based on a matched
short keyID is a mistake.  "unknown keys" themselves may or may not have
properly signed a message -- since we don't have the key handy, we don't
have a way of checking.

note that "unknown key" is different from "good signature from known key
but we do not know who controls the key"

> I recognize some people's keyids in the
> short form, but do not in the longform.

You can derive the short form from the long form by ignoring everything
but the last 8 hex characters.  But if you actually recognize the short
form, i'd expect you to have the relevant key in your keyring; is this
really a use case worth bothering with?

do you have a suggestion for how you think it should behave differently?

--dkg

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Thu, 03 Feb 2011 20:42:38 +, Darren McGuicken  wrote:
> On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  finestructure.net> wrote:
> > Please test and provide feedback.  I would really like to see this
> > series merged into the mainline for the next release, if at all
> > possible.
> 
> Fan.  Tastic.  Thanks so much for this, incredible work!

Thanks!  I hope people find it useful.

> Grabbed and testing now, the only slight niggle I see is that when I
> reply to an encrypted mail I still get:
> 
>   On Thu, 03 Feb 2011 19:17:45 +, Someone  wrote:
>   Non-text part: application/pgp-encrypted
>   Non-text part: application/octet-stream
> 
> Can we pass the decrypted text to message mode on a reply?

Yeah, some folks pointed this out on #notmuch this morning.  The
decryption is only happening in notmuch-show, but we need to do it in
notmuch-reply as well for the decrypted text to show up in the reply.
I'm working on it now.

jamie.

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

On 02/02/2011 08:18 PM, Jameson Rollins wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.

I have tested this, and am now using it.  I'm very happy with it.  I
support its inclusion in the mainline.

Thanks for doing this, Jamie.  This is excellent!

 less important stuff follows

I want to raise one (non-blocking) question about the decryption to see
if anyone has any suggestions:

If you do "notmuch show --format=json" on a PGP/MIME-encrypted plaintext
message, it emits the base message, which is structured like this:

1 ???multipart/encrypted
2  ??application/pgp-encrypted attachment
3  ??application/octet-stream inline [msg.asc]

with these patches, if you do "notmuch show --format=json --decrypt", it
emits this instead:

1 ???multipart/encrypted
2  ??text/plain inline

and it attaches an encstatus (and possibly sigstatus, if the message was
signed) to part 1.  I'll call this "method A".

There are other methods that could be used as well, and it's worth
making sure we've chosen one that we think is what we'll want in the
future.  here are two other proposals:

Method B:

1 ???multipart/encrypted
2  ??application/pgp-encrypted attachment
3  ??text/plain inline

That is, just replace part 3 (the actual encrypted body) with the
decrypted material.

This has the advantage that for single-part encrypted messages, the
structure and part numbers of the message remains the same as without
--decrypt.


Method C:

1 ??text/plain inline

That is, replace the entire multipart/encrypted with the decrypted material.

This avoids having an explicitly-labeled "multipart/encrypted" wrapper
around cleartext (which might be considered odd).  It would mean
attaching the encstatus and sigstatus directly to the decrypted part,
though.



I don't actually see any of these methods as being significantly better
than the others -- i think they all have some inherent ugliness.  So i'm
fine with going with method A as Jamie chose it and has it working.  But
i wanted to see if anyone had strong arguments in favor of the other
methods (or if there are other --decrypt methods we could use, for that
matter)

--dkg

-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: 

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor  wrote:
> On 02/03/2011 11:25 AM, micah anderson wrote:
> > 1. I personally think notmuch-show-process-pgpmime should default to
> > true
> 
> note that with it set to false, you can still M-RET (instead of RET) on
> an item in the summary window to have it set for that particular view.

This is also useful if you set notmuch-show-process-pgpmime and ever
come across a message that is causing crypto problems.  M-RET will
return you to the normal view.

> > 3. i'm not sure expired/revoked keys are handled properly - tested on a
> > message that was encrypted by a key that was revoked and got "End of
> > file during parsing"
> 
> when you say "encrypted by" do you mean "encrypted to"?  do you have
> access to the corresponding secret key?

I also seem to be noticing issues with revoked keys.  I'm looking in to
the issue.  If anyone else notices something similar, please do relay
your experience.

> > 4. messages that I sent encrypted to someone are not also encrypted to
> > myself, which means that a thread which contains my replies isn't able
> > to decrypt my messages in that thread and results in a purple
> > 'decryption error'. Perhaps this is an emacs UI tweak that needs to be
> > made to get messages also encrypted to my own key?
> 
> this is an issue for the emacs message modes (or maybe for your gpg
> configuration), not for notmuch.
> 
> You either want to fix this in your emacs config by putting your
> fingerprint into mml2015-signers and setting mml2015-encrypt-to-self
> 
> Or you want to set gpg's default-recipient-self option  (and
> default-recipient option if you hold more than one secret key and want
> to be sure it chooses the right one)

Actually, I think the gpg option we're looking for here is
"encrypt-to".  "default-recipient-self" sets the recipient only if none
other is specified.  I just set "encrypt-to " in my gpg.conf
and it seems to do as expected (all encrypted messages are also
encrypted to myself).

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new "crypto" branch providing full PGP/MIME support

[micah anderson]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins  wrote:
> Hi, all.  I have pushed a new branch called "crypto" to my notmuch
> repository [0].  This branch provides full support for PGP/MIME signed
> and encrypted messages, including emacs UI support.  It has been applied
> on top of cworth's current master (21e97c50).  It includes the
> following:
> 
> * David Edmondson's improved multipart handling patch series (cherry-picked)
> * Daniel Gillmor's PGP/MIME signature verification patch series 
> (cherry-picked)
> * my PGP/MIME decryption+verification patch series
> * a test suite for signature verification and decryption
> * emacs support for the above

Don't forget that you also included man page changes!

> Please test and provide feedback.  I would really like to see this
> series merged into the mainline for the next release, if at all
> possible.

I've really really really wanted this functionality, so I pulled this
right away and have been testing it, its really slick! I like how the
emacs UI gives you good visual feedback for different signature states
(I had red for a signature from Sebastian Spaeth because I did not have
the key; orange for when I obtained that key; and green for Jameson and
dkg's mails because I have exchanged keys with them and have full
validity for them; and purple for a decryption error). The minor delay
in opening a thread with signatures is not bad, and is to be expected.

And messages that are PGP/MIME encrypted are decrypted automatically,
wow, this is amazing. I enthusiastically support merging this into
mainline for the next release.

I have a couple points of feedback that I do not think should hold up
merging this work:

1. I personally think notmuch-show-process-pgpmime should default to
true

2. in-line pgp messages don't have any processing done on them. getting
the mime-encoded processing work is a huge step and I'm happy that
works, in-line can (and IMHO, should) come later

3. i'm not sure expired/revoked keys are handled properly - tested on a
message that was encrypted by a key that was revoked and got "End of
file during parsing"

4. messages that I sent encrypted to someone are not also encrypted to
myself, which means that a thread which contains my replies isn't able
to decrypt my messages in that thread and results in a purple
'decryption error'. Perhaps this is an emacs UI tweak that needs to be
made to get messages also encrypted to my own key?

5. unknown keys are represented in a long format,
eg. '0x5585F58CC827A062' when most tools represent them just with their
shortened keyid (in this case this one would be: 0xC827A062), is there a
particular reason for this? I recognize some people's keyids in the
short form, but do not in the longform.

6. this is awesome, huge thanks to everyone who has worked on this!

micah
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 

new crypto branch providing full PGP/MIME support

[Jameson Rollins]

Hi, all.  I have pushed a new branch called crypto to my notmuch
repository [0].  This branch provides full support for PGP/MIME signed
and encrypted messages, including emacs UI support.  It has been applied
on top of cworth's current master (21e97c50).  It includes the
following:

* David Edmondson's improved multipart handling patch series (cherry-picked)
* Daniel Gillmor's PGP/MIME signature verification patch series (cherry-picked)
* my PGP/MIME decryption+verification patch series
* a test suite for signature verification and decryption
* emacs support for the above

I have signed-off on all cherry-picked commits, as I have reviewed and
tested them extensively (I have been using them in my daily notmuch use
for many months now).  The multipart and sig-verification patches were
cherry-picked to get around a couple of accidental spurious commits in
their originating branches that would have needed to be reverted.

Please test and provide feedback.  I would really like to see this
series merged into the mainline for the next release, if at all
possible.

jamie.

[0] git://finestructure.net/notmuch


pgppeOSNllxAM.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

On 02/02/2011 08:18 PM, Jameson Rollins wrote:
 Hi, all.  I have pushed a new branch called crypto to my notmuch
 repository [0].  This branch provides full support for PGP/MIME signed
 and encrypted messages, including emacs UI support.

I have tested this, and am now using it.  I'm very happy with it.  I
support its inclusion in the mainline.

Thanks for doing this, Jamie.  This is excellent!

 less important stuff follows

I want to raise one (non-blocking) question about the decryption to see
if anyone has any suggestions:

If you do notmuch show --format=json on a PGP/MIME-encrypted plaintext
message, it emits the base message, which is structured like this:

1 └┬╴multipart/encrypted
2  ├╴application/pgp-encrypted attachment
3  └╴application/octet-stream inline [msg.asc]

with these patches, if you do notmuch show --format=json --decrypt, it
emits this instead:

1 └┬╴multipart/encrypted
2  └╴text/plain inline

and it attaches an encstatus (and possibly sigstatus, if the message was
signed) to part 1.  I'll call this method A.

There are other methods that could be used as well, and it's worth
making sure we've chosen one that we think is what we'll want in the
future.  here are two other proposals:

Method B:

1 └┬╴multipart/encrypted
2  ├╴application/pgp-encrypted attachment
3  └╴text/plain inline

That is, just replace part 3 (the actual encrypted body) with the
decrypted material.

This has the advantage that for single-part encrypted messages, the
structure and part numbers of the message remains the same as without
--decrypt.


Method C:

1 └╴text/plain inline

That is, replace the entire multipart/encrypted with the decrypted material.

This avoids having an explicitly-labeled multipart/encrypted wrapper
around cleartext (which might be considered odd).  It would mean
attaching the encstatus and sigstatus directly to the decrypted part,
though.



I don't actually see any of these methods as being significantly better
than the others -- i think they all have some inherent ugliness.  So i'm
fine with going with method A as Jamie chose it and has it working.  But
i wanted to see if anyone had strong arguments in favor of the other
methods (or if there are other --decrypt methods we could use, for that
matter)

--dkg



signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Daniel Kahn Gillmor]

Hi Micah--

just wanted to follow up on your points/questions:

On 02/03/2011 11:25 AM, micah anderson wrote:
 1. I personally think notmuch-show-process-pgpmime should default to
 true

note that with it set to false, you can still M-RET (instead of RET) on
an item in the summary window to have it set for that particular view.

 2. in-line pgp messages don't have any processing done on them. getting
 the mime-encoded processing work is a huge step and I'm happy that
 works, in-line can (and IMHO, should) come later

yeah, inline is a whole different thing, and much more difficult to
manage programmatically in the notmuch backend, i think.  I dealing with
inline signatures and encryption should be a job for the emacs (or vim
or whatever) frontend.

 3. i'm not sure expired/revoked keys are handled properly - tested on a
 message that was encrypted by a key that was revoked and got End of
 file during parsing

when you say encrypted by do you mean encrypted to?  do you have
access to the corresponding secret key?

 4. messages that I sent encrypted to someone are not also encrypted to
 myself, which means that a thread which contains my replies isn't able
 to decrypt my messages in that thread and results in a purple
 'decryption error'. Perhaps this is an emacs UI tweak that needs to be
 made to get messages also encrypted to my own key?

this is an issue for the emacs message modes (or maybe for your gpg
configuration), not for notmuch.

You either want to fix this in your emacs config by putting your
fingerprint into mml2015-signers and setting mml2015-encrypt-to-self

Or you want to set gpg's default-recipient-self option  (and
default-recipient option if you hold more than one secret key and want
to be sure it chooses the right one)

 5. unknown keys are represented in a long format,
 eg. '0x5585F58CC827A062' when most tools represent them just with their
 shortened keyid (in this case this one would be: 0xC827A062), is there a
 particular reason for this?

Short keyIDs are easily spoofable; believing anything based on a matched
short keyID is a mistake.  unknown keys themselves may or may not have
properly signed a message -- since we don't have the key handy, we don't
have a way of checking.

note that unknown key is different from good signature from known key
but we do not know who controls the key

 I recognize some people's keyids in the
 short form, but do not in the longform.

You can derive the short form from the long form by ignoring everything
but the last 8 hex characters.  But if you actually recognize the short
form, i'd expect you to have the relevant key in your keyring; is this
really a use case worth bothering with?

do you have a suggestion for how you think it should behave differently?

--dkg



signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Darren McGuicken]

On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins 
jroll...@finestructure.net wrote:
 Please test and provide feedback.  I would really like to see this
 series merged into the mainline for the next release, if at all
 possible.

Fan.  Tastic.  Thanks so much for this, incredible work!

Grabbed and testing now, the only slight niggle I see is that when I
reply to an encrypted mail I still get:

  On Thu, 03 Feb 2011 19:17:45 +, Someone foo@bar wrote:
  Non-text part: application/pgp-encrypted
  Non-text part: application/octet-stream

Can we pass the decrypted text to message mode on a reply?


pgpvaIMNrRYvK.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: new crypto branch providing full PGP/MIME support

[Jameson Rollins]

On Thu, 03 Feb 2011 20:42:38 +, Darren McGuicken 
mailing-notm...@fernseed.info wrote:
 On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins 
 jroll...@finestructure.net wrote:
  Please test and provide feedback.  I would really like to see this
  series merged into the mainline for the next release, if at all
  possible.
 
 Fan.  Tastic.  Thanks so much for this, incredible work!

Thanks!  I hope people find it useful.

 Grabbed and testing now, the only slight niggle I see is that when I
 reply to an encrypted mail I still get:
 
   On Thu, 03 Feb 2011 19:17:45 +, Someone foo@bar wrote:
   Non-text part: application/pgp-encrypted
   Non-text part: application/octet-stream
 
 Can we pass the decrypted text to message mode on a reply?

Yeah, some folks pointed this out on #notmuch this morning.  The
decryption is only happening in notmuch-show, but we need to do it in
notmuch-reply as well for the decrypted text to show up in the reply.
I'm working on it now.

jamie.



pgpU7r4oBBd3U.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

always encrypting messages to self [was: Re: new crypto branch providing full PGP/MIME support]

[Daniel Kahn Gillmor]

On 02/03/2011 03:34 PM, Jameson Rollins wrote:
 On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor 
 d...@fifthhorseman.net wrote:
 You either want to fix this in your emacs config by putting your
 fingerprint into mml2015-signers and setting mml2015-encrypt-to-self

 Or you want to set gpg's default-recipient-self option  (and
 default-recipient option if you hold more than one secret key and want
 to be sure it chooses the right one)
 
 Actually, I think the gpg option we're looking for here is
 encrypt-to.  default-recipient-self sets the recipient only if none
 other is specified.  I just set encrypt-to mykeyid in my gpg.conf
 and it seems to do as expected (all encrypted messages are also
 encrypted to myself).

Yes, this is correct.  thanks for figuring that out, Jamie.

--dkg



signature.asc
Description: OpenPGP digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

new "crypto" branch providing full PGP/MIME support

[Jameson Rollins]

Hi, all.  I have pushed a new branch called "crypto" to my notmuch
repository [0].  This branch provides full support for PGP/MIME signed
and encrypted messages, including emacs UI support.  It has been applied
on top of cworth's current master (21e97c50).  It includes the
following:

* David Edmondson's improved multipart handling patch series (cherry-picked)
* Daniel Gillmor's PGP/MIME signature verification patch series (cherry-picked)
* my PGP/MIME decryption+verification patch series
* a test suite for signature verification and decryption
* emacs support for the above

I have signed-off on all cherry-picked commits, as I have reviewed and
tested them extensively (I have been using them in my daily notmuch use
for many months now).  The multipart and sig-verification patches were
cherry-picked to get around a couple of accidental spurious commits in
their originating branches that would have needed to be reverted.

Please test and provide feedback.  I would really like to see this
series merged into the mainline for the next release, if at all
possible.

jamie.

[0] git://finestructure.net/notmuch
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: