Re: Protection mail at rest
On Tue, Jun 03, 2008 at 04:37:20PM -0400, Eric Cronin wrote: On Jun 3, 2008, at 11:51 AM, Adam Aviv wrote: Depending on the level of protection you want, you could just add a script to your .forward to encrypt your email before delivery using PGP/GPG. However, this will leave the headers in the clear, so you will likely want to create an entirely new envelope for the message with the original message encrypted as the body or an attachment. Does anybody have a recipe for this first mode handy? plain text e- mails seem simple enough, but there needs to be a bit of MIME unwrapping and rewrapping to correctly handle attachments so that the client sees/decrypts them correctly I think. I've searched from time to time and never found a good HowTo... S/MIME supports enveloped MIME objects, if PGP does not work out for MIME entities, you could try that. S/MIME is natively supported in Thunderbird, Apple Mail, and similar MUAs. -- Viktor. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
On Tue, Jun 3, 2008 at 4:37 PM, Eric Cronin [EMAIL PROTECTED] wrote: On Jun 3, 2008, at 11:51 AM, Adam Aviv wrote: Depending on the level of protection you want, you could just add a script to your .forward to encrypt your email before delivery using PGP/GPG. However, this will leave the headers in the clear, so you will likely want to create an entirely new envelope for the message with the original message encrypted as the body or an attachment. Does anybody have a recipe for this first mode handy? plain text e-mails seem simple enough, but there needs to be a bit of MIME unwrapping and rewrapping to correctly handle attachments so that the client sees/decrypts them correctly I think. I've searched from time to time and never found a good HowTo... Thanks, Eric I have written a script that does that in python, as part of the email handling for the project. It encrypts each message part separately and then construct a new email with each encrypted part as the payload of a new MIME multipart message. Contained within is also the encrypted session key, and necessary info to reconstruct. On the client side, the message can then be unwrap, decrypted, and the original email reconstructed. Or the client can request just the headers, the body, or any attachment (becomes iffy with a combination of 'text/plain' and 'text/html' content type) instead of the entire message. Depending on how you want to do the wrapping (if you want to meet some optimization like headers can be requested before the rest of the message), then the simplest approach is, if a MIME multipart message, to work with each part individually and encrypt. I decided to encrypt the headers for each part separatly from the payload, so that the client can see what the message contains before actually decrypting it. But, different approaches can be taken depending on the end goal (i.e. just encrypt the whole thing into a glob, and download the glob and decrypt it on the client directly into a mbox or maildir that is locally served). adam -- Adam Aviv Ph. D. Candidate Computer and Information Science University of Pennsylvania - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
Victor Duchovni [EMAIL PROTECTED] writes: On Tue, Jun 03, 2008 at 04:37:20PM -0400, Eric Cronin wrote: On Jun 3, 2008, at 11:51 AM, Adam Aviv wrote: Depending on the level of protection you want, you could just add a script to your .forward to encrypt your email before delivery using PGP/GPG. However, this will leave the headers in the clear, so you will likely want to create an entirely new envelope for the message with the original message encrypted as the body or an attachment. Does anybody have a recipe for this first mode handy? plain text e- mails seem simple enough, but there needs to be a bit of MIME unwrapping and rewrapping to correctly handle attachments so that the client sees/decrypts them correctly I think. I've searched from time to time and never found a good HowTo... S/MIME supports enveloped MIME objects, if PGP does not work out for MIME entities, you could try that. S/MIME is natively supported in Thunderbird, Apple Mail, and similar MUAs. Actually, PGP/MIME uses the same high-level mechanism to wrap MIME objects as S/MIME does: http://www.ietf.org/rfc/rfc1847.txt The PGP/MIME description is in: http://www.ietf.org/rfc/rfc3156.txt Specification wise both should work equally well, but implementation quality may differ. What is often overlooked is that the e-mail envelope (including the Subject: header field) is not protected or even encrypted under RFC 3156 unless you forward the entire e-mail as a message/rfc822 MIME part within the PGP/MIME (or S/MIME) message. Interoperability of that has historically been poor, but the more modern MUAs should handle it today. Writing a .forward proxy that wraps incoming e-mails into PGP/MIME encrypted message/rfc822 attachments should be simple, probably simpler than PGP/MIME wrapping all the individual MIME parts in the incoming e-mail. /Simon - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Fwd: Protection mail at rest
I agree with you, that this is not nearly fast enough. However, this is 10 times faster then our original results, where we were searching 100 emails in about the same amount of time. With production code, some more optimization, esp. client side optimizations (i.e. message caching when possible), and increased parallelism, it may just be possible to reach the 4x faster searches a heavy user like yourself would need. I am just not a good enough coder to write it myself, but I believe that it can be done. adam On Mon, Jun 2, 2008 at 10:42 PM, Greg Black [EMAIL PROTECTED] wrote: On 2008-06-02, Adam Aviv wrote: I recently implemented SSARES directly in python and also added parallelism to the searching. We can now search the a large inbox (1000+) messages in about 2-4 minutes. Not to rain on your parade, but 1,000 messages is *not* a large inbox and 2 to 4 minutes is a very long time to wait. You'd need to make this two orders of magnitude faster before it would have a hope of being interesting. (And for me, it would have to be at least four orders of magnitude faster before I could consider it to be useful.) Greg -- Adam Aviv Ph. D. Candidate Computer and Information Science University of Pennsylvania - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Fwd: Protection mail at rest
On 2008-06-02, Adam Aviv wrote: I recently implemented SSARES directly in python and also added parallelism to the searching. We can now search the a large inbox (1000+) messages in about 2-4 minutes. Not to rain on your parade, but 1,000 messages is *not* a large inbox and 2 to 4 minutes is a very long time to wait. You'd need to make this two orders of magnitude faster before it would have a hope of being interesting. (And for me, it would have to be at least four orders of magnitude faster before I could consider it to be useful.) Greg - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Fwd: Protection mail at rest
Greg Black wrote: On 2008-06-02, Adam Aviv wrote: I recently implemented SSARES directly in python and also added parallelism to the searching. We can now search the a large inbox (1000+) messages in about 2-4 minutes. Not to rain on your parade, but 1,000 messages is *not* a large inbox and 2 to 4 minutes is a very long time to wait. You'd need to make this two orders of magnitude faster before it would have a hope of being interesting. (And for me, it would have to be at least four orders of magnitude faster before I could consider it to be useful.) Thunderbird, at least, downloads imap mail locally for searching. So all I need is the automatic public key encryption on the server side (no searching). Is there such an application already? -- Nate - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Fwd: Protection mail at rest
[Moderator's note: Please don't top post. --Perry] Depending on the level of protection you want, you could just add a script to your .forward to encrypt your email before delivery using PGP/GPG. However, this will leave the headers in the clear, so you will likely want to create an entirely new envelope for the message with the original message encrypted as the body or an attachment. But then you will need a thunderbird extension to unwrap the encrypted original email out of the body, and store the message locally unencrypted so that you can search. The problem comes when you start accessing your email from multiple locations. At one place you have built up a large cache of unencrypted messages and you can use them in the normal way, but when you access from another machine or a blackberry, the lack of cache will greatly hinder your performance. This is the reason we wanted to not only have the client cache capability to searching, but also a server side mechanism to compensate when accessing from multiple locations. adam On Tue, Jun 3, 2008 at 11:34 AM, Nate Lawson [EMAIL PROTECTED] wrote: Greg Black wrote: On 2008-06-02, Adam Aviv wrote: I recently implemented SSARES directly in python and also added parallelism to the searching. We can now search the a large inbox (1000+) messages in about 2-4 minutes. Not to rain on your parade, but 1,000 messages is *not* a large inbox and 2 to 4 minutes is a very long time to wait. You'd need to make this two orders of magnitude faster before it would have a hope of being interesting. (And for me, it would have to be at least four orders of magnitude faster before I could consider it to be useful.) Thunderbird, at least, downloads imap mail locally for searching. So all I need is the automatic public key encryption on the server side (no searching). Is there such an application already? -- Nate -- Adam Aviv Ph. D. Candidate Computer and Information Science University of Pennsylvania - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
On Jun 3, 2008, at 11:51 AM, Adam Aviv wrote: Depending on the level of protection you want, you could just add a script to your .forward to encrypt your email before delivery using PGP/GPG. However, this will leave the headers in the clear, so you will likely want to create an entirely new envelope for the message with the original message encrypted as the body or an attachment. Does anybody have a recipe for this first mode handy? plain text e- mails seem simple enough, but there needs to be a bit of MIME unwrapping and rewrapping to correctly handle attachments so that the client sees/decrypts them correctly I think. I've searched from time to time and never found a good HowTo... Thanks, Eric PGP.sig Description: This is a digitally signed message part
Re: Protection mail at rest
A more recent version, which appeared at ACSAC in December 2007 can be found at: http://www1.cs.columbia.edu/~angelos/Papers/2007/SSARES_ACSAC.pdf Since then, the student primarily working on this(*) has improved performance to the point of being able to search a couple of email messages per minute or so, with further scope for improvement. The very large storage overhead remains, but can probably be reduced by half or so. (*) Adam Aviv; he was an undergrad at Columbia, now pursuing his Phd with Matt Blaze at UPenn... -Angelos On Jun 1, 2008, at 8:53 AM, Perry E. Metzger wrote: Leichter, Jerry [EMAIL PROTECTED] writes: Does anyone know of existing work in this area? SSARES: Secure Searchable Automated Remote Email Storage by Keromytis et al, http://www1.cs.columbia.edu/~angelos/Papers/2006/SSARES_short.pdf There is probably other work out there. In some small part, this also looks like the problem that Matt Blaze's CFS addressed, though in that case it was to deal with untrusted remote file servers rather than email servers. Perry - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
At 11:36 AM -0400 6/1/08, Ivan Krstiç wrote: The easiest thing for people who _do_ care is still running their own mail server. Fully agree. You're in control, all the way to root of the box. The emergence of reasonably priced VM hosting providers (e.g. slicehost.com) makes it fairly uncomplicated, modulo initial setup. And, if you want to host on FreeBSD instead of Linux, see http://www.rootbsd.net/. Same price, good service. --Paul Hoffman, Director --VPN Consortium - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
| There's an option 2b that might be even more practical: an S/MIME or | PGP/MIME forwarder. That is, have a trusted party receive your mail, | but rather than forwarding it intact encrypt it and then forward it to | your favorite IMAP provider. Excellent idea! I like it. Of course, it's another piece of a distributed solution that you need to keep running. It would make for an interesting third-party service. (On the surface, letting a third party run this for you seems hazardous, but as always your stuff is exposed on the way to the forwarder whatever you do A forwarded like this as a pre-packaged EC2 VM, perhaps? -- Jerry - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Fwd: Protection mail at rest
I recently implemented SSARES directly in python and also added parallelism to the searching. We can now search the a large inbox (1000+) messages in about 2-4 minutes. Technically, this could be done on a large scale and be practical, since my implementation is not fully optimized nor free of bugs. The implementation is available on my web site, http://fling.seas.upenn.edu/~aviv/wiki/index.php?n=SSARESApp.SSARESApp as well as some current benchmarks. I am not a cryptographer (so implementation may not be perfect), nor do I guarantee that the code doesn't have bugs. This is grad-ware and for research purposes only. Yet, as a proof of concept, feel free to play around with it and let me know what you think. I can supply more python scripts for searching and what not if anyone wants. thanks, adam On Sun, Jun 1, 2008 at 8:09 PM, Angelos D. Keromytis [EMAIL PROTECTED] wrote: A more recent version, which appeared at ACSAC in December 2007 can be found at: http://www1.cs.columbia.edu/~angelos/Papers/2007/SSARES_ACSAC.pdf Since then, the student primarily working on this(*) has improved performance to the point of being able to search a couple of email messages per minute or so, with further scope for improvement. The very large storage overhead remains, but can probably be reduced by half or so. (*) Adam Aviv; he was an undergrad at Columbia, now pursuing his Phd with Matt Blaze at UPenn... -Angelos On Jun 1, 2008, at 8:53 AM, Perry E. Metzger wrote: Leichter, Jerry [EMAIL PROTECTED] writes: Does anyone know of existing work in this area? SSARES: Secure Searchable Automated Remote Email Storage by Keromytis et al, http://www1.cs.columbia.edu/~angelos/Papers/2006/SSARES_short.pdf There is probably other work out there. In some small part, this also looks like the problem that Matt Blaze's CFS addressed, though in that case it was to deal with untrusted remote file servers rather than email servers. Perry - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED] -- Adam Aviv Ph. D. Candidate Computer and Information Science University of Pennsylvania -- Adam Aviv Ph. D. Candidate Computer and Information Science University of Pennsylvania - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
On Fri, May 30, 2008 at 03:04:34PM -0400, Leichter, Jerry wrote: 1. Client only. The client, whenever it sees a new message, (a) downloads it; (b) encrypts it using a secret key; (c) stores the encrypted version back on the server; (d) deletes the unencrypted version. The client can put the encrypted messages in a different folder, or it can mark them with a header line. 2. Server-assisted. The client gives the server its public key. When a message arrives at the server, the server (a) generates a session key; (b) encrypts the message using the session key; (c) encrypts the session key with the client's public key; (d) adds a header containing the encrypted session key to the encrypted message; (e) stores the encrypted message. The necessary work for the client is obvious. 3. The server that stores your mail is not the first one to receive it. It is just the storage layer. A previous non-storing server, encrypts the mail and *then* forwards it to the store. In each case, one would probably chose some headers to encrypt separately - e.g., the subject - so that one could more easily pull them out without decrypting the whole message. S/MIME does not encrypt any headers. It only encrypts the payload. Some S/MIME applications don't leave any useful headers in the outer message, others leave the sender and subject in the clear. Does anyone know of existing work in this area? Take PGP Universal gateway and turn-it inside-out. Clear mail on the Internal encrypted mail on the intranet between the gateway and the mail store. Take a vanity domain, run an encryption gateway, forward everything to to an ESP. The ESP's search engine will not do you much good with encrypted mail, so indexing is up to your IMAP client, if it can cache/index decrypted content. Not much demand for this yet, so I don't expect mature offerings any time soon. We'd have to build a boutique service for cipher-punks. -- /\ ASCII RIBBON NOTICE: If received in error, \ / CAMPAIGN Victor Duchovni please destroy and notify X AGAINST IT Security, sender. Sender does not waive / \ HTML MAILMorgan Stanley confidentiality or privilege, and use is prohibited. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Re: Protection mail at rest
On Jun 1, 2008, at 12:07 AM, Victor Duchovni wrote: Not much demand for this yet, so I don't expect mature offerings any time soon. We'd have to build a boutique service for cipher-punks. I doubt there'll ever be much demand. The tinfoil hat crowd will be bothered by the n-1 hops (and however many Narus boxes in between) being traversed unencrypted, while most everyone else seemingly doesn't care and uses GMail/Hotmail/YahooMail, forfeiting any expectation of privacy right from the start. The easiest thing for people who _do_ care is still running their own mail server. The emergence of reasonably priced VM hosting providers (e.g. slicehost.com) makes it fairly uncomplicated, modulo initial setup. -- Ivan Krstić [EMAIL PROTECTED] | http://radian.org - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
Protection mail at rest
At one time, mail delivery was done to the end-user's system, and all mail was stored there. These days, most people find it convenient to leave their mail on a IMAP server: It can be accessed from anywhere, it can be on a system kept under controlled conditions (unlike a laptop), and so on. In fact, most people these days - even the technically savvy - not only leave their mail on an IMAP server, but let some provider deal with the headaches of maintaining that server. So, most people's mail spends most of its life sitting on a disk owned, managed, and controlled by some third party, whose responsibilities, not to mention abilities, for keeping that stuff secure are unclear to say the least. On top of that, the legal protections for data held by a third party are limited. We have mechanisms for providing end-to-end encryption of messages. Messages sent using, say, S/MIME are encrypted on the IMAP server just as they are out on the net. But this only helps for mail exchanged between correspondents who both choose to use it. Suppose I ask for a simpler thing: That my mail, as stored in my IMAP server, spends most of its life encrypted, inaccessible even to whoever has access to the physical media on which the server stores its mail. Now, this is a funny goal. If mail arrives unencrypted, anyone with access to the data stream can copy it and do what they like. It will inevitably be buffered, even likely stored on a disk, in the raw, unencrypted form. We explicitly leave dealing with this out of the equation - only end-to-end encryption can deal with it. Here are two ways of implementation something in this direction: 1. Client only. The client, whenever it sees a new message, (a) downloads it; (b) encrypts it using a secret key; (c) stores the encrypted version back on the server; (d) deletes the unencrypted version. The client can put the encrypted messages in a different folder, or it can mark them with a header line. 2. Server-assisted. The client gives the server its public key. When a message arrives at the server, the server (a) generates a session key; (b) encrypts the message using the session key; (c) encrypts the session key with the client's public key; (d) adds a header containing the encrypted session key to the encrypted message; (e) stores the encrypted message. The necessary work for the client is obvious. In each case, one would probably chose some headers to encrypt separately - e.g., the subject - so that one could more easily pull them out without decrypting the whole message. Obviously, approach 2 greatly decreases the time that messages may hang around unencrypted; but approach 1 can be implemented without any cooperation from the IMAP provider, which allows it to be rolled out even for those who use the large providers without having Google and Hotmail and Yahoo! buy into it. Does anyone know of existing work in this area? -- Jerry - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to [EMAIL PROTECTED]
