Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
-- From: Kevin W. Wall kevin.w.w...@gmail.com Subject: Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI So given these limited choices, what are the best options to the questions I posed in my original post yesterday? As Peter mentioned, we want to give web app developers something that will work out-of-the-box. It isn't difficult to implement CMAC and CTR modes in pure Java. The NIST specs for CMAC and CTR are plenty clear. You'll be looking for the AES/ECB/NoPadding option. From there use update it returns a byte []. I've used the standard JCE implementation in this way to implement unsupported modes before, it works. Joe - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
On 17/09/2009 21:42, David Wagner wrote: Kevin W. Wall wrote: So given these limited choices, what are the best options to the questions I posed in my original post yesterday? Given these choices, I'd suggest that you first encrypt with AES-CBC mode. Then apply a message authentication code (MAC) to the whole ciphertext (including the IV). You then send the ciphertext followed the MAC digest. SHA1-HMAC would be a reasonable choice of algorithm for message authentication. I have to add vote+1 on this selection. For various reasons, today's safe choice seems to be: * CBC * AES-128 * HMAC-SHA-1 on the outside of the ciphertext What is left is padding so that the message is clearly deliminated. I suggest you treat this as a software engineering thing, not a crypto thing, and make sure that you have a length in your packet layout so that it is totally clear what is the packet and what is not. If you want to see such a design exercise, following Dave's prescription, have a look at SDP1 which Zooko and I did a few years back. http://www.webfunds.org/guide/sdp/sdp1.html http://www.webfunds.org/guide/sdp/ It's a straight forward secret-key encrypted packet layout. It has one novelty in it, which is how it solves the padding / IV issues. Other than that it should be boring. iang PS: you are on the right track in trying to avoid any sensitivity to JCE. As long as you can design your layout without any dependency on JCE it should work. JCE is basically a slock design that was put in place for market- and crypto-control reasons, it has no place in software engineering. I speak from experience, I managed the Cryptix project, which was the first Java crypto engine. PPS: you haven't said enough about the application (or I missed it) to be able to comment on keys. Generally, try to separate the protocol around the key: every good protocol divides into two parts, the first of which says to the second, trust this key completely. Software engineering ... http://iang.org/ssl/h2_divide_and_conquer.html - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
Peter Gutmann wrote: David Wagner d...@cs.berkeley.edu writes: (You could replace AES-CMAC with SHA1-HMAC, but why would you want to?) The answer to that depends on whether you need to support an existing base of crypto software and hardware. Even though (in this case) it's a new standard, it still requires support from the underlying crypto libraries. If little or none of those do AES-CMAC yet (I don't think Windows CryptoAPI does, only very recent versions of OpenSSL do... it's not looking good) then you'd want to stick with HMAC-SHA1. (Forestalling the inevitable but developers can implement AES-CMAC themselves from raw AES that I'm sure someone will follow up with, the target audience for this is web application developers, not cryptographers, so you need to give them something that works as required out of the box). Peter has hit the proverbial nail right on the head. I apologize if I did not make this clear in my original post, but but one goal of OWASP ESAPI is to not require a whole lot of dependencies. In the context of Java crypto, this means that we *ideally* would like to have no other dependency other than the SunJCE, that is, the Sun reference implementation for JCE. Recently we decided to required Java 6, but even with that, our choices for cipher algorithms, cipher modes, and padding schemes are limited. For SunJCE, this is what we have available to choose from: Supported symmetric cipher algorithms: AES, DES, DESede, Blowfish, RC2, ARCFOUR Supported cipher modes: CBC, CFB, CTR, CTS, ECB, OFB, PCBC Supported padding schemes: NoPadding, PKCS5Padding ISO10126Padding OAEPPadding, OAEPWithdigestAndmgfPadding PKCS1Padding, SSL3Padding (Obviously some of these padding schemes such as OAEP are not suitable with symmetric ciphers. Or at least I don't think they are.) So given these limited choices, what are the best options to the questions I posed in my original post yesterday? As Peter mentioned, we want to give web app developers something that will work out-of-the-box. For that reason we don't even want to require that developers use some other JCE provider like Bouncy Castle, Cryptix, IAIK, etc. even though they may have more suitable cipher modes or padding schemes. Lastly, I wanted to respond to one other point that David Wagner brought up in an earlier reply: Advice: Provide one mode, and make it secure. Try to avoid configurability, because then someone will choose a poor configuration. There's a few reasons for supporting different configurations here. One is the backward compatibility with previous ESAPI versions, and the second is to support legacy cases. My experience at my day job is that no one really changes the crypto defaults anyway if you make it easy enough for them to use. The main exception is if they have to be compatible with something else such as some 3rd party vendor software that uses a different mode, etc. What we can try to do is provide adequate warning in documentation or in logged warnings if one tries to use anything other then the default. BTW, thanks to all who replied. I've learned quite a bit from all your responses, but it looks like I have a lot of research to do before I understand everything that all of you said. Regards, -kevin -- Kevin W. Wall The most likely way for the world to be destroyed, most experts agree, is by accident. That's where we come in; we're computer professionals. We cause accidents.-- Nathaniel Borenstein, co-creator of MIME - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
Kevin W. Wall kevin.w.w...@gmail.com writes: (Obviously some of these padding schemes such as OAEP are not suitable with symmetric ciphers. Or at least I don't think they are.) You'd be surprised at what JCE developers will implement just because they can, and what therefore gets used by application developers. I've seen RSA-CBC used on more than one occasion. (No, that's not a typo, RSA in CBC mode. The app developers wondered why it was so slow). Peter. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
On Sep 17, 2009, at 1:20 AM, Peter Gutmann wrote: Kevin W. Wall kevin.w.w...@gmail.com writes: (Obviously some of these padding schemes such as OAEP are not suitable with symmetric ciphers. Or at least I don't think they are.) You'd be surprised at what JCE developers will implement just because they can, and what therefore gets used by application developers. I've seen RSA-CBC used on more than one occasion. (No, that's not a typo, RSA in CBC mode. The app developers wondered why it was so slow). Interesting. It sounds as if the JCE developers have gone from one extreme to another. I no longer remember the details, but a number of years back, in a project I was involved with, we needed to implement some particular (sane) combination of a cipher and a mode. JCE at the time had a fixed list of combinations it was willing to let you use; ours wasn't on that list. ECB wasn't an accepted mode, so it wasn't easy to build your own mode out of what the API provided. -- Jerry - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
Hi all, I was referred to this site by a former colleague who thought this is something that someone with professional cryptanalysis experience should comment on. Also, I apologize in advance for the length of this post (especially since it's my first one). Just trying to be thorough. I have been working with Jeff Williams, Jim Manico and others on the OWASP ESAPI Java implementation (http://www.esapi.org). Work is underway for the 2.0 release. I am re-implementing the somewhat broken symmetric encryption / decryption mechanisms implemented in the earlier 1.x versions that only supported ECB cipher mode, restricted encryption and decryption operations to the use of a single encryption key, and used whatever the default padding scheme was for whatever JCE provider that happened to be used. As such, this is still very much a work in progress, but rather than risk fouling up encryption in the 2.0 release as well, I wanted to gather some input that we are on the right track. Since this work is for a free open source project (developed under the BSD license), I'm hoping that one of you on this list will take some time to address my questions. In OWASP ESAPI Java 2.0, I started out by deprecating the older methods that only supported ECB mode and used the provider's default padding scheme (which can vary, depending on JCE provider). The new default for the new encryption / decryption methods is to be 128-bit AES/CBC/PKCS5Padding and use of a random IV. (There is some thought on my part to not allow different cipher modes or padding schemes, but I go back and forth on this. (Certainly if someone tries to use something like OFB we'll at least probably long a warning that there's some chance that the same IV could be reused even though it is chosen randomly.) But for the remainder of this discussion, you can assume some suitably strong cipher algorithm and key size = 128 bits with encryption using CBC cipher mode and PKCS5Padding. (Eventually we may allow others, but for now, that is probably sufficient and certainly a big improvement over only supporting ECB mode.) Based on experience at my day job, what I have found is when you try to decrypt something using PKCS5Padding using the wrong secret key, about 99.9% of the time you will get a BadPaddingException in Java and about the other .1% of the time you just get random garbage back for the plaintext. This is bad because if one is performing _manual_ key change operations (which many small IT shops using OWASP ESAPI will likely do for PCI DSS compliance reasons), then one could end up storing the resulting (incorrect) plaintext value and not discovering the error until much further downstream making and/or at a much later time. This makes the error very hard to troubleshoot as it typically will a long ways away in both code space and timeline away from the true root cause of the problem. I would like to be able to support such key change operations for ESAPI in a way that one has a much higher probability of detecting that decryptions using the incorrect secret key that do NOT result in a BadPaddingException have a much higher probability of being detected. Unfortunately, since this is generally reusable class library I can make no assumptions about the original plaintext. Specifically, it might not be text in the conventional sense at all, but rather it could be something like another random secret key that someone encrypted, etc. (key wrapping notwithstanding, but it is not currently supported in ESAPI 2.0). For ESAPI Java 2.0, I've created a serializable CipherText class that contains everything one generally needs to know to perform the decryption operation. That is, the CipherText object contains things like the the specification cipher transformation used to encrypt it, the IV used (unless ECB mode), and the raw ciphertext value as a byte array. My first thought was to also include a digital signature that was created along the lines one of these (note: here 'DSig(msg)' includes both the hashing and private key signature operations and probably would use DSA since it is not subject to any US export regulations): DSig( IV + secretKey ) // I explain below why IV is included or DSig( IV + plaintext ) // '+' is just the usual byte concatenation signed by the encrypter's private key. The decrypting side would then validate the digital signature using the encrypter's public key and only use the plaintext value if the digital signature was valid. If I were to do this, I might also add the encrypter's identity or alias as part of the CipherText class, although in most of the simple cases, it should be obvious based on from whom one is receiving the encrypted data. However, the main problem with using digital signatures is the performance hit. In my day job, I have made two observations involving encryption: 1) If you wish to ensure that application developers use solid algorithms such as AES, you must ensure that the
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
Advice: if you're creating something for general-purpose use, at a minimum make sure it provides authentication, integrity, *and* confidentiality. A reasonable choice might be Encrypt-then-Authenticate where you first encrypt with AES-CBC, then append a AES-CMAC message authentication code on the ciphertext (including the IV). As a bonus, this will detect decrypting with the wrong key. Use key separation -- with a pseudorandom function -- to derive the encryption key and authentication key from the master crypto key supplied by the user: e.g., Encryptkey = AES(K, all-zeros), Authkey = AES(K, all-ones). (You could replace AES-CMAC with SHA1-HMAC, but why would you want to?) (From a cryptotheory point of view, what you want is a mode of operation that meets IND-CCA2 /\ INT-CTXT, or at least IND-CCA2 /\ INT-PTXT.) Advice: Provide one mode, and make it secure. Try to avoid configurability, because then someone will choose a poor configuration. Suggestion: Provide documentation to warn the programmer against using a password (or something based on a password) as the crypto key. Provide support for PBKDF2, with a reasonable default choice of parameters and appropriate warnings in the documentation, for those who absolutely must use a password-derived crypto key. Recommendation: Read the book Practical Cryptography by Ferguson and Schneier, or at least the chapters on message security. It's the best source I know to teach you some of the crypto-engineering practicum. Kevin W. Wall wrote: I have considered using an HMAC-SHA1 as a keyless MIC to do this, using something like: MIC = HMAC-SHA1( nonce, IV + secretKey ) or MIC = HMAC-SHA1( nonce, IV + plaintext ) and then also include the random nonce and the MIC itself in the CipherText class so it can be validated later by the decrypter, with the understanding that the plaintext resulting from the decryption operation should only be used if the MIC can be properly validated. (Probably an exception would be thrown if the dsig was not valid so there would be no choice.) I would not recommend either of these. It's better to use a MAC as I suggest at the top of this email. Whatever you do, don't choose your second MIC option: if the plaintext comes from a low-entropy space, it leaks the value of the plaintext (the plaintext can be recovered by brute-force search over all possible plaintexts). - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
-- From: Kevin W. Wall kevin.w.w...@gmail.com Subject: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI The new default for the new encryption / decryption methods is to be 128-bit AES/CBC/PKCS5Padding and use of a random IV. That's a good solution, except for the missing MAC, considering the environment I would suggest hard coding it, there's really no reason to offer options. MIC = HMAC-SHA1( nonce, IV + secretKey ) or MIC = HMAC-SHA1( nonce, IV + plaintext ) MIC = HMAC-SHA1( IV, plaintext ) So, having provided all of that background, in summary, here are my three questions: 1) Is using either of these MIC calculations cryptographically secure? 2) If answer to #1, is 'yes', which one is safer / more secure? 3) If answer to #1 is 'no', do you have any suggestions less computationally expensive then digital signatures that would allow us to detect attempts to decrypt with the incorrect secret key and/or an adversary attempting to alter the IV prior to the decryption. You are looking at the correct type of construct, the solution for your problem is known as a Message Authentication Code or MAC. The biggest concerns are what are you MACing, and that it must use a second key. There are a number of solutions. The first part of the solution is to store an additional key, you're storing 128-bits hopefully securely store 256, and split it, trust me you'll thank yourself when the security issues are investigated. The second key is for the MAC algorithm. Since you already have CBC available, my first suggestion would be CBC-MAC (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), it has good strong security proofs behind it, and is fast. Apply the MAC algorithm, prepend the MAC value to the plaintext (just because indexing is easier this way), then encrypt to ciphertext, store. To decrypt, retrieve, decrypt the ciphertext, parse out the MAC value and the plaintext, run MAC algorithm on plaintext to find MAC2, check MAC == MAC2 . This will give you a failure rate of 1/2^128 or something on the order of 0.3%, you are more likely to have the system burst into flame than see a false positive on this. Overall, this will reduce your speed to 50% of the current. You might also want to take a look at Poly1305, it is faster. As you noted HMAC is also available, but I would recommend against using SHA-1 for anything, it simply raises too many questions that will take too long to answer. The secure hashes available are simply not as fast unless you have bare-metal coding ability which Java really doesn't like. The other, and arguably better, option would be a combined mode. The good combined modes are a legal minefield, so using them for an open project is difficult. It is claimed that GCM and EAX are public domain, but I'm more inclined to recommend the conservative approach and avoid them. While there has been no concern raised by cryptanalysts about CBC with CBC-MAC, to many laypeople it doesn't look good. So, for purely politic reasons, I'm recommending the shift to CCM mode. At the same time I recommend moving to an IV counter instead of random IV, in CTR mode it is necessary to make sure an IV is never reused and randomness is irrelevant. This will give you a thoroughly analyzed standard mode of operation. Joe - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
I don't exactly follow the argument for using CCM mode instead AES-CBC encryption followed by AES-CMAC, and I'm not familiar with the political/perception arguments (who complains about the latter?), but whatever. It's hardly worth arguing over. The cryptographic mode of operation is unlikely to be the weakest link in your system, and the security differences between CCM mode vs AES-CBC + AES-CMAC seem minor, so it doesn't seem worth worrying too much about it: CCM mode seems good enough. I'm not sure I'm familiar with the arguments against EAX mode (full disclosure: I'm a co-author on the EAX paper and hence probably biased), but again, whatever. These three choices are all good enough and the security differences between them seem minor. In my view, choosing any of the three would be a reasonable choice. Just my personal opinion. ObNitpick: Joseph Ashwood wrote: Since you already have CBC available, my first suggestion would be CBC-MAC (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), it has good strong security proofs behind it, and is fast. [...] Are you sure? For vanilla CBC-MAC, the security proofs don't apply to variable-length messages, and I recall that there are known attacks on vanilla CBC-MAC when message lengths can vary (I'm not claiming those attacks are necessarily realistic in all applications, but they may be). AES-CMAC is a nice design that addresses this problem. CMAC is based upon CBC-MAC, but addresses the imperfections of vanilla CBC-MAC. Personally, I wouldn't recommend vanilla CBC-MAC as a choice of message authentication primitive; CMAC seems better in every dimension. CMAC is basically a CBC-MAC, but with all the details done right. CMAC also has the benefit that it has been standardized by NIST. http://en.wikipedia.org/wiki/CMAC http://csrc.nist.gov/publications/nistpubs/800-38B/SP_800-38B.pdf Bottom line: If you're going to use a standalone CBC-based MAC together with a standalone encryption algorithm, I'd recommend using CMAC as your message authentication code, not AES-CBC. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
-- From: David Wagner d...@cs.berkeley.edu Sent: Wednesday, September 16, 2009 5:19 PM To: cryptography@metzdowd.com Subject: Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI I don't exactly follow the argument for using CCM mode instead AES-CBC encryption followed by AES-CMAC, and I'm not familiar with the political/perception arguments (who complains about the latter?), but whatever. I've actually had a few clients ask for a more detailed explaination of why it is ok, so there are people who are confused. Some people get confused. It's hardly worth arguing over. The cryptographic mode of operation is unlikely to be the weakest link in your system, and the security differences between CCM mode vs AES-CBC + AES-CMAC seem minor, so it doesn't seem worth worrying too much about it: CCM mode seems good enough. I'm not sure I'm familiar with the arguments against EAX mode (full disclosure: I'm a co-author on the EAX paper and hence probably biased), but again, whatever. Actually I think EAX great, and if I had known you were replying while I was writing mine I wouldn't have replied at all. My problem is that I haven't taken the time to look over the patents on bordering technologies to see if I believe it is patent safe. Lately, I've been dealing with a lot of patent weirdness, so I'm more aware of patent issues. ObNitpick: Joseph Ashwood wrote: Since you already have CBC available, my first suggestion would be CBC-MAC (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), it has good strong security proofs behind it, and is fast. [...] Are you sure? For vanilla CBC-MAC, the security proofs don't apply to variable-length messages, and I recall that there are known attacks on vanilla CBC-MAC when message lengths can vary (I'm not claiming those attacks are necessarily realistic in all applications, but they may be). AES-CMAC is a nice design that addresses this problem. CMAC is based upon CBC-MAC, but addresses the imperfections of vanilla CBC-MAC. I could try and justify my position, but honestly, CMAC really doesn't any real downsides, and the proof is tighter. (I moved this down here) These three choices are all good enough and the security differences between them seem minor. In my view, choosing any of the three would be a reasonable choice. Just my personal opinion. As opinions go, its hard to find a better source than David Wagner. Joe BTW: Anyone looking to make a venture capital investment? - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
David Wagner d...@cs.berkeley.edu writes: (You could replace AES-CMAC with SHA1-HMAC, but why would you want to?) The answer to that depends on whether you need to support an existing base of crypto software and hardware. Even though (in this case) it's a new standard, it still requires support from the underlying crypto libraries. If little or none of those do AES-CMAC yet (I don't think Windows CryptoAPI does, only very recent versions of OpenSSL do... it's not looking good) then you'd want to stick with HMAC-SHA1. (Forestalling the inevitable but developers can implement AES-CMAC themselves from raw AES that I'm sure someone will follow up with, the target audience for this is web application developers, not cryptographers, so you need to give them something that works as required out of the box). Peter. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
