Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
Hi all, I was referred to this site by a former colleague who thought this is something that someone with professional cryptanalysis experience should comment on. Also, I apologize in advance for the length of this post (especially since it's my first one). Just trying to be thorough. I have been working with Jeff Williams, Jim Manico and others on the OWASP ESAPI Java implementation (http://www.esapi.org). Work is underway for the 2.0 release. I am re-implementing the somewhat broken symmetric encryption / decryption mechanisms implemented in the earlier 1.x versions that only supported ECB cipher mode, restricted encryption and decryption operations to the use of a single encryption key, and used whatever the default padding scheme was for whatever JCE provider that happened to be used. As such, this is still very much a work in progress, but rather than risk fouling up encryption in the 2.0 release as well, I wanted to gather some input that we are on the right track. Since this work is for a free open source project (developed under the BSD license), I'm hoping that one of you on this list will take some time to address my questions. In OWASP ESAPI Java 2.0, I started out by deprecating the older methods that only supported ECB mode and used the provider's default padding scheme (which can vary, depending on JCE provider). The new default for the new encryption / decryption methods is to be 128-bit AES/CBC/PKCS5Padding and use of a random IV. (There is some thought on my part to not allow different cipher modes or padding schemes, but I go back and forth on this. (Certainly if someone tries to use something like OFB we'll at least probably long a warning that there's some chance that the same IV could be reused even though it is chosen randomly.) But for the remainder of this discussion, you can assume some suitably strong cipher algorithm and key size = 128 bits with encryption using CBC cipher mode and PKCS5Padding. (Eventually we may allow others, but for now, that is probably sufficient and certainly a big improvement over only supporting ECB mode.) Based on experience at my day job, what I have found is when you try to decrypt something using PKCS5Padding using the wrong secret key, about 99.9% of the time you will get a BadPaddingException in Java and about the other .1% of the time you just get random garbage back for the plaintext. This is bad because if one is performing _manual_ key change operations (which many small IT shops using OWASP ESAPI will likely do for PCI DSS compliance reasons), then one could end up storing the resulting (incorrect) plaintext value and not discovering the error until much further downstream making and/or at a much later time. This makes the error very hard to troubleshoot as it typically will a long ways away in both code space and timeline away from the true root cause of the problem. I would like to be able to support such key change operations for ESAPI in a way that one has a much higher probability of detecting that decryptions using the incorrect secret key that do NOT result in a BadPaddingException have a much higher probability of being detected. Unfortunately, since this is generally reusable class library I can make no assumptions about the original plaintext. Specifically, it might not be text in the conventional sense at all, but rather it could be something like another random secret key that someone encrypted, etc. (key wrapping notwithstanding, but it is not currently supported in ESAPI 2.0). For ESAPI Java 2.0, I've created a serializable CipherText class that contains everything one generally needs to know to perform the decryption operation. That is, the CipherText object contains things like the the specification cipher transformation used to encrypt it, the IV used (unless ECB mode), and the raw ciphertext value as a byte array. My first thought was to also include a digital signature that was created along the lines one of these (note: here 'DSig(msg)' includes both the hashing and private key signature operations and probably would use DSA since it is not subject to any US export regulations): DSig( IV + secretKey ) // I explain below why IV is included or DSig( IV + plaintext ) // '+' is just the usual byte concatenation signed by the encrypter's private key. The decrypting side would then validate the digital signature using the encrypter's public key and only use the plaintext value if the digital signature was valid. If I were to do this, I might also add the encrypter's identity or alias as part of the CipherText class, although in most of the simple cases, it should be obvious based on from whom one is receiving the encrypted data. However, the main problem with using digital signatures is the performance hit. In my day job, I have made two observations involving encryption: 1) If you wish to ensure that application developers use solid algorithms such as AES, you must ensure that the
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
Advice: if you're creating something for general-purpose use, at a minimum make sure it provides authentication, integrity, *and* confidentiality. A reasonable choice might be Encrypt-then-Authenticate where you first encrypt with AES-CBC, then append a AES-CMAC message authentication code on the ciphertext (including the IV). As a bonus, this will detect decrypting with the wrong key. Use key separation -- with a pseudorandom function -- to derive the encryption key and authentication key from the master crypto key supplied by the user: e.g., Encryptkey = AES(K, all-zeros), Authkey = AES(K, all-ones). (You could replace AES-CMAC with SHA1-HMAC, but why would you want to?) (From a cryptotheory point of view, what you want is a mode of operation that meets IND-CCA2 /\ INT-CTXT, or at least IND-CCA2 /\ INT-PTXT.) Advice: Provide one mode, and make it secure. Try to avoid configurability, because then someone will choose a poor configuration. Suggestion: Provide documentation to warn the programmer against using a password (or something based on a password) as the crypto key. Provide support for PBKDF2, with a reasonable default choice of parameters and appropriate warnings in the documentation, for those who absolutely must use a password-derived crypto key. Recommendation: Read the book Practical Cryptography by Ferguson and Schneier, or at least the chapters on message security. It's the best source I know to teach you some of the crypto-engineering practicum. Kevin W. Wall wrote: I have considered using an HMAC-SHA1 as a keyless MIC to do this, using something like: MIC = HMAC-SHA1( nonce, IV + secretKey ) or MIC = HMAC-SHA1( nonce, IV + plaintext ) and then also include the random nonce and the MIC itself in the CipherText class so it can be validated later by the decrypter, with the understanding that the plaintext resulting from the decryption operation should only be used if the MIC can be properly validated. (Probably an exception would be thrown if the dsig was not valid so there would be no choice.) I would not recommend either of these. It's better to use a MAC as I suggest at the top of this email. Whatever you do, don't choose your second MIC option: if the plaintext comes from a low-entropy space, it leaks the value of the plaintext (the plaintext can be recovered by brute-force search over all possible plaintexts). - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Bringing Tahoe ideas to HTTP
On Sep 15, 2009, at 4:12 PM, James A. Donald wrote: The ideas used in Tahoe are useful tools that can be used to solve important problems. Yes, and I'd be happy to opine on that as soon as someone told me what those important problems are. -- Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: [tahoe-dev] Bringing Tahoe ideas to HTTP
On Wednesday,2009-09-16, at 14:44 , Ivan Krstić wrote: Yes, and I'd be happy to opine on that as soon as someone told me what those important problems are. The message that you quoted from Brian Warner, which ended with him wondering aloud what new applications could be enabled by such features, began with him mentioning a specific use case that he cares about and sees how to improve: authentication of Mozilla plugins. Brian has an admirable habit of documenting the use cases that motivate his engineering decisions. I think in this case he omitted some explanation of why he finds the current solution unsatisfactory, perhaps because he assumed the audience already shared his view. (I think he mentioned something in his letter like the well-known failures of the SSL/CA approach to this problem.) Regards, Zooko - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
-- From: Kevin W. Wall kevin.w.w...@gmail.com Subject: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI The new default for the new encryption / decryption methods is to be 128-bit AES/CBC/PKCS5Padding and use of a random IV. That's a good solution, except for the missing MAC, considering the environment I would suggest hard coding it, there's really no reason to offer options. MIC = HMAC-SHA1( nonce, IV + secretKey ) or MIC = HMAC-SHA1( nonce, IV + plaintext ) MIC = HMAC-SHA1( IV, plaintext ) So, having provided all of that background, in summary, here are my three questions: 1) Is using either of these MIC calculations cryptographically secure? 2) If answer to #1, is 'yes', which one is safer / more secure? 3) If answer to #1 is 'no', do you have any suggestions less computationally expensive then digital signatures that would allow us to detect attempts to decrypt with the incorrect secret key and/or an adversary attempting to alter the IV prior to the decryption. You are looking at the correct type of construct, the solution for your problem is known as a Message Authentication Code or MAC. The biggest concerns are what are you MACing, and that it must use a second key. There are a number of solutions. The first part of the solution is to store an additional key, you're storing 128-bits hopefully securely store 256, and split it, trust me you'll thank yourself when the security issues are investigated. The second key is for the MAC algorithm. Since you already have CBC available, my first suggestion would be CBC-MAC (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), it has good strong security proofs behind it, and is fast. Apply the MAC algorithm, prepend the MAC value to the plaintext (just because indexing is easier this way), then encrypt to ciphertext, store. To decrypt, retrieve, decrypt the ciphertext, parse out the MAC value and the plaintext, run MAC algorithm on plaintext to find MAC2, check MAC == MAC2 . This will give you a failure rate of 1/2^128 or something on the order of 0.3%, you are more likely to have the system burst into flame than see a false positive on this. Overall, this will reduce your speed to 50% of the current. You might also want to take a look at Poly1305, it is faster. As you noted HMAC is also available, but I would recommend against using SHA-1 for anything, it simply raises too many questions that will take too long to answer. The secure hashes available are simply not as fast unless you have bare-metal coding ability which Java really doesn't like. The other, and arguably better, option would be a combined mode. The good combined modes are a legal minefield, so using them for an open project is difficult. It is claimed that GCM and EAX are public domain, but I'm more inclined to recommend the conservative approach and avoid them. While there has been no concern raised by cryptanalysts about CBC with CBC-MAC, to many laypeople it doesn't look good. So, for purely politic reasons, I'm recommending the shift to CCM mode. At the same time I recommend moving to an IV counter instead of random IV, in CTR mode it is necessary to make sure an IV is never reused and randomness is irrelevant. This will give you a thoroughly analyzed standard mode of operation. Joe - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
I don't exactly follow the argument for using CCM mode instead AES-CBC encryption followed by AES-CMAC, and I'm not familiar with the political/perception arguments (who complains about the latter?), but whatever. It's hardly worth arguing over. The cryptographic mode of operation is unlikely to be the weakest link in your system, and the security differences between CCM mode vs AES-CBC + AES-CMAC seem minor, so it doesn't seem worth worrying too much about it: CCM mode seems good enough. I'm not sure I'm familiar with the arguments against EAX mode (full disclosure: I'm a co-author on the EAX paper and hence probably biased), but again, whatever. These three choices are all good enough and the security differences between them seem minor. In my view, choosing any of the three would be a reasonable choice. Just my personal opinion. ObNitpick: Joseph Ashwood wrote: Since you already have CBC available, my first suggestion would be CBC-MAC (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), it has good strong security proofs behind it, and is fast. [...] Are you sure? For vanilla CBC-MAC, the security proofs don't apply to variable-length messages, and I recall that there are known attacks on vanilla CBC-MAC when message lengths can vary (I'm not claiming those attacks are necessarily realistic in all applications, but they may be). AES-CMAC is a nice design that addresses this problem. CMAC is based upon CBC-MAC, but addresses the imperfections of vanilla CBC-MAC. Personally, I wouldn't recommend vanilla CBC-MAC as a choice of message authentication primitive; CMAC seems better in every dimension. CMAC is basically a CBC-MAC, but with all the details done right. CMAC also has the benefit that it has been standardized by NIST. http://en.wikipedia.org/wiki/CMAC http://csrc.nist.gov/publications/nistpubs/800-38B/SP_800-38B.pdf Bottom line: If you're going to use a standalone CBC-based MAC together with a standalone encryption algorithm, I'd recommend using CMAC as your message authentication code, not AES-CBC. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
-- From: David Wagner d...@cs.berkeley.edu Sent: Wednesday, September 16, 2009 5:19 PM To: cryptography@metzdowd.com Subject: Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI I don't exactly follow the argument for using CCM mode instead AES-CBC encryption followed by AES-CMAC, and I'm not familiar with the political/perception arguments (who complains about the latter?), but whatever. I've actually had a few clients ask for a more detailed explaination of why it is ok, so there are people who are confused. Some people get confused. It's hardly worth arguing over. The cryptographic mode of operation is unlikely to be the weakest link in your system, and the security differences between CCM mode vs AES-CBC + AES-CMAC seem minor, so it doesn't seem worth worrying too much about it: CCM mode seems good enough. I'm not sure I'm familiar with the arguments against EAX mode (full disclosure: I'm a co-author on the EAX paper and hence probably biased), but again, whatever. Actually I think EAX great, and if I had known you were replying while I was writing mine I wouldn't have replied at all. My problem is that I haven't taken the time to look over the patents on bordering technologies to see if I believe it is patent safe. Lately, I've been dealing with a lot of patent weirdness, so I'm more aware of patent issues. ObNitpick: Joseph Ashwood wrote: Since you already have CBC available, my first suggestion would be CBC-MAC (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), it has good strong security proofs behind it, and is fast. [...] Are you sure? For vanilla CBC-MAC, the security proofs don't apply to variable-length messages, and I recall that there are known attacks on vanilla CBC-MAC when message lengths can vary (I'm not claiming those attacks are necessarily realistic in all applications, but they may be). AES-CMAC is a nice design that addresses this problem. CMAC is based upon CBC-MAC, but addresses the imperfections of vanilla CBC-MAC. I could try and justify my position, but honestly, CMAC really doesn't any real downsides, and the proof is tighter. (I moved this down here) These three choices are all good enough and the security differences between them seem minor. In my view, choosing any of the three would be a reasonable choice. Just my personal opinion. As opinions go, its hard to find a better source than David Wagner. Joe BTW: Anyone looking to make a venture capital investment? - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI
David Wagner d...@cs.berkeley.edu writes: (You could replace AES-CMAC with SHA1-HMAC, but why would you want to?) The answer to that depends on whether you need to support an existing base of crypto software and hardware. Even though (in this case) it's a new standard, it still requires support from the underlying crypto libraries. If little or none of those do AES-CMAC yet (I don't think Windows CryptoAPI does, only very recent versions of OpenSSL do... it's not looking good) then you'd want to stick with HMAC-SHA1. (Forestalling the inevitable but developers can implement AES-CMAC themselves from raw AES that I'm sure someone will follow up with, the target audience for this is web application developers, not cryptographers, so you need to give them something that works as required out of the box). Peter. - The Cryptography Mailing List Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com
