Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-18 Thread Joseph Ashwood

--
From: Kevin W. Wall kevin.w.w...@gmail.com
Subject: Re: Detecting attempts to decrypt with incorrect secret key in 
OWASP ESAPI



So given these limited choices, what are the best options to the
questions I posed in my original post yesterday? As Peter mentioned, we
want to give web app developers something that will work out-of-the-box.


It isn't difficult to implement CMAC and CTR modes in pure Java. The NIST 
specs for CMAC and CTR are plenty clear. You'll be looking for the 
AES/ECB/NoPadding option. From there use update it returns a byte []. I've 
used the standard JCE implementation in this way to implement unsupported 
modes before, it works.
   Joe 


-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-18 Thread Ian G

On 17/09/2009 21:42, David Wagner wrote:

Kevin W. Wall wrote:

So given these limited choices, what are the best options to the
questions I posed in my original post yesterday?


Given these choices, I'd suggest that you first encrypt with AES-CBC mode.
Then apply a message authentication code (MAC) to the whole ciphertext
(including the IV).  You then send the ciphertext followed the MAC digest.

SHA1-HMAC would be a reasonable choice of algorithm for message
authentication.



I have to add vote+1 on this selection.  For various reasons, today's 
safe choice seems to be:


  * CBC
  * AES-128
  * HMAC-SHA-1 on the outside of the ciphertext

What is left is padding so that the message is clearly deliminated.  I 
suggest you treat this as a software engineering thing, not a crypto 
thing, and make sure that you have a length in your packet layout so 
that it is totally clear what is the packet and what is not.


If you want to see such a design exercise, following Dave's 
prescription, have a look at SDP1 which Zooko and I did a few years back.


http://www.webfunds.org/guide/sdp/sdp1.html
http://www.webfunds.org/guide/sdp/

It's a straight forward secret-key encrypted packet layout.  It has one 
novelty in it, which is how it solves the padding / IV issues.  Other 
than that it should be boring.



iang


PS: you are on the right track in trying to avoid any sensitivity to 
JCE.  As long as you can design your layout without any dependency on 
JCE it should work.  JCE is basically a slock design that was put in 
place for market- and crypto-control reasons, it has no place in 
software engineering.  I speak from experience, I managed the Cryptix 
project, which was the first Java crypto engine.




PPS: you haven't said enough about the application (or I missed it) to 
be able to comment on keys.  Generally, try to separate the protocol 
around the key:  every good protocol divides into two parts, the first 
of which says to the second, trust this key completely.  Software 
engineering ...


http://iang.org/ssl/h2_divide_and_conquer.html

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-17 Thread Kevin W. Wall
Peter Gutmann wrote:
 David Wagner d...@cs.berkeley.edu writes:
 
 (You could replace AES-CMAC with SHA1-HMAC, but why would you want to?)
 
 The answer to that depends on whether you need to support an existing base of
 crypto software and hardware.  Even though (in this case) it's a new standard,
 it still requires support from the underlying crypto libraries.  If little or
 none of those do AES-CMAC yet (I don't think Windows CryptoAPI does, only very
 recent versions of OpenSSL do... it's not looking good) then you'd want to
 stick with HMAC-SHA1.
 
 (Forestalling the inevitable but developers can implement AES-CMAC 
 themselves 
 from raw AES that I'm sure someone will follow up with, the target audience 
 for this is web application developers, not cryptographers, so you need to 
 give them something that works as required out of the box).

Peter has hit the proverbial nail right on the head. I apologize if I did
not make this clear in my original post, but but one goal of OWASP ESAPI
is to not require a whole lot of dependencies. In the context of Java crypto,
this means that we *ideally* would like to have no other dependency other than
the SunJCE, that is, the Sun reference implementation for JCE. Recently we
decided to required Java 6, but even with that, our choices for cipher
algorithms, cipher modes, and padding schemes are limited. For SunJCE,
this is what we have available to choose from:

Supported symmetric cipher algorithms:
AES, DES, DESede, Blowfish, RC2, ARCFOUR

Supported cipher modes:
CBC, CFB, CTR, CTS, ECB, OFB, PCBC

Supported padding schemes:
NoPadding, PKCS5Padding ISO10126Padding
OAEPPadding, OAEPWithdigestAndmgfPadding
PKCS1Padding, SSL3Padding

(Obviously some of these padding schemes such as OAEP are not suitable
with symmetric ciphers. Or at least I don't think they are.)

So given these limited choices, what are the best options to the
questions I posed in my original post yesterday? As Peter mentioned, we
want to give web app developers something that will work out-of-the-box.
For that reason we don't even want to require that developers use some other
JCE provider like Bouncy Castle, Cryptix, IAIK, etc. even though they may
have more suitable cipher modes or padding schemes.

Lastly, I wanted to respond to one other point that David Wagner brought
up in an earlier reply:

 Advice: Provide one mode, and make it secure.  Try to avoid
 configurability, because then someone will choose a poor configuration.

There's a few reasons for supporting different configurations here. One
is the backward compatibility with previous ESAPI versions, and the second
is to support legacy cases. My experience at my day job is that no one
really changes the crypto defaults anyway if you make it easy enough for them
to use. The main exception is if they have to be compatible with something else
such as some 3rd party vendor software that uses a different mode, etc.
What we can try to do is provide adequate warning in documentation or
in logged warnings if one tries to use anything other then the default.

BTW, thanks to all who replied. I've learned quite a bit from all your
responses, but it looks like I have a lot of research to do before I
understand everything that all of you said.

Regards,
-kevin
-- 
Kevin W. Wall
The most likely way for the world to be destroyed, most experts agree,
is by accident. That's where we come in; we're computer professionals.
We cause accidents.-- Nathaniel Borenstein, co-creator of MIME

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-17 Thread Peter Gutmann
Kevin W. Wall kevin.w.w...@gmail.com writes:

(Obviously some of these padding schemes such as OAEP are not suitable with
symmetric ciphers. Or at least I don't think they are.)

You'd be surprised at what JCE developers will implement just because they
can, and what therefore gets used by application developers.  I've seen 
RSA-CBC used on more than one occasion.

(No, that's not a typo, RSA in CBC mode.  The app developers wondered why it
was so slow).

Peter.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-17 Thread David Wagner
Kevin W. Wall wrote:
 So given these limited choices, what are the best options to the
 questions I posed in my original post yesterday?

Given these choices, I'd suggest that you first encrypt with AES-CBC mode.
Then apply a message authentication code (MAC) to the whole ciphertext
(including the IV).  You then send the ciphertext followed the MAC digest.

SHA1-HMAC would be a reasonable choice of algorithm for message
authentication.  Sun's JCA appears to support SHA1-HMAC.

http://java.sun.com/javase/6/docs/technotes/guides/security/crypto/CryptoSpec.html#Mac
http://java.sun.com/javase/6/docs/technotes/guides/security/StandardNames.html#Mac

You'll want to use key separation to derive two separate keys.  So
if the key K is the master key, you could use

Kenc  = SHA1-HMAC(K, encrypt)
Kauth = SHA1-HMAC(K, authenticate)

or you could use

Kenc  = AES-ECB(K, all-zeros)
Kauth = AES-ECB(K, all-ones)

(Either is fine.)  Then use Kenc as the crypto key for AES-CBC encryption
and Kauth as the crypto key for SHA1-HMAC authentication.

If you are encrypting messages that will be sent over a two-way channel,
you'll probably want to either use a different crypto key for each
direction or include a direction bit in the inputs to the key separation
step.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-17 Thread Jerry Leichter

On Sep 17, 2009, at 1:20 AM, Peter Gutmann wrote:


Kevin W. Wall kevin.w.w...@gmail.com writes:

(Obviously some of these padding schemes such as OAEP are not  
suitable with

symmetric ciphers. Or at least I don't think they are.)


You'd be surprised at what JCE developers will implement just  
because they
can, and what therefore gets used by application developers.  I've  
seen

RSA-CBC used on more than one occasion.

(No, that's not a typo, RSA in CBC mode.  The app developers  
wondered why it

was so slow).
Interesting.  It sounds as if the JCE developers have gone from one  
extreme to another.  I no longer remember the details, but a number of  
years back, in a project I was involved with, we needed to implement  
some particular (sane) combination of a cipher and a mode.  JCE at the  
time had a fixed list of combinations it was willing to let you use;  
ours wasn't on that list.  ECB wasn't an accepted mode, so it wasn't  
easy to build your own mode out of what the API provided.

-- Jerry

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-16 Thread Kevin W. Wall
Hi all,

I was referred to this site by a former colleague who thought this
is something that someone with professional cryptanalysis experience should
comment on. Also, I apologize in advance for the length of this
post (especially since it's my first one). Just trying to be thorough.

I have been working with Jeff Williams, Jim Manico and others on the OWASP ESAPI
Java implementation (http://www.esapi.org). Work is underway for the 2.0
release.

I am re-implementing the somewhat broken symmetric encryption / decryption
mechanisms implemented in the earlier 1.x versions that only supported ECB
cipher mode, restricted encryption and decryption operations to the use
of a single encryption key, and used whatever the default padding scheme
was for whatever JCE provider that happened to be used. As such, this is
still very much a work in progress, but rather than risk fouling up
encryption in the 2.0 release as well, I wanted to gather some input
that we are on the right track. Since this work is for a free open source
project (developed under the BSD license), I'm hoping that one of you
on this list will take some time to address my questions.

In OWASP ESAPI Java 2.0, I started out by deprecating the older methods that
only supported ECB mode and used the provider's default padding scheme (which
can vary, depending on JCE provider).

The new default for the new encryption / decryption methods is to be
128-bit AES/CBC/PKCS5Padding and use of a random IV. (There is some
thought on my part to not allow different cipher modes or padding
schemes, but I go back and forth on this. (Certainly if someone
tries to use something like OFB we'll at least probably long a
warning that there's some chance that the same IV could be reused
even though it is chosen randomly.)

But for the remainder of this discussion, you can assume some suitably
strong cipher algorithm and key size = 128 bits with encryption using
CBC cipher mode and PKCS5Padding. (Eventually we may allow others, but
for now, that is probably sufficient and certainly a big improvement over
only supporting ECB mode.)

Based on experience at my day job, what I have found is when you try to
decrypt something using PKCS5Padding using the wrong secret key, about
99.9% of the time you will get a BadPaddingException in Java and about
the other .1% of the time you just get random garbage back for the plaintext.
This is bad because if one is performing _manual_ key change operations
(which many small IT shops using OWASP ESAPI will likely do for PCI DSS
compliance reasons), then one could end up storing the resulting (incorrect)
plaintext value and not discovering the error until much further downstream
making and/or at a much later time. This makes the error very hard to
troubleshoot as it typically will a long ways away in both code space
and timeline away from the true root cause of the problem.

I would like to be able to support such key change operations for ESAPI in
a way that one has a much higher probability of detecting that decryptions
using the incorrect secret key that do NOT result in a BadPaddingException
have a much higher probability of being detected. Unfortunately, since
this is generally reusable class library I can make no assumptions about
the original plaintext. Specifically, it might not be text in the conventional
sense at all, but rather it could be something like another random secret key
that someone encrypted, etc. (key wrapping notwithstanding, but it is not
currently supported in ESAPI 2.0).

For ESAPI Java 2.0, I've created a serializable CipherText class that
contains everything one generally needs to know to perform the decryption
operation. That is, the CipherText object contains things like the
the specification cipher transformation used to encrypt it, the IV used
(unless ECB mode), and the raw ciphertext value as a byte array.

My first thought was to also include a digital signature that was created
along the lines one of these (note: here 'DSig(msg)' includes both the
hashing and private key signature operations and probably would use DSA
since it is not subject to any US export regulations):

DSig( IV + secretKey )  // I explain below why IV is included
or
DSig( IV + plaintext )  // '+' is just the usual byte concatenation

signed by the encrypter's private key. The decrypting side would then validate
the digital signature using the encrypter's public key and only use the
plaintext value if the digital signature was valid. If I were to do this,
I might also add the encrypter's identity or alias as part of the CipherText
class, although in most of the simple cases, it should be obvious based on
from whom one is receiving the encrypted data.

However, the main problem with using digital signatures is the performance
hit. In my day job, I have made two observations involving encryption:

1) If you wish to ensure that application developers use solid
   algorithms such as AES, you must ensure that the 

Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-16 Thread David Wagner
Advice: if you're creating something for general-purpose use, at a minimum
make sure it provides authentication, integrity, *and* confidentiality.
A reasonable choice might be Encrypt-then-Authenticate where you first
encrypt with AES-CBC, then append a AES-CMAC message authentication
code on the ciphertext (including the IV).  As a bonus, this will detect
decrypting with the wrong key.

Use key separation -- with a pseudorandom function -- to derive the
encryption key and authentication key from the master crypto key
supplied by the user: e.g., Encryptkey = AES(K, all-zeros), Authkey =
AES(K, all-ones).

(You could replace AES-CMAC with SHA1-HMAC, but why would you want to?)

(From a cryptotheory point of view, what you want is a mode of operation
that meets IND-CCA2 /\ INT-CTXT, or at least IND-CCA2 /\ INT-PTXT.)

Advice: Provide one mode, and make it secure.  Try to avoid
configurability, because then someone will choose a poor configuration.

Suggestion: Provide documentation to warn the programmer against using a
password (or something based on a password) as the crypto key.  Provide
support for PBKDF2, with a reasonable default choice of parameters and
appropriate warnings in the documentation, for those who absolutely must
use a password-derived crypto key.

Recommendation: Read the book Practical Cryptography by Ferguson and
Schneier, or at least the chapters on message security.  It's the best
source I know to teach you some of the crypto-engineering practicum.




Kevin W. Wall wrote:
I have considered using an HMAC-SHA1 as a keyless MIC to do this,
using something like:

   MIC = HMAC-SHA1( nonce, IV + secretKey )
or
   MIC = HMAC-SHA1( nonce, IV + plaintext )

and then also include the random nonce and the MIC itself in the CipherText
class so it can be validated later by the decrypter, with the understanding
that the plaintext resulting from the decryption operation should only be
used if the MIC can be properly validated. (Probably an exception would
be thrown if the dsig was not valid so there would be no choice.)

I would not recommend either of these.  It's better to use a MAC as I
suggest at the top of this email.

Whatever you do, don't choose your second MIC option: if the plaintext
comes from a low-entropy space, it leaks the value of the plaintext
(the plaintext can be recovered by brute-force search over all possible
plaintexts).

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-16 Thread Joseph Ashwood

--
From: Kevin W. Wall kevin.w.w...@gmail.com
Subject: Detecting attempts to decrypt with incorrect secret key in OWASP 
ESAPI




The new default for the new encryption / decryption methods is to be
128-bit AES/CBC/PKCS5Padding and use of a random IV.


That's a good solution, except for the missing MAC, considering the 
environment I would suggest hard coding it, there's really no reason to 
offer options.




MIC = HMAC-SHA1( nonce, IV + secretKey )
or
MIC = HMAC-SHA1( nonce, IV + plaintext )
MIC = HMAC-SHA1( IV, plaintext )
So, having provided all of that background, in summary, here are my
three questions:
   1) Is using either of these MIC calculations cryptographically secure?
   2) If answer to #1, is 'yes', which one is safer / more secure?
   3) If answer to #1 is 'no', do you have any suggestions less
  computationally expensive then digital signatures that would
  allow us to detect attempts to decrypt with the incorrect secret
  key and/or an adversary attempting to alter the IV prior to the
  decryption.


You are looking at the correct type of construct, the solution for your 
problem is known as a Message Authentication Code or MAC. The biggest 
concerns are what are you MACing, and that it must use a second key. There 
are a number of solutions.


The first part of the solution is to store an additional key, you're storing 
128-bits hopefully securely store 256, and split it, trust me you'll thank 
yourself when the security issues are investigated. The second key is for 
the MAC algorithm.


Since you already have CBC available, my first suggestion would be CBC-MAC 
(IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), 
it has good strong security proofs behind it, and is fast. Apply the MAC 
algorithm, prepend the MAC value to the plaintext (just because indexing is 
easier this way), then encrypt to ciphertext, store. To decrypt, retrieve, 
decrypt the ciphertext, parse out the MAC value and the plaintext, run MAC 
algorithm on plaintext to find MAC2, check MAC == MAC2 . This will give you 
a failure rate of 1/2^128 or something on the order of 
0.3%, you are more likely to have the 
system burst into flame than see a false positive on this. Overall, this 
will reduce your speed to 50% of the current.


You might also want to take a look at Poly1305, it is faster.

As you noted HMAC is also available, but I would recommend against using 
SHA-1 for anything, it simply raises too many questions that will take too 
long to answer. The secure hashes available are simply not as fast unless 
you have bare-metal coding ability which Java really doesn't like.


The other, and arguably better, option would be a combined mode. The good 
combined modes are a legal minefield, so using them for an open project is 
difficult. It is claimed that GCM and EAX are public domain, but I'm more 
inclined to recommend the conservative approach and avoid them.


While there has been no concern raised by cryptanalysts about CBC with 
CBC-MAC, to many laypeople it doesn't look good. So, for purely politic 
reasons, I'm recommending the shift to CCM mode. At the same time I 
recommend moving to an IV counter instead of random IV, in CTR mode it is 
necessary to make sure an IV is never reused and randomness is irrelevant. 
This will give you a thoroughly analyzed standard mode of operation.
   Joe 


-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-16 Thread David Wagner
I don't exactly follow the argument for using CCM mode instead
AES-CBC encryption followed by AES-CMAC, and I'm not familiar with
the political/perception arguments (who complains about the latter?),
but whatever.  It's hardly worth arguing over.  The cryptographic mode
of operation is unlikely to be the weakest link in your system, and the
security differences between CCM mode vs AES-CBC + AES-CMAC seem minor,
so it doesn't seem worth worrying too much about it: CCM mode seems good
enough.  I'm not sure I'm familiar with the arguments against EAX mode
(full disclosure: I'm a co-author on the EAX paper and hence probably
biased), but again, whatever.  These three choices are all good enough and
the security differences between them seem minor.  In my view, choosing
any of the three would be a reasonable choice.  Just my personal opinion.


ObNitpick:

Joseph Ashwood wrote:
 Since you already have CBC available, my first suggestion would be CBC-MAC 
 (IV = 0x000, okcs5 padding works fine, MAC = final block of ciphertext), 
 it has good strong security proofs behind it, and is fast. [...]

Are you sure?  For vanilla CBC-MAC, the security proofs don't apply to
variable-length messages, and I recall that there are known attacks on
vanilla CBC-MAC when message lengths can vary (I'm not claiming those
attacks are necessarily realistic in all applications, but they may be).
AES-CMAC is a nice design that addresses this problem.  CMAC is based
upon CBC-MAC, but addresses the imperfections of vanilla CBC-MAC.

Personally, I wouldn't recommend vanilla CBC-MAC as a choice of message
authentication primitive; CMAC seems better in every dimension.  CMAC is
basically a CBC-MAC, but with all the details done right.  CMAC also
has the benefit that it has been standardized by NIST.

http://en.wikipedia.org/wiki/CMAC
http://csrc.nist.gov/publications/nistpubs/800-38B/SP_800-38B.pdf 

Bottom line: If you're going to use a standalone CBC-based MAC together
with a standalone encryption algorithm, I'd recommend using CMAC as your
message authentication code, not AES-CBC.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-16 Thread Joseph Ashwood



--
From: David Wagner d...@cs.berkeley.edu
Sent: Wednesday, September 16, 2009 5:19 PM
To: cryptography@metzdowd.com
Subject: Re: Detecting attempts to decrypt with incorrect secret key in 
OWASP ESAPI



I don't exactly follow the argument for using CCM mode instead
AES-CBC encryption followed by AES-CMAC, and I'm not familiar with
the political/perception arguments (who complains about the latter?),
but whatever.


I've actually had a few clients ask for a more detailed explaination of why 
it is ok, so there are people who are confused. Some people get confused.



It's hardly worth arguing over.  The cryptographic mode
of operation is unlikely to be the weakest link in your system, and the
security differences between CCM mode vs AES-CBC + AES-CMAC seem minor,
so it doesn't seem worth worrying too much about it: CCM mode seems good
enough.  I'm not sure I'm familiar with the arguments against EAX mode
(full disclosure: I'm a co-author on the EAX paper and hence probably
biased), but again, whatever.


Actually I think EAX great, and if I had known you were replying while I was 
writing mine I wouldn't have replied at all. My problem is that I haven't 
taken the time to look over the patents on bordering technologies to see if 
I believe it is patent safe. Lately, I've been dealing with a lot of patent 
weirdness, so I'm more aware of patent issues.



ObNitpick:

Joseph Ashwood wrote:
Since you already have CBC available, my first suggestion would be 
CBC-MAC
(IV = 0x000, okcs5 padding works fine, MAC = final block of 
ciphertext),

it has good strong security proofs behind it, and is fast. [...]


Are you sure?  For vanilla CBC-MAC, the security proofs don't apply to
variable-length messages, and I recall that there are known attacks on
vanilla CBC-MAC when message lengths can vary (I'm not claiming those
attacks are necessarily realistic in all applications, but they may be).
AES-CMAC is a nice design that addresses this problem.  CMAC is based
upon CBC-MAC, but addresses the imperfections of vanilla CBC-MAC.


I could try and justify my position, but honestly, CMAC really doesn't any 
real downsides, and the proof is tighter.


(I moved this down here)


These three choices are all good enough and
the security differences between them seem minor.  In my view, choosing
any of the three would be a reasonable choice.  Just my personal opinion.


As opinions go, its hard to find a better source than David Wagner.
   Joe

BTW: Anyone looking to make a venture capital investment? 


-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com


Re: Detecting attempts to decrypt with incorrect secret key in OWASP ESAPI

2009-09-16 Thread Peter Gutmann
David Wagner d...@cs.berkeley.edu writes:

(You could replace AES-CMAC with SHA1-HMAC, but why would you want to?)

The answer to that depends on whether you need to support an existing base of
crypto software and hardware.  Even though (in this case) it's a new standard,
it still requires support from the underlying crypto libraries.  If little or
none of those do AES-CMAC yet (I don't think Windows CryptoAPI does, only very
recent versions of OpenSSL do... it's not looking good) then you'd want to
stick with HMAC-SHA1.

(Forestalling the inevitable but developers can implement AES-CMAC themselves 
from raw AES that I'm sure someone will follow up with, the target audience 
for this is web application developers, not cryptographers, so you need to 
give them something that works as required out of the box).

Peter.

-
The Cryptography Mailing List
Unsubscribe by sending unsubscribe cryptography to majord...@metzdowd.com