On Tue, Apr 3, 2012 at 9:35 AM, ianG <i...@iang.org> wrote: > Belated reply!
On belated reply deserves another. Actually Uncle Sam has me busy doing taxes and GSoC busy with mentoring activities. > On 21/02/12 06:40 AM, Kevin W. Wall wrote: >> >> First of all, let me thank all who have responded for lending >> your expertise. I am just picking out Ian's to respond to >> because of his suggesting dividing up the IV into >> >> random||counter||time >> >> but I do appreciate everyone else's comments as well. >> >> On Mon, Feb 20, 2012 at 7:11 AM, ianG<i...@iang.org> wrote: >>> >>> On 20/02/12 18:11 PM, Kevin W. Wall wrote: >>>> >>>> >>>> Hi list, >>>> >>>> This should be a pretty simple question for this list, so please pardon >>>> my ignorance. But better to ask than to continue in ignorance. :-) >>>> >>>> NIST refers to "combined" cipher modes as those supporting *both* >>>> authenticity and confidentiality, such as GCM and CCM. >>> >>> >>> My personal impression of such things is that although they can give a >>> paper >>> sense of authenticity, it is not good enough to rely on at an application >>> level because of software layering issues. In the past, I've preferred >>> to >>> use a heavyweight signed plaintext packet, then encrypted with a >>> light-level >>> HMAC or similar. So, yes it is authenticated twice, but they are at >>> different strengths / semantics / layers. >> >> >> Yes, well, that is all well and good for some things, but when the primary >> use of encryption nowadays is to encrypt short strings like credit card >> numbers and bank account numbers, most developers are not going to >> put up with the additional space& CPU overhead of both a dsig and an >> >> HMAC. Based on your recommendation from several years ago, we had >> originally >> used an HMAC-SHA1, but changed it to an HMAC-SHA256 after recommendations >> from the initial NSA review. However, we (OWASP ESAPI) only do this for >> when the user decides to use an unauthenticated cipher mode. > > > > Those statements are ... interesting. When protecting bank account numbers, > credit card numbers, etc, you really don't care that much. Most all of them > are available for a price on your local phishing market, and they aren't > transactions. You're protected other ways. Which is to say, you don't need > bullet proof protection, you can afford to be efficient. The software won't > be bullet proof ... Phish markets aside, most developers encrypt bank account #s and credit card info because of compliance issues like NACHA and PCI DSS. And truth be told, about the only threat that this addresses is one of a rough DBA heisting your DB of CC information or bank account #s. If your code has SQLi vulnerabilities, an adversary is still going to walk away with your database. > Secondly, think of your architecture. Anyone who is protecting CCs with > this model will likely not be able to show that the authentication is in > good nick. So if there is a problem, you are left with the statement "our > security is good" which sits oddly with the statement "somebody lost their > money." The big risk in having CCs or banking info stolen is the subsequent (usually class action) lawsuits that usually follow. So these things are done as part of following industry "best practice", in which case you can at least claim that you've done due diligence and if you lose, at least the claimants in the lawsuit are awarded treble damages. So the your "our security is good" approach is mostly done to satisfy this due diligence and to a lesser extent to show the PCI and NACHA auditors that you done what you were asked to do so that you don't end up having to pay fines. It's classic CYA security. > This is why I use digsigs over the money packets - the digsig > carries with the money instruction; the HMACs are thrown away as ephemeral. > Layering is important for security too... Dsigs are harder from an operational perspective as well as harder to teach developers. Plus it can be a significant performance hit. But I agree, both have their place. > In that sense, lower layer HMAC-SHA1 is overkill. IMHO, and, in my future > design I intend to reduce that a lot. The overkill seems to be mostly in the overhead of storing it along with the IV+ciphertext. The overhead is significant when encrypting short strings such as CC#s, SSNs, etc. Generally these issues come up when trying to make legacy code PCI or NACHA compliant. The DB already exists and table space is already allocated so storage space is at a premium. Usually, I recommend that development teams use something like Oracle's Transparent Data Encryption and Oracle Wallet. But our DBAs are reluctant to allow developers to use it; say it's too much of a performance impact on the DB engine. I say take the development savings and put it into some new upgraded hardware. I generally loose those arguments. :-( > To tip my hand here somewhat I'm thinking of GCM. > > (Digression.) Now, this thread was useful to me because I started reading > up on new modes and so forth, and combined that with my past experiences. > What I wanted was a fast AES mode coupled with a heavyweight keyed CRC for > opportunistic/DOS protection. > > Hey presto - GCM is that! (I think, haven't finished reading yet.) If you > look at the formula for Galois, it is basically a CRC expanded out to 128 > bits. Perfect! Fast! > > But notice I sat heavyweight keyed CRC. I consider a Hash MAC to be > overkill - SHA1 is way too much trouble for this. > > So your statement above is ... curious. On the one hand, the NSA apparently > advised you to turn the overkill of HMAC-SHA1 into the blended horse > milkshake of HMAC-SHA256 ... which you do, unless you're using GCM! Or > CCM... > > I remain bemused and can't wait to hear what the NSA says in the next review > :) Me too. I have some rather pointed questions to ask them about some other things the supposedly reviewed and are weaknesses that I think (given their reputation) they should have caught. They didn't mention it. I'm not sure if that was intentional or they just had junior people looking at ESAPI or if they had too many other higher priorities and are overworked like most of us so didn't get sufficient time to study it or what. (I'm hoping it's the latter, but not so sure.) > > >>>> So my first question: Are there ANY "combined" cipher modes >>>> for block ciphers that do not cause the ciphers to act as a key >>>> stream? (That seems to be cause most of the ones I found build >>>> the confidentiality piece around CTR mode.) If "yes", please name >>>> a few (especially those with no patent restrictions). >>>> >>>> I know when you have a cipher that acts in a streaming mode, >>>> that you need to be careful to use a unique IV for every encryption >>>> performed with the same key. >>> >>> >>> >>> Well. With basic modes like CBC, if there is no variation in the early >>> parts of the packet, those blocks will encrypt the same. >>> >>> A good plaintext packet design can push strong variation into the first >>> bytes. e.g., the MAC can go at the beginning not the end. It used to be >>> customary to put the MAC at the end because hardware calculated it and >>> streamed it at the same time, but software doesn't work that way. >>> >>> (There was a paper suggesting that encrypt-then-mac was better than >>> mac-then-encrypt, but I vaguely recall this result only applies under >>> some >>> circumstances. Does anyone recall how important this issue was?) >> >> >> I've read a few papers and blogs on this topic. The one that sticks in >> my mind was one of Nate Lawson's blogs. I just looked it up and I >> think it was: >> >> http://rdist.root.org/2010/09/10/another-reason-to-mac-ciphertext-not-plaintext/ > > > > Um. There are a lot of assumptions in that post. Firstly, that someone is > sending you garbage packets and you can get a benefit by dropping them at > the MAC phase not the decrypt. Think about it ... And that could be the case, depending on how people use ESAPI. Duong and Rizzo mentioned that early versions of ESAPI 2.0 were vulnerable to padding oracle attacks. (That was caused by sloppiness on my part. Besides a subtle side-channel timing attack that could be used as an oracle, I also stupidly had the exception message in two different spots with slightly different error messages. Duh!) > Surface area - ditto. If you are subject to that, you have other problems. Well, the thing is, you just don't know. We are not developing an application here--where we would have a fairly good idea of the context and thus could develop an appropriate threat model. Rather we are developing a class library full of common security controls. It's very hard to tell how developers using your class library are going to expose your API so you more or less plan for the worst case scenario. Thus the choice to MAC the IV+ciphertext. But my first recommendation is to use an authenticated mode such as GCM or CCM. Unfortunately, the off-the-shelf Oracle SunJCE, which is the default, reference implementation of the Java Cryptography Extensions does not support ANY authenticated cipher mode. If you want this, you need to use something like Bouncy Castle or an alternate JCE provider. Unfortunately, in many commercial IT shops, trying to get permission to use Bouncy Castle is a hard sell. It was in my organization; I had to jump through all kind of hoops to first get it on our "approved product list" before finally getting approval to use it. It took about 3 or 4 weeks. And buying a commercial crypto JCE library is usually even more work. So most developers just give up and settle for using SunJCE and one of the cipher modes that it supports. > Finally - subject to side-channel attacks. If you are, then we are in > another ballgame. I don't think you are. Having multiple "stacked applications" on a single physical host is common. Unfortunately, applications are generally not assigned hosts based on any security requirements; rather they are "stacked" to maximize the CPU utilization and place as many applications on a server as possible. So one possible threat model is being attacked by a hostile application (developed by a rogue developer) that is running on the same physical server as your application. So getting detailed timing information is not that difficult. Plus, once you know about certain timing side channel attacks it is not that difficult to address them. Becoming aware of them is the hard part. (Note: I only addressed _timing_ side channels, not any other type. And also, only the ones I was aware of. Duong and Rizzo's research on padding oracles convinced me that this was worth doing. You do what you can and what you know about. It didn't add very much complexity to fix.) >> Based on this and some additional comments from Duong& Rizzo, we decided >> to >> >> use the encrypt-then-MAC approach to ensure the integrity of the >> ciphertext. >> (Keep in mind this was designed around the time that Duong and Rizzo >> automated a padding oracle attack with their POET software.) >> >> However, we skipped the MAC calculation if the cipher mode chosen >> was an authenticated mode like CCM or GCM. The assumption (hopefully >> a correct one), was that an authenticated cipher mode was sufficient. > > > > That bit! To repeat my earlier words - this is why I went for a lightweight > MAC approach. It fits 99% of needs. And the remaining needs won't be > properly served either, because architecture will likely interfere. Not sure what you mean by the "architecture will likely interfere". Could you explain please? >> In ESAPI (ignoring all error / exception handling, etc.), using such >> a "combined" mode, distills down essentially to something like this >> for the encryption: >> >> // This is in essence what ESAPI does for combined cipher modes like CCM& >> GCM >> >> // Assume a 128-bit AES key for this example. >> public class Encryptor { >> private static SecureRandoma prng = new SecureRandom(); >> ... >> public byte[] encrypt(SecretKey sk, byte[] plain) throws Exception { >> ... >> Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding"); >> byte[] ivBytes = new byte[ 128 / 8 ]; // 16 bytes (AES block >> size) >> prng.nextBytes(ivBytes); >> IvParameterSpec ivSpec = new IvParameterSpec(ivBytes); >> cipher.init(Cipher.ENCRYPT_MODE, sk, ivSpec); >> return cipher.doFinal( plain ); >> } >> ... >> } >> /////////////////////////// >> >> However, as you can plainly see, there is no attempt here to prevent >> the reuse of an IV for a given key. The assumption was originally, >> that the random IV use was sufficient, but after reading recent comments >> on entropy pools at boot-up time (when application servers typically >> are started from /etc/init.d scripts), I'm not so sure. > > > > Yes, exactly. When Zooko and I designed the random||counter||time construct > it is because we knew that some or many servers could get into a > pathological mode w.r.t. entropy. And saying "have good entropy" is like > telling teenaged girls not to hang around teenaged boys. I'm definitely going to replace the prng.nextBytes(ivBytes); with what you and Zooko used...the random||counter||time construct. That's now on my list for 2.1. Fixing this will allow the use of streaming cipher modes too. (For now, I make it difficult--although not impossible--to use such modes.) > >> However, above more or less is considered best practice to using Java for >> symmetric encryption when you are using an authenticated mode. >> >> The question is whether or not this is sufficient. I suppose like >> most everything in information security, it depends on one's threat >> model. > > > > (Business model.) You are protecting account numbers. It's probably OK. > Nobody has ever cracked encryption to steal account numbers - coz the > winnings just ain't worth a packet level attack. The threat it is meant to prevent is a DBA walking away with 20M bank accounts or credit card #s. Hey, it happed at my credit union (although with significantly lower #s, about 8-10 yrs ago--definitely it was pre-PCI / pre-NACHA). > Also, the servers that suffer pathalogical entropy problems are typically > embedded devices like routers, not business servers such as you are > postulating. > > That said, what we proposed isn't any more expensive than collecting some > stuff from the OS's /dev/random. Ooh...did that in a custom EntropyPool class we did in a proprietary API. Had 'high', 'medium', and 'low' settings. Default was 'medium' which read from /dev/urandom (non-blocking). 'High' read from '/dev/random' (blocking) and low just used SecureRandom. The Javadoc clearly documented the blocking behavior of setting the entropy pool security to 'high', but the developer never read it. Put it out into production. Worked great for awhile until one week when they had usually high traffic, they started getting timeouts which they traced to the blocking behavior. Glad I wasn't holding the pager that day! 8-) > >> Unfortunately, when trying to provide a generally reusable >> (and simple) security API that does not require ordinary developers >> to be experts in applied cryptography, one cannot go over-the-top. >> If the crypto is too hard to use, has too much CPU overhead, or >> makes the resulting cipher text excessively long, developers simply >> will not use it. The will instead just use ECB mode and not worry >> about things like cryptographic attacks that they don't know anything >> about and couldn't be bothered with anyhow, even if they did. >> >> So our goal is to ensure that the crypto is strong enough to protect >> things like credit card #s, bank account info, passwords (where >> cleartext passwords are needed for things like DB connections), >> and producing cryptographic tokens. We are not concerned about >> protecting nuclear lauch codes. Besides, it is more than likely >> if the crypto is the weakest link (and it rarely is; usually >> XSS and SQLi vulnerabilities abound), the weak part of the crypto >> will be in the way that the dev teams manage their encryption keys. > > > > Yeah. But you can eliminate weak spots like "have good entropy" without > introducing any other costs :) Well 'good' is a relative term. How 'good' is reading from /dev/urandom? Probably pretty good. But what about when you are starting up 30 or 40 WebLogic Server or Tomcat applications right after boot-up? Still 'good'? Maybe, maybe not. There's been a lot of debate about that on this list. Only thing I can say is given the high price you pay for servers, it sure is surprising that they all don't come with a standard hardware-based CRNG. >>>> So my second question is, if all the "combined" cipher modes all >>>> cause a cipher to act as if it is in a streaming mode, is it okay >>>> to just choose a completely RANDOM IV for each encryption? >>>> Because it sure doesn't seem to be feasible to record all the IVs >>>> for a given key to make sure that an IV isn't reused. If that is not >>>> acceptable, then how does one ever address this? >>> >>> >>> >>> Random is good. Also, using a counter that always goes up is good. >>> Another >>> possibility is to use a sufficiently fine timesource. >>> >>> All of these devices look good on paper but have some edge cases. One >>> way >>> is to cram them all into the IV as one lump: >>> >>> random||counter||time >> >> >> So, are you advising that rather than requesting 16 bytes of >> randomness and turning that into an IvParameterSpec, I should request >> something less and then combine it w/ a counter and time? > > > > Yes. Our thesis is that your downstream developers are as much your threat > as anyone else :) > Or developers in general! ;-) >> In Java, >> time is a long (4 bytes)? So how much for the 'random' piece and >> how much to allocate for the 'counter' portion? > > > > 4 bytes time, 4 bytes counter, 8 bytes random? > > When I wrote all this into SDP1 I did it differently, but the details were > more exotic. > > > >> And if a dev team were to use 3DES, we only have an IV of 8 bytes? >> How should that be divided up? > > > > I would take the above 16 bytes of random||counter||time, run it through > SHA3 10,000 times, then take the first 8 bytes of the result. Explain to > them that this is an NSA recommendation for improving the strength of TDES, > but really the NSA recommends AES... > Yeah...doing that ought to make it run slow enough that they will switch from TDES to AES for sure. ;-) > >> Also, if we are doing this, is it important to remember the 'counter' >> across Java VM restarts or can we just reset it to 0 at that point? > > > > Exactly - that is why the time is there. You can't reliably remember the > counter across restarts - it is impossible for the crypto designer to > predict how the counter is stored; so we have to assume that it could be > ephemeral. Indeed, we probably want it to be - imagine the complications of > insisting that the IV fields be stored in a SQL database, and what that does > to security :P So the time is in there for the moments when we are talking > about fast restarts, and counter := 0. > > If the mobo clock's battery is flat, the counter is being reset by some mad > power glitch, and entropy is a half filled glass, then you can at least say > you did the best you could. > > > >>> With most algorithms these days, you've got 16 bytes in the first block. >> >> >> Major exception being w/ 3DES, which some developers still use for >> legacy reasons. You have no idea! Sure is easier to create shitty software than it is to get rid of it though. > > If you are designing a mode and protocol, it is unclear to me why they are > insisting on using 3DES with it. > It usually is one of two lines of reasoning. One is that we have a vendor product that uses 3DES and we have not control and we've been too cheap to by maintenance for the software since 1999 or so. The other line I've given is that "this application is strictly in a maintenance-only mode; we have no developers to change it. We don't even know where the software is". Not sure how common this is in the finance industry, but it is very common in telecommunications sector. Anyhow, thanks for all the suggestions. Definitely going to use them in ESAPI 2.1, which I'll start working on as soon as GSoC is finished in late summer. Thanks again, -kevin -- Blog: http://off-the-wall-security.blogspot.com/ "The most likely way for the world to be destroyed, most experts agree, is by accident. That's where we come in; we're computer professionals. We *cause* accidents." -- Nathaniel Borenstein _______________________________________________ cryptography mailing list cryptography@randombit.net http://lists.randombit.net/mailman/listinfo/cryptography
