RE: A few thoughts on Haproxy and weakdh/logjam
On Thursday, May 28, 2015 2:29 PM, Lukas Tribus wrote: If your refer to long EOL'ed system, then they probably don't support DHE at all. Alas EOL'ed systems doesn't hinder its use - even if it unwise.. Thats not what I'm saying. What I'm saying is that since they are so old they don't even support DHE, therefor the dh group doesn't matter. Ahhh, that makes sense - thanks for clearing that up. How much dos the size of my chosen DH group affect clients and the server when negotiating the connection? *Very* much on the server side. It will kill your CPU. OK, good to know (it was a bit unclear until now). Could I (at least in theory) make a 8192 bit DH group, and not expect any performance problems? Absolutely not, no, not even in theory. Don't do this. HAProxy users have had severe performance regression because of this. OK, thanks for clearing that up. Regards, Jens Dueholm Christensen Survey IT
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Lukas, On Mon, Jun 01, 2015 at 12:36:13PM +0200, Lukas Tribus wrote: I pushed this into 1.6. I'd rather issue -dev2 with it, wait a little bit then backport it into 1.5 if we don't get any negative feedback. We might have to help distro maintainers prepare some arguments to backport this. For the record I checked out current nginx [1] and apache [2] sources and they don't seem to care about this at all. Nginx has a static 1024bit group in the source (nothing else) and Apache gets 2048bit+ groups from openssl (as we did previously). Great, thanks for checking! I still think that our approach is suboptimal, mainly because I would rather not get involved (by introducing a static key) in such advanced crypto stuff. Sure but if you think about it from a different perspective, it's a reasonable tradeoff between using a well-known param that make it a good candidate for attacks, and having one per clueless user who will never ever change it. And I count myself in the clueless users. If Hervé had not talked to me about logjam, I would have never heard about it. Now I know that I can simply upgrade haproxy to fix the issue. Having to go through a process consisting in carefully editing existing certs just to change a set of params inside is not the thought that motivates me the most in the morning to force me to get up. A proper solution or proposal should imho come from openssl. They can't possibly expect application developers to take of such low-level crypto things. I agree with you, not by laziness but because they provide the stuff that users don't have to understand to remain safe. At least a recommendation would be nice (get_rfc2409_prime_1024 is unsafe, don't use it? get_rfc2409_prime_2048 can be considered safe?). Good point. Anyway, it doesn't look like there the is a simple answer to the question about whats the right thing to do ... That's how we end up with several solutions, as usual. That makes me think about something I talk about from time to time : to have a diagnostic mode for haproxy, which goes further than warnings, by triggering on many things that might be unintentional but which cannot be fixed in a way to make the warning disappear if intentional. There are examples such as using the same cookie for several servers. And using a 1024-bit key, or even using the oakley group2 typically is such a usecase for the diag mode, where we could alert the user about something suspicious but possibly desired. We could even do it by combining the debug mode with the config check mode. I'm seeing between 2 and 5 levels possible, that might be adjusted using -d followed by an optional number : - possibly not secure unless you know what you're doing - suspicious configuration that may be wrong, but you'd rather check - it is not the correct way to do this - it is not the optimal way to do this - configuration could be simplified Here with logjam we have a good candidate for the first level, as we could even imagine detecting oakley group 2 in loaded certs and emitting a warning for them. We could have a diag() function working like Warning() and calling it, or even simpler, we change the Warning() API to pass a level (level 0 being the current one, meaning this is wrong but I could work around it). In fact we already have and I couldn't work around it, so possibly we could shift some warnings already. If someone's interested in working on this, it would be nice. Cheers, Willy
RE: A few thoughts on Haproxy and weakdh/logjam
Hi Willy, Thank you, that was pretty clear and easy. I checked that I was running with about 2 kb of entropy before the tests and that I was alone on the machine, so I'm confident that what I did wasn't skewed. I pushed this into 1.6. I'd rather issue -dev2 with it, wait a little bit then backport it into 1.5 if we don't get any negative feedback. We might have to help distro maintainers prepare some arguments to backport this. For the record I checked out current nginx [1] and apache [2] sources and they don't seem to care about this at all. Nginx has a static 1024bit group in the source (nothing else) and Apache gets 2048bit+ groups from openssl (as we did previously). I still think that our approach is suboptimal, mainly because I would rather not get involved (by introducing a static key) in such advanced crypto stuff. A proper solution or proposal should imho come from openssl. They can't possibly expect application developers to take of such low-level crypto things. At least a recommendation would be nice (get_rfc2409_prime_1024 is unsafe, don't use it? get_rfc2409_prime_2048 can be considered safe?). Anyway, it doesn't look like there the is a simple answer to the question about whats the right thing to do ... Regards, Lukas [1] http://hg.nginx.org/nginx/file/e034af368274/src/event/ngx_event_openssl.c#l905 [2] https://github.com/apache/httpd/blob/trunk/modules/ssl/ssl_engine_init.c#L70
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Rémi, On Fri, May 29, 2015 at 04:36:49PM +0200, Remi Gacogne wrote: I expect to be able to send the ssl-dh-param-file patch tomorrow, as it is mostly written (but not well tested yet), as well as the patch to move from 1024-bit DH to 2048-bit by default. Great! Do you think it would make sense to backport the ssl-dh-param-file to 1.5 ? I mean, will some users need this in the short term (or said differently, may we use this as an incentive to be more careful about that ?). Here it is. Yes, while I am of course a bit reluctant about the idea of adding a new feature in 1.5, I think it makes sense to backport this one because it makes it easier to use custom DH parameters, which is the best option security-wise. I agree. Note that if we decide to go the safe way by not backporting it, it is still possible to work around and do the same thing by adding custom DH parameters to each cert file. Sure but I can already bet that most users who have their DH params in their cert file will not update them anyway :-/ Also for 1.5.13 as I understand it, I should regenerate a new dhparam-1024 to get rid of oakley group 2. I'll need some directions on how to do this correctly. Yes, of course. I am attaching a patch that replace all the hard-coded DH parameters by new ones, removing the 8192-bit one in the process because I don't think it will ever be used (it's just too CPU-intensive, especially now that ECDHE is widely available). Just replace the content of dh1024_p, dh1024_g, dh2048_p, dh2048_g, dh4096_p and dh4096_g by the values you get from running those commands on your own host (preferably with some entropy available): $ openssl dhparam 1024 -C $ openssl dhparam 2048 -C $ openssl dhparam 4096 -C Please don't hesitate to get back to me if needed, I know I have the bad habit of skipping crucial steps in my explanations. Thank you, that was pretty clear and easy. I checked that I was running with about 2 kb of entropy before the tests and that I was alone on the machine, so I'm confident that what I did wasn't skewed. I pushed this into 1.6. I'd rather issue -dev2 with it, wait a little bit then backport it into 1.5 if we don't get any negative feedback. We might have to help distro maintainers prepare some arguments to backport this. Thanks, Willy
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Willy, On 05/28/2015 06:19 PM, Willy Tarreau wrote: RSA 1024 is (at last) being phased out, and we have seen on this mailing-list that a DH greater than 2048-bit is simply too costly for a lot of users. OK so that means that probably people will prefer to build their own DH-1024 to use with 2048 keys. I hope most users are going to use DH-2048 with RSA-4096 instead, but yeah, that's the idea :) I didn't have enough time to implement the 3 patches I promised yet, I'm not the one who will blame you for that, if you knew the number of things I don't have the time to do! I can't even begin to imagine! I expect to be able to send the ssl-dh-param-file patch tomorrow, as it is mostly written (but not well tested yet), as well as the patch to move from 1024-bit DH to 2048-bit by default. Great! Do you think it would make sense to backport the ssl-dh-param-file to 1.5 ? I mean, will some users need this in the short term (or said differently, may we use this as an incentive to be more careful about that ?). Here it is. Yes, while I am of course a bit reluctant about the idea of adding a new feature in 1.5, I think it makes sense to backport this one because it makes it easier to use custom DH parameters, which is the best option security-wise. Note that if we decide to go the safe way by not backporting it, it is still possible to work around and do the same thing by adding custom DH parameters to each cert file. Also for 1.5.13 as I understand it, I should regenerate a new dhparam-1024 to get rid of oakley group 2. I'll need some directions on how to do this correctly. Yes, of course. I am attaching a patch that replace all the hard-coded DH parameters by new ones, removing the 8192-bit one in the process because I don't think it will ever be used (it's just too CPU-intensive, especially now that ECDHE is widely available). Just replace the content of dh1024_p, dh1024_g, dh2048_p, dh2048_g, dh4096_p and dh4096_g by the values you get from running those commands on your own host (preferably with some entropy available): $ openssl dhparam 1024 -C $ openssl dhparam 2048 -C $ openssl dhparam 4096 -C Please don't hesitate to get back to me if needed, I know I have the bad habit of skipping crucial steps in my explanations. -- Rémi From fe610d85b580c8dcf2aa9e07d5ed98abd6604ebd Mon Sep 17 00:00:00 2001 From: Remi Gacogne rgaco...@aquaray.fr Date: Fri, 29 May 2015 15:53:22 +0200 Subject: [PATCH] MEDIUM: ssl: add the possibility to use a global DH parameters file This patch adds the ssl-dh-param-file global setting. It sets the default DH parameters that will be used during the SSL/TLS handshake when ephemeral Diffie-Hellman (DHE) key exchange is used, for all bind lines which do not explicitely define theirs. --- doc/configuration.txt| 17 ++- include/proto/ssl_sock.h | 3 +++ src/cfgparse.c | 18 +++ src/ssl_sock.c | 57 +--- 4 files changed, 81 insertions(+), 14 deletions(-) diff --git a/doc/configuration.txt b/doc/configuration.txt index 9676643..655ede0 100644 --- a/doc/configuration.txt +++ b/doc/configuration.txt @@ -766,6 +766,20 @@ ssl-default-server-options [option]... default ssl-options to force on all server lines. Please check the server keyword to see available options. +ssl-dh-param-file file + This setting is only available when support for OpenSSL was built in. It sets + the default DH parameters that are used during the SSL/TLS handshake when + ephemeral Diffie-Hellman (DHE) key exchange is used, for all bind lines + which do not explicitely define theirs. It will be overridden by custom DH + parameters found in a bind certificate file if any. If custom DH parameters + are not specified either by using ssl-dh-param-file or by setting them directly + in the certificate file, pre-generated DH parameters of the size specified + by tune.ssl.default-dh-param will be used. Custom parameters are known to be + more secure and therefore their use is recommended. + Custom DH parameters may be generated by using the OpenSSL command + openssl dhparam size, where size should be at least 2048, as 1024-bit DH + parameters should not be considered secure anymore. + ssl-server-verify [none|required] The default behavior for SSL verify on servers side. If specified to 'none', servers certificates are not verified. The default is 'required' except if @@ -1224,7 +1238,8 @@ tune.ssl.default-dh-param number this maximum value. Default value if 1024. Only 1024 or higher values are allowed. Higher values will increase the CPU load, and values greater than 1024 bits are not supported by Java 7 and earlier clients. This value is not - used if static Diffie-Hellman parameters are supplied via the certificate file. + used if static Diffie-Hellman parameters are supplied either directly + in the certificate file or by using the ssl-dh-param-file parameter.
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Julien, On 05/27/2015 12:05 PM, Julien Vehent wrote: This is by far the best write-up on DHE compatibility issues I've seen. Would you mind organizing your research into something we could publish on https://wiki.mozilla.org/Security/Server_Side_TLS ? I've added some notes about compatibility between DHE and Java, but nothing concerning the pre-defined DH groups yet. If you had time to submit it as a pull-request on https://github.com/mozilla/server-side-tls/blob/gh-pages/Server_Side_TLS.mediawiki that would be absolutely fantastic! I see that you have already added a new section about DHE and Java, that's great! I will have at look at whether I can improve the page and will submit a pull request if it makes sense. By the way this document is awesome, thank your for writing it! -- Rémi signature.asc Description: OpenPGP digital signature
RE: A few thoughts on Haproxy and weakdh/logjam
On Tuesday, May 26, 2015 5:12 PM Remi Gacogne wrote: On 05/23/2015 08:47 AM, Willy Tarreau wrote: Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? Basically, by totally removing DHE, we would be losing forward secrecy for: - Java = 6 ; - OpenSSL = 1.0.0 ; - Android = 3. Note that moving to a DH size of 2048-bit is an issue if you have Java 6 clients anyway (Java 7 does not support DHE 1024-bit either, but does support ECDHE). What about other clients (ie. browsers running on different OS combinations) - especially legacy systems? If your refer to long EOL'ed system, then they probably don't support DHE at all. Will IE7 on Windows XP have problems if I change to a 2048 or even a 4096 DH group? Scannel on Windows XP doesn't support DHE with RSA, therefor IE6/7/8 will connect just fine (without FS). If changing to a higher DH group breaks connectivity with even just a few ordinary browser/OS combinations I am afraid that I have no choice to stick with the current vunerable group.. DHE is not necessary to connect. Your legacy clients will just negotiate a non-FS ciphersuite. Lukas
RE: A few thoughts on Haproxy and weakdh/logjam
On Tuesday, May 26, 2015 5:12 PM Remi Gacogne wrote: On 05/23/2015 08:47 AM, Willy Tarreau wrote: Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? Basically, by totally removing DHE, we would be losing forward secrecy for: - Java = 6 ; - OpenSSL = 1.0.0 ; - Android = 3. Note that moving to a DH size of 2048-bit is an issue if you have Java 6 clients anyway (Java 7 does not support DHE 1024-bit either, but does support ECDHE). What about other clients (ie. browsers running on different OS combinations) - especially legacy systems? Will IE7 on Windows XP have problems if I change to a 2048 or even a 4096 DH group? What about other combinations of client/OS? We've only offer TLS (no SSLv3) and use Mozillas Intermediate compatibility ciphersuite, but I need to keep backwards compatibility with a lot of legacy clients that I have no control over. If changing to a higher DH group breaks connectivity with even just a few ordinary browser/OS combinations I am afraid that I have no choice to stick with the current vunerable group.. Regards, Jens Dueholm Christensen Survey IT
RE: A few thoughts on Haproxy and weakdh/logjam
On Thursday, May 28, 2015 12:35 PM Lukas Tribus wrote: What about other clients (ie. browsers running on different OS combinations) - especially legacy systems? If your refer to long EOL'ed system, then they probably don't support DHE at all. Alas EOL'ed systems doesn't hinder its use - even if it unwise.. We've also got customers who I know are paying extortionately large amounts of money to Microsoft for continued Windows XP support.. Will IE7 on Windows XP have problems if I change to a 2048 or even a 4096 DH group? Scannel on Windows XP doesn't support DHE with RSA, therefor IE6/7/8 will connect just fine (without FS). I assume you mean Schannel, and yes - I just did a small test on a public low volume site using a VM based IE7 and SSLLabs SSLTest[1], and can see that both IE7 and IE8 on Windows XP uses the cipher TLS_RSA_WITH_3DES_EDE_CBC_SHA (the OpenSSL name is DES-CBC3-SHA) when connecting. As far as I can see the only client that cannot connect in that test is a Java 1.6 based one - all others are fine (just as you said). A follow up question: How much dos the size of my chosen DH group affect clients and the server when negotiating the connection? The SSLLabs test did not take any longer using a 4096 bit DH group instead of a 2048bit one. Could I (at least in theory) make a 8192 bit DH group, and not expect any performance problems? Regards, Jens Dueholm Christensen Survey IT [1]: https://www.ssllabs.com/ssltest/index.html
RE: A few thoughts on Haproxy and weakdh/logjam
If your refer to long EOL'ed system, then they probably don't support DHE at all. Alas EOL'ed systems doesn't hinder its use - even if it unwise.. Thats not what I'm saying. What I'm saying is that since they are so old they don't even support DHE, therefor the dh group doesn't matter. Scannel on Windows XP doesn't support DHE with RSA, therefor IE6/7/8 will connect just fine (without FS). I assume you mean Schannel, and yes - I just did a small test on a public low volume site using a VM based IE7 and SSLLabs SSLTest[1], and can see that both IE7 and IE8 on Windows XP uses the cipher TLS_RSA_WITH_3DES_EDE_CBC_SHA (the OpenSSL name is DES-CBC3-SHA) when connecting. As far as I can see the only client that cannot connect in that test is a Java 1.6 based one - all others are fine (just as you said). Ok, thanks for confirming. A follow up question: How much dos the size of my chosen DH group affect clients and the server when negotiating the connection? *Very* much on the server side. It will kill your CPU. The SSLLabs test did not take any longer using a 4096 bit DH group instead of a 2048bit one. Because you have 1 server dedicated to 1 client. Also SSLLabs is not exactly a performance test. Could I (at least in theory) make a 8192 bit DH group, and not expect any performance problems? Absolutely not, no, not even in theory. Don't do this. HAProxy users have had severe performance regression because of this. Lukas
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Rémi, On Thu, May 28, 2015 at 05:45:43PM +0200, Remi Gacogne wrote: Just a question, does it make sense to have different dh-param files per key size so that depending on the cert key size we use a different file, or are they totally decorrelated ? I used to think that it made sense, but clearly the trend is to decorrelate the two. OK. RSA 1024 is (at last) being phased out, and we have seen on this mailing-list that a DH greater than 2048-bit is simply too costly for a lot of users. OK so that means that probably people will prefer to build their own DH-1024 to use with 2048 keys. I am pretty sure that there is no use anymore trying to adapt the DH size to the RSA key size. We should probably remove this logic in 1.6 and just use the size specified by tune.ssl.default-dh-param, regardless of what the RSA key size is. That's fine by me. I didn't have enough time to implement the 3 patches I promised yet, I'm not the one who will blame you for that, if you knew the number of things I don't have the time to do! so here is the most important one for now, fixing the bug reported by Hervé Commowick. I think it should be backported to 1.5. OK I merged it, I had to apply it by hand on 1.5 but it was OK. I added a small patch for 1.6 that add a destructor to clean up at exit the memory allocated internally by OpenSSL. It makes my life easier when using valgrind to check for leaks, so I figured it could be of interest to others as well :) Thank you, I've applied it to both branches since people tend to debug 1.5 as well (especially distro maintainers who might get bug reports on their own distro channels). I expect to be able to send the ssl-dh-param-file patch tomorrow, as it is mostly written (but not well tested yet), as well as the patch to move from 1024-bit DH to 2048-bit by default. Great! Do you think it would make sense to backport the ssl-dh-param-file to 1.5 ? I mean, will some users need this in the short term (or said differently, may we use this as an incentive to be more careful about that ?). Also for 1.5.13 as I understand it, I should regenerate a new dhparam-1024 to get rid of oakley group 2. I'll need some directions on how to do this correctly. Thanks! Willy
Re: A few thoughts on Haproxy and weakdh/logjam
I think the new setting for a global dhparam file is a great idea. On May 26, 2015 11:12 AM, Remi Gacogne rgaco...@aquaray.fr wrote: Hi, On 05/23/2015 08:47 AM, Willy Tarreau wrote: Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? Basically, by totally removing DHE, we would be losing forward secrecy for: - Java = 6 ; - OpenSSL = 1.0.0 ; - Android = 3. Note that moving to a DH size of 2048-bit is an issue if you have Java 6 clients anyway (Java 7 does not support DHE 1024-bit either, but does support ECDHE). Then I'd insert a 5th option between 3 and 4 above : use an haproxy-specific version, which becomes a target but has less chances of being targetted if it only represents a small percentage of the traffic and requires a nation- wide grid to pre-compute the values. I mean, this could be an option that would even improve the situation for the stable branches. All users of 1.5 running with dhparam 1024 could use a different one than the known vulnerable one. That would not make them safe, but less likely vulnerable. The doc should be clear about this though. Ok, moving from Oakley group 2 by default seems to be a smart move anyway. We probably should use this opportunity to move to non-standardized groups for all group sizes. It can't hurt, especially since it took so long to cryptographers to realize that using the Oakley group 2 everywhere was a liability. And for future versions we'd support loading it from a file, and emit a warning whenever a pre-computed 1024 is used without being picked from a file. Yes, I think it's a good idea. I will post 3 proposals for 1.6 later this week: - a proper fix for the bug found by Hervé Commowick (thanks for the bug report and the testing, by the way) ; - a new configuration option, something like ssl-dh-param-file, allowing the use of a global DH parameters file (which may still be overridden by setting the DH parameters directly in a certificate file) ; - a patch to alter the default DH size from 1024-bit to 2048-bit. I think it may be a good idea to backport the first one to 1.5, and to add one replacing default groups with random, non-standardized ones. I would rather prefer someone from HAProxy generate the new ones, for transparency's sake, but I can help for the process :) I was thinking about what Lukas Tribus said in an other thread, and I agree that the DH configuration is too complicated. Maybe we should try to deprecate the pre-computed DH parameters in 1.6 by issuing a warning whenever they are used, ie then at least one ciphersuite uses a DHE key exchange and no user-specified DH parameters are used (globally or in the certificate file). But as you pointed out, Willy, it means that we cannot fix invalid / weak groups that are stored in files (most probably forever) in the future, so I am not sure what the smartest move is. -- Rémi
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Rémi, On Tue, May 26, 2015 at 05:11:36PM +0200, Remi Gacogne wrote: Hi, On 05/23/2015 08:47 AM, Willy Tarreau wrote: Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? Basically, by totally removing DHE, we would be losing forward secrecy for: - Java = 6 ; - OpenSSL = 1.0.0 ; - Android = 3. Note that moving to a DH size of 2048-bit is an issue if you have Java 6 clients anyway (Java 7 does not support DHE 1024-bit either, but does support ECDHE). Ah good point. Now I remember you explained that to us already in the past. Then I'd insert a 5th option between 3 and 4 above : use an haproxy-specific version, which becomes a target but has less chances of being targetted if it only represents a small percentage of the traffic and requires a nation- wide grid to pre-compute the values. I mean, this could be an option that would even improve the situation for the stable branches. All users of 1.5 running with dhparam 1024 could use a different one than the known vulnerable one. That would not make them safe, but less likely vulnerable. The doc should be clear about this though. Ok, moving from Oakley group 2 by default seems to be a smart move anyway. We probably should use this opportunity to move to non-standardized groups for all group sizes. It can't hurt, especially since it took so long to cryptographers to realize that using the Oakley group 2 everywhere was a liability. Yes I think so as well and that's the issue I'm trying to avoid to re-induce in the future. And for future versions we'd support loading it from a file, and emit a warning whenever a pre-computed 1024 is used without being picked from a file. Yes, I think it's a good idea. I will post 3 proposals for 1.6 later this week: - a proper fix for the bug found by Hervé Commowick (thanks for the bug report and the testing, by the way) ; Yes, definitely. - a new configuration option, something like ssl-dh-param-file, allowing the use of a global DH parameters file (which may still be overridden by setting the DH parameters directly in a certificate file) ; Just a question, does it make sense to have different dh-param files per key size so that depending on the cert key size we use a different file, or are they totally decorrelated ? - a patch to alter the default DH size from 1024-bit to 2048-bit. OK. I think it may be a good idea to backport the first one to 1.5, and to add one replacing default groups with random, non-standardized ones. Agreed, based on your explanations. I would rather prefer someone from HAProxy generate the new ones, for transparency's sake, but I can help for the process :) I can do it if you prefer, after all I'm committing most of the code so if people don't trust me there for the key, I'd urge them to switch to another project because my bugs probably have more impact. And we could go further, by mentionning in the doc how it was generated and how to replace it. I was thinking about what Lukas Tribus said in an other thread, and I agree that the DH configuration is too complicated. Maybe we should try to deprecate the pre-computed DH parameters in 1.6 by issuing a warning whenever they are used, ie then at least one ciphersuite uses a DHE key exchange and no user-specified DH parameters are used (globally or in the certificate file). But as you pointed out, Willy, it means that we cannot fix invalid / weak groups that are stored in files (most probably forever) in the future, so I am not sure what the smartest move is. I think that we could reserve the keyword default for the dhparam file, or default-1024, default-2048, etc... which makes it clear enough about the fact that it's shipped with the product. People who care about security tend to stay away from defaults and can use their own file instead. And those who don't understand anything about this will leave it to the default and trust their distro to ship updated keys if a breach is found. It avoids one level of fallback, as it's the user who decides where the params come from. And it participates in educating users, telling them that DHE doesn't work by itself and that it needs some parameters, which could possibly be weakened and which they should take care of. It also offers a reasonable solution against breaches in well-known dh-params that users could put into their own files (because that's what will happen by copy-paste), as the quick fix for those who don't know how to regenerate that or fear to make thing worse would simply have to replace their file with default-2048 for example to switch back to something supposedly safe but widely used. Regards, Willy
Re: A few thoughts on Haproxy and weakdh/logjam
Hi, On 05/23/2015 08:47 AM, Willy Tarreau wrote: Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? Basically, by totally removing DHE, we would be losing forward secrecy for: - Java = 6 ; - OpenSSL = 1.0.0 ; - Android = 3. Note that moving to a DH size of 2048-bit is an issue if you have Java 6 clients anyway (Java 7 does not support DHE 1024-bit either, but does support ECDHE). Then I'd insert a 5th option between 3 and 4 above : use an haproxy-specific version, which becomes a target but has less chances of being targetted if it only represents a small percentage of the traffic and requires a nation- wide grid to pre-compute the values. I mean, this could be an option that would even improve the situation for the stable branches. All users of 1.5 running with dhparam 1024 could use a different one than the known vulnerable one. That would not make them safe, but less likely vulnerable. The doc should be clear about this though. Ok, moving from Oakley group 2 by default seems to be a smart move anyway. We probably should use this opportunity to move to non-standardized groups for all group sizes. It can't hurt, especially since it took so long to cryptographers to realize that using the Oakley group 2 everywhere was a liability. And for future versions we'd support loading it from a file, and emit a warning whenever a pre-computed 1024 is used without being picked from a file. Yes, I think it's a good idea. I will post 3 proposals for 1.6 later this week: - a proper fix for the bug found by Hervé Commowick (thanks for the bug report and the testing, by the way) ; - a new configuration option, something like ssl-dh-param-file, allowing the use of a global DH parameters file (which may still be overridden by setting the DH parameters directly in a certificate file) ; - a patch to alter the default DH size from 1024-bit to 2048-bit. I think it may be a good idea to backport the first one to 1.5, and to add one replacing default groups with random, non-standardized ones. I would rather prefer someone from HAProxy generate the new ones, for transparency's sake, but I can help for the process :) I was thinking about what Lukas Tribus said in an other thread, and I agree that the DH configuration is too complicated. Maybe we should try to deprecate the pre-computed DH parameters in 1.6 by issuing a warning whenever they are used, ie then at least one ciphersuite uses a DHE key exchange and no user-specified DH parameters are used (globally or in the certificate file). But as you pointed out, Willy, it means that we cannot fix invalid / weak groups that are stored in files (most probably forever) in the future, so I am not sure what the smartest move is. -- Rémi signature.asc Description: OpenPGP digital signature
Re: A few thoughts on Haproxy and weakdh/logjam
Generating dhparams can result in wildly different runtimes... Just running a dhparm 1024 here resulted in times between 1.3 and 12 second... I've generated a bunch of 2048 dhparams, which took between 1 and 30 minutes depends on luck, finding the right set of good primes. But generating 1024 should normally not take too long, maybe too long to do at startup, but something to do at the first startup when a dhparam file isn't present. Mark On Sat, May 23, 2015 at 8:47 AM, Willy Tarreau w...@1wt.eu wrote: Hi Rémi, On Fri, May 22, 2015 at 10:53:08AM +0200, Remi Gacogne wrote: Otherwise it makes no sense, sorry about that. ah ? Well, with the previous command I was basically saying if a DH 2048-bit group is too much CPU-consuming for you, just use a 2048-bit group, which makes no sense at all! Ah OK, thanks! I think you have to realize that I don't understand anything at all here, I have no idea with an Oakley group 2 is. I'm just a regular user when it comes to SSL. Is it the thing that is assigned by default when using default-dh-param ? Ah, I really have to find the time to write a better documentation for this! :-) When you use the openssl dhparam command, it generates a custom, random DH group (more or less a set of possible values) of the specified size. If this group is added to the certificate file, Haproxy will use it for DHE key exchanges for all binds using that certificate file, no matter what the value of default-dh-param is. If you don't set a custom DH group in the certificate file, Haproxy will use one of the pre-configured ones based on the MIN(default-dh-param, certificate key size): - Oakley group 2 for 1024-bit - MODP group 14 for 2048-bit - MODP group 16 for 4096-bit - MODP group 18 for 8192-bit These groups are fine, they work correctly, but it may be possible to pre-compute all the possible values in a given DH group = 1024. The most used group on the internet is the Oakley group 2, because it was standardized in rfc2409, and therefore it is now advised not to use it. The best fix is simply to move to a larger group, containing so much more values that pre-computation seems out of reach for the time being. This can be done by setting default-dh-param to 2048, or by using a custom group of size 2048. Thanks for the detailed explanation, it is very clear now. However this upgrade will impact the CPU-load, so a possible, weaker fix is to move to a custom 1024 group. Out of curiosity I just checked and my desktop PC (core2quad 3 GHz) takes 2.9s to generate a new 1024 dhparam, so it could sound acceptable to say that if people want to use 1024 (for performance reasons), we could have to generate it during startup. People running on lower performance machines would be impacted so probably they would just use a fixed larger dhparam, impacting the CPU usage for traffic, but on the other hand when you're using a small CPU you probably don't care much about performance. Because this custom group is randomly generated by openssl dhparam, you are probably the only one using it, so there is a lot less benefit for an adversary to try to pre-compute this group. And it is very likely not possible to pre-compute every possible group at this time. Yes absolutely. I'm also thinking that we should support loading the dhparam from a file that every user could generate. This would improve the situation even for those with small CPUs because their startup script could generate this file the first time. FWIW I just tried generating a 2048-bit dhparam, and it took 8mn33 on the machine where the 1024-bit took 2.9s. So probably generating this on the fly upon every startup was a bad idea in the end and doing it once per deployment is better. Note that we could move to another default 1024-bit group in haproxy, but as long as we ship one, it becomes a valuable target because it will be used by a lot of people, so I believe the correct fix is to move to a 2048-bit group by default, as you suggested in another thread :) Sure, but you remember that some people were seriously impacted, reason why I'm still searching the best solution for them. I mean, if their performance is impacted too much, they'll just force dhparam 1024 in the config, resulting in falling back to the well-known, vulnerable one and we want to avoid that. In this case, does it mean that generating a random dh-param at boot would solve the issue ? Yes, or even only once. The basic idea is: - use ECDHE instead of DHE if you can, but not all clients support that Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? - failing that, use a large DH group, a least 2048-bit - failing that, use a group you generated yourself randomly, so that not everybody is using the same - failing that, it is better to use the Oakley group
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Rémi, On Fri, May 22, 2015 at 10:53:08AM +0200, Remi Gacogne wrote: Otherwise it makes no sense, sorry about that. ah ? Well, with the previous command I was basically saying if a DH 2048-bit group is too much CPU-consuming for you, just use a 2048-bit group, which makes no sense at all! Ah OK, thanks! I think you have to realize that I don't understand anything at all here, I have no idea with an Oakley group 2 is. I'm just a regular user when it comes to SSL. Is it the thing that is assigned by default when using default-dh-param ? Ah, I really have to find the time to write a better documentation for this! :-) When you use the openssl dhparam command, it generates a custom, random DH group (more or less a set of possible values) of the specified size. If this group is added to the certificate file, Haproxy will use it for DHE key exchanges for all binds using that certificate file, no matter what the value of default-dh-param is. If you don't set a custom DH group in the certificate file, Haproxy will use one of the pre-configured ones based on the MIN(default-dh-param, certificate key size): - Oakley group 2 for 1024-bit - MODP group 14 for 2048-bit - MODP group 16 for 4096-bit - MODP group 18 for 8192-bit These groups are fine, they work correctly, but it may be possible to pre-compute all the possible values in a given DH group = 1024. The most used group on the internet is the Oakley group 2, because it was standardized in rfc2409, and therefore it is now advised not to use it. The best fix is simply to move to a larger group, containing so much more values that pre-computation seems out of reach for the time being. This can be done by setting default-dh-param to 2048, or by using a custom group of size 2048. Thanks for the detailed explanation, it is very clear now. However this upgrade will impact the CPU-load, so a possible, weaker fix is to move to a custom 1024 group. Out of curiosity I just checked and my desktop PC (core2quad 3 GHz) takes 2.9s to generate a new 1024 dhparam, so it could sound acceptable to say that if people want to use 1024 (for performance reasons), we could have to generate it during startup. People running on lower performance machines would be impacted so probably they would just use a fixed larger dhparam, impacting the CPU usage for traffic, but on the other hand when you're using a small CPU you probably don't care much about performance. Because this custom group is randomly generated by openssl dhparam, you are probably the only one using it, so there is a lot less benefit for an adversary to try to pre-compute this group. And it is very likely not possible to pre-compute every possible group at this time. Yes absolutely. I'm also thinking that we should support loading the dhparam from a file that every user could generate. This would improve the situation even for those with small CPUs because their startup script could generate this file the first time. FWIW I just tried generating a 2048-bit dhparam, and it took 8mn33 on the machine where the 1024-bit took 2.9s. So probably generating this on the fly upon every startup was a bad idea in the end and doing it once per deployment is better. Note that we could move to another default 1024-bit group in haproxy, but as long as we ship one, it becomes a valuable target because it will be used by a lot of people, so I believe the correct fix is to move to a 2048-bit group by default, as you suggested in another thread :) Sure, but you remember that some people were seriously impacted, reason why I'm still searching the best solution for them. I mean, if their performance is impacted too much, they'll just force dhparam 1024 in the config, resulting in falling back to the well-known, vulnerable one and we want to avoid that. In this case, does it mean that generating a random dh-param at boot would solve the issue ? Yes, or even only once. The basic idea is: - use ECDHE instead of DHE if you can, but not all clients support that Do you have any idea about the ratio of clients (on the net) which don't support ECDHE right now but support DHE ? - failing that, use a large DH group, a least 2048-bit - failing that, use a group you generated yourself randomly, so that not everybody is using the same - failing that, it is better to use the Oakley group 2 than no DHE at all. OK that last point is pretty clear and very important because I was wondering if it would be better not to support it at all :-) Then I'd insert a 5th option between 3 and 4 above : use an haproxy-specific version, which becomes a target but has less chances of being targetted if it only represents a small percentage of the traffic and requires a nation- wide grid to pre-compute the values. I mean, this could be an option that would even improve the situation for the stable branches. All users of 1.5 running with dhparam 1024 could use a different one than the
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Rémi, On Fri, May 22, 2015 at 09:11:39AM +0200, Remi Gacogne wrote: Hello, On 05/22/2015 07:32 AM, Willy Tarreau wrote: That makes me think about something, as you advocated a long time ago for increasing the dh-param default size. Do you think we should take the opportunity of 1.6 to increase the default size ? It will use more CPUs for people who migrate from 1.5 only, and such people are expected to run tests during the migration anyway so they should not be surprized. I think that would be great! We could alter the warning so that people not explicitly setting the value in the configuration are aware that it is now set to 2048. Yes as well. If you cannot increase the DH key size above 1024-bit, please at least generate a custom DH group with the openssl dhparam 2048 command, and add the result of this command to your certificate file. Does that improve the situation regarding the CPU usage ? I must confess this is still very cryptic to me (no pun intended). Oh, I used the wrong group size on the openssl dhparam command, it should have been: openssl dhparam 1024 Otherwise it makes no sense, sorry about that. ah ? So yes, using a custom 1024-bit DH group instead of the default Oakley group 2 makes it a lot harder to do pre-computation while having no impact on the CPU usage. I think you have to realize that I don't understand anything at all here, I have no idea with an Oakley group 2 is. I'm just a regular user when it comes to SSL. Is it the thing that is assigned by default when using default-dh-param ? In this case, does it mean that generating a random dh-param at boot would solve the issue ? Thanks! Willy
Re: A few thoughts on Haproxy and weakdh/logjam
Hi Remi, On Thu, May 21, 2015 at 06:07:34PM +0200, Remi Gacogne wrote: In the default configuration, Haproxy uses a 1024-bit DH key generated from the second Oakley group [2] for Diffie-Hellman Ephemeral (DHE) key exchange. This group is widely used, and is likely to be the first target for pre-computation by an adversary with large enough computing capabilities. I would advise using instead a 2048-bit key generated from the MODP group 14, by setting the tune.ssl.default-dh-param parameter to 2048, or even disabling DHE altogether if you are expecting every client to support ECDHE key exchange. Note that increasing the tune.ssl.default-dh-param will increase the CPU load on your server, and may therefore increase the connection establishment latency. That makes me think about something, as you advocated a long time ago for increasing the dh-param default size. Do you think we should take the opportunity of 1.6 to increase the default size ? It will use more CPUs for people who migrate from 1.5 only, and such people are expected to run tests during the migration anyway so they should not be surprized. If you cannot increase the DH key size above 1024-bit, please at least generate a custom DH group with the openssl dhparam 2048 command, and add the result of this command to your certificate file. Does that improve the situation regarding the CPU usage ? I must confess this is still very cryptic to me (no pun intended). Thanks! Willy
A few thoughts on Haproxy and weakdh/logjam
Haproxy and weakdh/logjam Hi, Everyone has probably heard about the recently disclosed weakdh/logjam attack [0] already. Here are a few personal thoughts on the impact on Haproxy. The weakdh issue is twofold: - if the HTTPS server is willing to accept a cipher suite using a very weak Diffie-Hellman (DH) group, like for example TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, then it is possible for an adversary in position of man-in-the-middle (MitM) to downgrade the security level of a given connection to this cipher suite, thus reducing the security to 512-bit DH ; - a nation-state adversary may be able to pre-compute all the possible values of a commonly-shared 1024-bit DH group, thus being able to decrypt all exchanges using that group. The first point depends on the cipher suite specified by the administrator with the ssl-default-bind-ciphers or ciphers parameters. I strongly encourage everyone to use the modern cipher suite described on the Mozilla wiki [1]. Please at least consider using the Intermediate one. In the default configuration, Haproxy uses a 1024-bit DH key generated from the second Oakley group [2] for Diffie-Hellman Ephemeral (DHE) key exchange. This group is widely used, and is likely to be the first target for pre-computation by an adversary with large enough computing capabilities. I would advise using instead a 2048-bit key generated from the MODP group 14, by setting the tune.ssl.default-dh-param parameter to 2048, or even disabling DHE altogether if you are expecting every client to support ECDHE key exchange. Note that increasing the tune.ssl.default-dh-param will increase the CPU load on your server, and may therefore increase the connection establishment latency. If you cannot increase the DH key size above 1024-bit, please at least generate a custom DH group with the openssl dhparam 2048 command, and add the result of this command to your certificate file. Best regards, [0]: https://weakdh.org/ [1]: https://wiki.mozilla.org/Security/Server_Side_TLS [2]: https://tools.ietf.org/html/rfc2409#section-6.2 -- Remi signature.asc Description: OpenPGP digital signature
