Re: [Openvpn-devel] Progress on Version negotiation
> > I use interactive debuggers when I can, and I find that debugging > windows on windows and linux on linux works best for me. The M$ tools > have their flaws, but they do have excellent links to API documentation > & system RTL's sources. For debugging the cryptoapi interface, that > would have been useful. > The "openvpn-build" subproject actually contains two separate buildsystems: "generic" (which "windows-nsis" uses) "msvc" What most people use is "generic", which is for cross-compiling for Windows on Linux (or *NIX). A few people have tried using "msvc" in recent times and we know it will not work without some fixes. However, if someone wants to take over the maintenance of the "msvc" build system we're more than happy to grant him/her the privilege :). -- Samuli Seppänen Community Manager OpenVPN Technologies, Inc irc freenode net: mattock
Re: [Openvpn-devel] Progress on Version negotiation
On 24-Apr-14 04:17, Gert Doering wrote: I do run these on a windows 7 machine, but can't reconfigure them just for debugging OpenVPN. No, I wasn't suggesting that you do that, I was just trying to clarify what build options we have. I find "add msg() calls, build on linux, run on windows, see what breaks" more natural to me than "build on windows" :-) The doc on the page indicated that one couldn't do a 'full' install of the tools, as it generated bad code. If it had been correct, it would have meant reconfiguring them. I use interactive debuggers when I can, and I find that debugging windows on windows and linux on linux works best for me. The M$ tools have their flaws, but they do have excellent links to API documentation & system RTL's sources. For debugging the cryptoapi interface, that would have been useful. Print statements and core dumps have their place - but they remind me of punch cards and 24-hour turn-around batch jobs :-) I find it crippling that it's still the way one has to debug webserver apps, 40 years later. But it's a question of style. Whatever works for you is good. I'm glad to know that the documentation that I found was out-of-date, and that it is possible to build without commercial tools. Yep, I saw the mail exchange, and I'm quite happy that we know where *your* issues are coming from. George Ross' issues are something else, though, as he's not using windows at all... The pkcs11 code also loads certificates, so if he uses that, it could have a parallel issue. smime.p7s Description: S/MIME Cryptographic Signature
Re: [Openvpn-devel] Progress on Version negotiation
> Hi, > > On Thu, Apr 24, 2014 at 04:05:20AM -0400, Timothe Litt wrote: >>> Uh, this is a double misinformation :-) >> It's good to know that cross-compiling is an option, though >> cross-debugging (e.g. with an interactive debugger) can be an adventure too. >> >> Source of my comment was: >> >> http://community.openvpn.net/openvpn/wiki/BuildingOnWindows, which says >>> his new build system allows building OpenVPN on Windows more easily, >>> but some parts of the build may r*equire a commercial version of the >>> Visual Studio development environment.* >>> /Visual Studio 2008 Professional/is used to build OpenVPN on Windows. >>> Note that the free Express edition might not work. >>> Full installation installs*/x86 cross-tools/*which *may cause nasty, >>> hard to debug issues*. >> (The professional tools are > $1,000 US, which is not in my budget.) >> >> You may want to reword that after validating your comment. M$'s name >> for the 'free' tools is 'express edition'. The license terms vary based >> on M$'s whims, the current statement is: >>> http://www.visualstudio.com/products/visual-studio-express-vs >>> Visual Studio Express products are available at no charge and may be >>> used for commercial, production usage subject to the license terms >>> provided with each product. For example, you can use Express for >>> Windows to create apps that you can then submit for sale in the >>> Windows Store. > Yeah, I think that page needs clarification (I think you need the commercial > edition to do code signing, which is not strictly required if you use > the pre-signed tap driver bundle), *and* it needs a pointer to the other > build system page. > Added a big fat warning to the top of the page and a pointer to current documentation. Visual Studio Express also works for driver signing - I know it because I built tap-windows6 with it a while back with no issues. -- Samuli Seppänen Community Manager OpenVPN Technologies, Inc irc freenode net: mattock
Re: [Openvpn-devel] Progress on Version negotiation
Hi, On Thu, Apr 24, 2014 at 04:05:20AM -0400, Timothe Litt wrote: > >Uh, this is a double misinformation :-) > It's good to know that cross-compiling is an option, though > cross-debugging (e.g. with an interactive debugger) can be an adventure too. > > Source of my comment was: > > http://community.openvpn.net/openvpn/wiki/BuildingOnWindows, which says > >his new build system allows building OpenVPN on Windows more easily, > >but some parts of the build may r*equire a commercial version of the > >Visual Studio development environment.* > >/Visual Studio 2008 Professional/is used to build OpenVPN on Windows. > >Note that the free Express edition might not work. > >Full installation installs*/x86 cross-tools/*which *may cause nasty, > >hard to debug issues*. > (The professional tools are > $1,000 US, which is not in my budget.) > > You may want to reword that after validating your comment. M$'s name > for the 'free' tools is 'express edition'. The license terms vary based > on M$'s whims, the current statement is: > >http://www.visualstudio.com/products/visual-studio-express-vs > >Visual Studio Express products are available at no charge and may be > >used for commercial, production usage subject to the license terms > >provided with each product. For example, you can use Express for > >Windows to create apps that you can then submit for sale in the > >Windows Store. Yeah, I think that page needs clarification (I think you need the commercial edition to do code signing, which is not strictly required if you use the pre-signed tap driver bundle), *and* it needs a pointer to the other build system page. Samuli...? > The current version requires at least windows 7 and a 2.2GHz+ > processor. (My XP laptop won't do.) > The 2008 Express edition > (http://www.microsoft.com/en-us/download/confirmation.aspx?id=7940) is > also a resource hog. > It doesn't include all of the templates and other files needed to make > many kinds of applications, though it is serviceable. > > I do run these on a windows 7 machine, but can't reconfigure them just > for debugging OpenVPN. No, I wasn't suggesting that you do that, I was just trying to clarify what build options we have. I find "add msg() calls, build on linux, run on windows, see what breaks" more natural to me than "build on windows" :-) > In any case, I think that we have found root cause of this issue the > old-fashioned way - code inspection based on some debugging I did on the > server and a hint from Steffan. > > It seems that the cryptoapi interface (and probably other external key > loaders, such as pkcs11 according to James) has not be updated for > TLS1.2. TLS1.2 adds some new signatures. The error that I saw comes - > I believe - from code that sanity checks the requested hash size against > the generated hash size; cryptoapi only knows how to generate md5/sha1 > signatures. Yep, I saw the mail exchange, and I'm quite happy that we know where *your* issues are coming from. George Ross' issues are something else, though, as he's not using windows at all... > > This makes it clear that: > - the key loaders need to be updated for TLS1.2 This includes the > cryptoAPI on windows, pkcs11, and the cert stores on other platforms > (IOS, Android, Mac - if that's ever merged). > - There does need to be a way to specify a maximum TLS version (1.1 > will do in this case) [..] Good points. We're having an IRC meeting tonight, and I hope that James, Steffan and Arne can agree on a short-term approach for 2.3.4, and a long-term approach for master/2.4 - what you describe sounds good to me, but I'm not the one who is going to implement and maintain it, so I'm not deciding. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpno53dDhwfI.pgp Description: PGP signature
Re: [Openvpn-devel] Progress on Version negotiation
Uh, this is a double misinformation :-)
It's good to know that cross-compiling is an option, though
cross-debugging (e.g. with an interactive debugger) can be an adventure too.
Source of my comment was:
http://community.openvpn.net/openvpn/wiki/BuildingOnWindows, which says
his new build system allows building OpenVPN on Windows more easily,
but some parts of the build may r*equire a commercial version of the
Visual Studio development environment.*
/Visual Studio 2008 Professional/is used to build OpenVPN on Windows.
Note that the free Express edition might not work.
Full installation installs*/x86 cross-tools/*which *may cause nasty,
hard to debug issues*.
(The professional tools are > $1,000 US, which is not in my budget.)
You may want to reword that after validating your comment. M$'s name
for the 'free' tools is 'express edition'. The license terms vary based
on M$'s whims, the current statement is:
http://www.visualstudio.com/products/visual-studio-express-vs
Visual Studio Express products are available at no charge and may be
used for commercial, production usage subject to the license terms
provided with each product. For example, you can use Express for
Windows to create apps that you can then submit for sale in the
Windows Store.
The current version requires at least windows 7 and a 2.2GHz+
processor. (My XP laptop won't do.)
The 2008 Express edition
(http://www.microsoft.com/en-us/download/confirmation.aspx?id=7940) is
also a resource hog.
It doesn't include all of the templates and other files needed to make
many kinds of applications, though it is serviceable.
I do run these on a windows 7 machine, but can't reconfigure them just
for debugging OpenVPN.
In any case, I think that we have found root cause of this issue the
old-fashioned way - code inspection based on some debugging I did on the
server and a hint from Steffan.
It seems that the cryptoapi interface (and probably other external key
loaders, such as pkcs11 according to James) has not be updated for
TLS1.2. TLS1.2 adds some new signatures. The error that I saw comes -
I believe - from code that sanity checks the requested hash size against
the generated hash size; cryptoapi only knows how to generate md5/sha1
signatures.
This makes it clear that:
- the key loaders need to be updated for TLS1.2 This includes the
cryptoAPI on windows, pkcs11, and the cert stores on other platforms
(IOS, Android, Mac - if that's ever merged).
- There does need to be a way to specify a maximum TLS version (1.1
will do in this case)
- I'm inclined to have components, such as the key loaders, specify
their min/max requirements so that if you specify an option (e.g.
cryptoapicert), the auto-negotiation does the right thing
transparently. And as things get fixed, the auto-negotiation will
upgrade to higher security. A quick fix would be for the options
invoking these features to adjust & lock the version max (and I suppose
min) value - that's all in options.h,c. Long term, a more expressive
API would be better.
James is suggesting that --tls-version-{min,max} should be the only
controls. The advantage is that the code is localized; the disadvantage
is that the config file writer gets involved. And since once 'things
work' they aren't changed, I suspect people will tend to stay with less
secure configurations forever. Especially on the client end.
I'll leave sorting that out to you folks.
Timothe Litt
ACM Distinguished Engineer
--
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.
On 24-Apr-14 03:01, Gert Doering wrote:
Hi,
On Wed, Apr 23, 2014 at 07:21:48PM -0400, Timothe Litt wrote:
As I can't build the windows client (it's really annoying that it
requires commercial tools), further debug will need help from folks who can.
Uh, this is a double misinformation :-)
- you can build with MSVC, and you can get a MSVC version for free for
some sort of "non-commercial use" (I'm not sure about the exact terms) -
but you don't really want that, as building on windows is more painful
than the alternative
- you build with mingw64 on linux, see here:
https://community.openvpn.net/openvpn/wiki/BuildingUsingGenericBuildsystem
(basically cross-compiling). It's still not as easy as building on
the linux target platform, but I find it much easier than with MSVC.
gert
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [Openvpn-devel] Progress on Version negotiation
Hi, On Wed, Apr 23, 2014 at 07:21:48PM -0400, Timothe Litt wrote: > As I can't build the windows client (it's really annoying that it > requires commercial tools), further debug will need help from folks who can. Uh, this is a double misinformation :-) - you can build with MSVC, and you can get a MSVC version for free for some sort of "non-commercial use" (I'm not sure about the exact terms) - but you don't really want that, as building on windows is more painful than the alternative - you build with mingw64 on linux, see here: https://community.openvpn.net/openvpn/wiki/BuildingUsingGenericBuildsystem (basically cross-compiling). It's still not as easy as building on the linux target platform, but I find it much easier than with MSVC. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpIEiGoRXHsI.pgp Description: PGP signature
Re: [Openvpn-devel] Progress on Version negotiation
On 23/04/2014 18:22, Timothe Litt wrote: I don't see that cryptoapi.c has been updated to work with TLS 1.2. Yes, just came to the same conclusion. Long-term the key-loaders need to get updated. Maybe short-term the options that invoke them could force NO_TLSv_1_2... That would make things work for most people in the short term. One option would be to have a tls-version-max, but I'm wondering if this might be overkill. It would also force people to add "tls-version-max 1.0" to their configs to go back to the original 2.3 behavior. My preferred solution is to simply turn off tls-version-min if it's not specified in the config, and use the 2.3 behavior. That basically forces TLS 1.0. I've seen a lot more breakage than just this... I believe the first significant real-world exposure was the iOS 1.0.2 and 1.0.3 releases from several months ago. There were hundreds of reports of breakage, mostly from countries behind government firewalls. This was using OpenVPN 3 with PolarSSL, so the issue seems to occur with different OpenVPN and SSL implementations. James
Re: [Openvpn-devel] Progress on Version negotiation
I don't see that cryptoapi.c has been updated to work with TLS 1.2.
Yes, just came to the same conclusion.
Long-term the key-loaders need to get updated.
Maybe short-term the options that invoke them could force NO_TLSv_1_2...
That would make things work for most people in the short term.
On 23-Apr-14 19:57, James Yonan wrote:
On 23/04/2014 17:21, Timothe Litt wrote:
On 23-Apr-14 16:06, Steffan Karger wrote:
I generated a matching pair of traces of the failure (client and
server)
& posted a summary.
Let me know if you would like the full traces.
Sent off-list.
I've been trying to reproduce the error. I grabbed my spare pi from the
desk drawer and built 2.3.3 from sources like you describe in #385. I
fired up a Windows 8.1 VM, and installed OpenVPN 2.3.3-I002 (x64). This
setup however happily connects with TLSv1.2. It's hard to get a hold on
this one...
My windows client is XP, 32-bit. It's a physical machine (old
notebook).
Although the problem first surfaced on 2.3.3, I'm now running off
git-master (of a few days back). How I did that is in #385 too.
For those on the list:
With a hint from Steffan,
I have established that the client certificate is not being sent. I
believe this is a client issue.
It appears that including the cert/key in the conf file, rather than
from the crypto API makes 1.2 work.
We suspect there's some issue due to 1.2's larger hashes/message size.
Not clear whether the cryptoapi interface is the cause or a victim.
As I can't build the windows client (it's really annoying that it
requires commercial tools), further debug will need help from folks who
can.
As always, let me know if I can do anything more to help.
I don't see that cryptoapi.c has been updated to work with TLS 1.2.
Note the comment in rsa_priv_enc that says "For now, we only support
NID_md5_sha1". But TLS 1.2 appears to require the support of
additional hash algorithms.
See section 4.7. "Cryptographic Attributes" in TLS 1.2 RFC.
Note that the signature algorithm is now specified along with the
signature:
struct {
SignatureAndHashAlgorithm algorithm;
opaque signature<0..2^16-1>;
} DigitallySigned;
So it would appear that any client-side private key offloading (such
as Crypto API, PKCS#11, OS-level KeyChains, etc. would need to be
aware of this feature so as to take into account the hash algorithm.
James
smime.p7s
Description: S/MIME Cryptographic Signature
Re: [Openvpn-devel] Progress on Version negotiation
On 23/04/2014 17:21, Timothe Litt wrote:
On 23-Apr-14 16:06, Steffan Karger wrote:
I generated a matching pair of traces of the failure (client and server)
& posted a summary.
Let me know if you would like the full traces.
Sent off-list.
I've been trying to reproduce the error. I grabbed my spare pi from the
desk drawer and built 2.3.3 from sources like you describe in #385. I
fired up a Windows 8.1 VM, and installed OpenVPN 2.3.3-I002 (x64). This
setup however happily connects with TLSv1.2. It's hard to get a hold on
this one...
My windows client is XP, 32-bit. It's a physical machine (old notebook).
Although the problem first surfaced on 2.3.3, I'm now running off
git-master (of a few days back). How I did that is in #385 too.
For those on the list:
With a hint from Steffan,
I have established that the client certificate is not being sent. I
believe this is a client issue.
It appears that including the cert/key in the conf file, rather than
from the crypto API makes 1.2 work.
We suspect there's some issue due to 1.2's larger hashes/message size.
Not clear whether the cryptoapi interface is the cause or a victim.
As I can't build the windows client (it's really annoying that it
requires commercial tools), further debug will need help from folks who
can.
As always, let me know if I can do anything more to help.
I don't see that cryptoapi.c has been updated to work with TLS 1.2.
Note the comment in rsa_priv_enc that says "For now, we only support
NID_md5_sha1". But TLS 1.2 appears to require the support of additional
hash algorithms.
See section 4.7. "Cryptographic Attributes" in TLS 1.2 RFC.
Note that the signature algorithm is now specified along with the signature:
struct {
SignatureAndHashAlgorithm algorithm;
opaque signature<0..2^16-1>;
} DigitallySigned;
So it would appear that any client-side private key offloading (such as
Crypto API, PKCS#11, OS-level KeyChains, etc. would need to be aware of
this feature so as to take into account the hash algorithm.
James
Re: [Openvpn-devel] Progress on Version negotiation
On 23-Apr-14 16:06, Steffan Karger wrote: I generated a matching pair of traces of the failure (client and server) & posted a summary. Let me know if you would like the full traces. Sent off-list. I've been trying to reproduce the error. I grabbed my spare pi from the desk drawer and built 2.3.3 from sources like you describe in #385. I fired up a Windows 8.1 VM, and installed OpenVPN 2.3.3-I002 (x64). This setup however happily connects with TLSv1.2. It's hard to get a hold on this one... My windows client is XP, 32-bit. It's a physical machine (old notebook). Although the problem first surfaced on 2.3.3, I'm now running off git-master (of a few days back). How I did that is in #385 too. For those on the list: With a hint from Steffan, I have established that the client certificate is not being sent. I believe this is a client issue. It appears that including the cert/key in the conf file, rather than from the crypto API makes 1.2 work. We suspect there's some issue due to 1.2's larger hashes/message size. Not clear whether the cryptoapi interface is the cause or a victim. As I can't build the windows client (it's really annoying that it requires commercial tools), further debug will need help from folks who can. As always, let me know if I can do anything more to help. smime.p7s Description: S/MIME Cryptographic Signature
Re: [Openvpn-devel] Progress on Version negotiation
Hi, On 23-04-14 17:36, Timothe Litt wrote: > Just to confirm that the issue is 1.2, not the negotiation: > > I added an unconditional > sslopt |= SSL_OP_NO_TLSv1_2; > in tls_ctx_set_options. > > With this (and the context initialized to SSL_v23_*_method, so we > negotiate), the tunnel comes up. > Without it, the tunnel does not come up. > > So it is the use of 1.2 that is the issue, not how it is selected. Good, this gives us a better starting point (and can make a temporary fix less intrusive). > I generated a matching pair of traces of the failure (client and server) > & posted a summary. > > Let me know if you would like the full traces. Yes please. You seem to have isolated the relevant parts, but just maybe I can spot something. > This is 100% reproducible here, so let me know if you need more > instrumentation. (However, I can't build a windows client, so if that's > necessary, you'll have to build it for me to run.) I've been trying to reproduce the error. I grabbed my spare pi from the desk drawer and built 2.3.3 from sources like you describe in #385. I fired up a Windows 8.1 VM, and installed OpenVPN 2.3.3-I002 (x64). This setup however happily connects with TLSv1.2. It's hard to get a hold on this one... -Steffan
Re: [Openvpn-devel] Progress on Version negotiation
Hi, On Wed, Apr 23, 2014 at 01:27:19PM -0400, Timothe Litt wrote: > >now - does that sound like it could be the problem? The initial handshake > >packet "under some conditions" (like: the local OpenSSL build having > >more available ciphers, depending on how it was built) being too big, > >causing "surprises"? > As I have noted, we're well past the client hello, so: >You are right that the number of ciphers does effect the hello size. >But if that was involved here, we should fail much sooner. OK, right. Sorry for wasting time here. I said I'm not the crypto expert... just something I've come across before. gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpLqCdaKx7Ws.pgp Description: PGP signature
Re: [Openvpn-devel] Progress on Version negotiation
Gert, while cycling home from $paidwork Cycling while thinking about TLS might be as bad as texting while driving... now - does that sound like it could be the problem? The initial handshake packet "under some conditions" (like: the local OpenSSL build having more available ciphers, depending on how it was built) being too big, causing "surprises"? As I have noted, we're well past the client hello, so: You are right that the number of ciphers does effect the hello size. But if that was involved here, we should fail much sooner. We're past the hellos, have done the certificate send from the server, the key exchange & the client cert request - which is where the client seems to be failing. Perhaps the trace that I sent will tell someone what is being sent. I have only one server set up. I did previously try different ciphers -- no change. But, I tested exactly as you requested (on the server only): --tls-cipher AES128-SHA fails --tls-cipher DHE-RSA-AES256-SHA No cipher match Set on both ends: --tls-cipher AES128-SHA fails Cipher list (Er, the "Deprecated DEFAULT, please use DEFAULT" should be fixed!) Server openvpn --tls-cipher DEFAULT --show-tls Wed Apr 23 12:53:49 2014 Deprecated TLS cipher name 'DEFAULT', please use IANA name 'DEFAULT' Available TLS Ciphers, listed in order of preference: TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384 TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384 TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA384 TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA384 TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA TLS-SRP-SHA-DSS-WITH-AES-256-CBC-SHA TLS-SRP-SHA-RSA-WITH-AES-256-CBC-SHA TLS-DHE-DSS-WITH-AES-256-GCM-SHA384 TLS-DHE-RSA-WITH-AES-256-GCM-SHA384 TLS-DHE-RSA-WITH-AES-256-CBC-SHA256 TLS-DHE-DSS-WITH-AES-256-CBC-SHA256 TLS-DHE-RSA-WITH-AES-256-CBC-SHA TLS-DHE-DSS-WITH-AES-256-CBC-SHA TLS-DHE-RSA-WITH-CAMELLIA-256-CBC-SHA TLS-DHE-DSS-WITH-CAMELLIA-256-CBC-SHA TLS-ECDH-RSA-WITH-AES-256-GCM-SHA384 TLS-ECDH-ECDSA-WITH-AES-256-GCM-SHA384 TLS-ECDH-RSA-WITH-AES-256-CBC-SHA384 TLS-ECDH-ECDSA-WITH-AES-256-CBC-SHA384 TLS-ECDH-RSA-WITH-AES-256-CBC-SHA TLS-ECDH-ECDSA-WITH-AES-256-CBC-SHA TLS-RSA-WITH-AES-256-GCM-SHA384 TLS-RSA-WITH-AES-256-CBC-SHA256 TLS-RSA-WITH-AES-256-CBC-SHA TLS-RSA-WITH-CAMELLIA-256-CBC-SHA TLS-PSK-WITH-AES-256-CBC-SHA TLS-ECDHE-RSA-WITH-3DES-EDE-CBC-SHA TLS-ECDHE-ECDSA-WITH-3DES-EDE-CBC-SHA TLS-SRP-SHA-DSS-WITH-3DES-EDE-CBC-SHA TLS-SRP-SHA-RSA-WITH-3DES-EDE-CBC-SHA TLS-DHE-RSA-WITH-3DES-EDE-CBC-SHA TLS-DHE-DSS-WITH-3DES-EDE-CBC-SHA TLS-ECDH-RSA-WITH-3DES-EDE-CBC-SHA TLS-ECDH-ECDSA-WITH-3DES-EDE-CBC-SHA TLS-RSA-WITH-3DES-EDE-CBC-SHA TLS-PSK-WITH-3DES-EDE-CBC-SHA TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256 TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256 TLS-ECDHE-RSA-WITH-AES-128-CBC-SHA256 TLS-ECDHE-ECDSA-WITH-AES-128-CBC-SHA256 TLS-ECDHE-RSA-WITH-AES-128-CBC-SHA TLS-ECDHE-ECDSA-WITH-AES-128-CBC-SHA TLS-SRP-SHA-DSS-WITH-AES-128-CBC-SHA TLS-SRP-SHA-RSA-WITH-AES-128-CBC-SHA TLS-DHE-DSS-WITH-AES-128-GCM-SHA256 TLS-DHE-RSA-WITH-AES-128-GCM-SHA256 TLS-DHE-RSA-WITH-AES-128-CBC-SHA256 TLS-DHE-DSS-WITH-AES-128-CBC-SHA256 TLS-DHE-RSA-WITH-AES-128-CBC-SHA TLS-DHE-DSS-WITH-AES-128-CBC-SHA TLS-DHE-RSA-WITH-SEED-CBC-SHA TLS-DHE-DSS-WITH-SEED-CBC-SHA TLS-DHE-RSA-WITH-CAMELLIA-128-CBC-SHA TLS-DHE-DSS-WITH-CAMELLIA-128-CBC-SHA TLS-ECDH-RSA-WITH-AES-128-GCM-SHA256 TLS-ECDH-ECDSA-WITH-AES-128-GCM-SHA256 TLS-ECDH-RSA-WITH-AES-128-CBC-SHA256 TLS-ECDH-ECDSA-WITH-AES-128-CBC-SHA256 TLS-ECDH-RSA-WITH-AES-128-CBC-SHA TLS-ECDH-ECDSA-WITH-AES-128-CBC-SHA TLS-RSA-WITH-AES-128-GCM-SHA256 TLS-RSA-WITH-AES-128-CBC-SHA256 TLS-RSA-WITH-AES-128-CBC-SHA TLS-RSA-WITH-SEED-CBC-SHA TLS-RSA-WITH-CAMELLIA-128-CBC-SHA TLS-PSK-WITH-AES-128-CBC-SHA TLS-ECDHE-RSA-WITH-RC4-128-SHA TLS-ECDHE-ECDSA-WITH-RC4-128-SHA TLS-ECDH-RSA-WITH-RC4-128-SHA TLS-ECDH-ECDSA-WITH-RC4-128-SHA TLS-RSA-WITH-RC4-128-SHA TLS-RSA-WITH-RC4-128-MD5 TLS-PSK-WITH-RC4-128-SHA TLS-DHE-RSA-WITH-DES-CBC-SHA TLS-DHE-DSS-WITH-DES-CBC-SHA TLS-RSA-WITH-DES-CBC-SHA TLS-DH-RSA-EXPORT-WITH-DES40-CBC-SHA TLS-DH-DSS-EXPORT-WITH-DES40-CBC-SHA TLS-RSA-EXPORT-WITH-DES40-CBC-SHA TLS-RSA-EXPORT-WITH-RC2-CBC-40-MD5 TLS-RSA-EXPORT-WITH-RC4-40-MD5 Client (windows) that fails: openvpn --tls-cipher DEFAULT --show-tls Wed Apr 23 12:53:49 2014 Deprecated TLS cipher name 'DEFAULT', please use IANA name 'DEFAULT' Available TLS Ciphers, listed in order of preference: TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384 TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384 TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA384 TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA384 TLS-ECDHE-RSA-WITH-AES-256-CBC-SHA TLS-ECDHE-ECDSA-WITH-AES-256-CBC-SHA TLS-SRP-SHA-DSS-WITH-AES-256-CBC-SHA TLS-SRP-SHA-RSA-WITH-AES-256-CBC-SHA TLS-DHE-DSS-WITH-AES-256-GCM-SHA384 TLS-DHE-RSA-WITH-AES-256-GCM-SHA384 TLS-DHE-RSA-WITH-AES-256-CBC-SHA256 TLS-DHE-DSS-WITH-AES-256-CBC-SHA256 TLS-DHE-RSA-WITH-AES-256-CBC-SHA TLS-DHE-DSS-WITH-AES-256-CBC-SHA TLS-DHE-RSA-WITH-CAMELLIA-256-CBC-SHA TLS-DHE-DSS-WITH-CAMELLIA-256-CBC-SHA
Re: [Openvpn-devel] Progress on Version negotiation
Hi,
On Wed, Apr 23, 2014 at 11:36:28AM -0400, Timothe Litt wrote:
> Just to confirm that the issue is 1.2, not the negotiation:
>
> I added an unconditional
> sslopt |= SSL_OP_NO_TLSv1_2;
> in tls_ctx_set_options.
>
> With this (and the context initialized to SSL_v23_*_method, so we
> negotiate), the tunnel comes up.
> Without it, the tunnel does not come up.
>
> So it is the use of 1.2 that is the issue, not how it is selected.
Thinking through this, while cycling home from $paidwork, I remembered
something I saw when debugging something similar ("if I enable TLS1.2,
things explode") last time.
From Perl's IO::Socket::SSL:
# older versions of F5 BIG-IP hang when getting SSL client hello >255 bytes
# http://support.f5.com/kb/en-us/solutions/public/13000/000/sol13037.html
# http://guest:gu...@rt.openssl.org/Ticket/Display.html?id=2771
# Debian works around this by disabling TLSv12 on the client side
# Chrome and IE11 use TLSv12 but use only a few ciphers, so that packet
# stays small enough
# The following list is taken from IE11, except that we don't do RC4-MD5,
# RC4-SHA is already bad enough. Also, we have a different sort order
# compared to IE11, because we put ciphers supporting forward secrecy on top
now - does that sound like it could be the problem? The initial handshake
packet "under some conditions" (like: the local OpenSSL build having
more available ciphers, depending on how it was built) being too big,
causing "surprises"?
(This question is more geared towards James, Arne and Steffan :-) )
Timothe, on your failing setup, could you try putting some variations of
"--tls-cipher" in your openvpn.conf? I'm not really sure I understand
the variants, but "openvpn --show-tls" suggests that some of these might
work
tls-cipher AES128-SHA
tls-cipher DHE-RSA-AES256-SHA
what does "openvpn --tls-cipher DEFAULT --show-tls" list on your systems
(or, phrased differently, if you have a system that does *not* fail on
TLS 1.2, does it show a shorter list)?
gert
--
USENET is *not* the non-clickable part of WWW!
//www.muc.de/~gert/
Gert Doering - Munich, Germany g...@greenie.muc.de
fax: +49-89-35655025g...@net.informatik.tu-muenchen.de
pgpntZEnGLiV3.pgp
Description: PGP signature
Re: [Openvpn-devel] Progress on Version negotiation
On 23-Apr-14 06:56, Steffan Karger wrote: Hi, On 04/23/2014 10:10 AM, Gert Doering wrote: On Tue, Apr 22, 2014 at 10:58:22PM -0400, Timothe Litt wrote: It does not appear to be the negotiation, rather it's TLS1.2. This is quite cool, thank you. (I'm not enough of a crypto geek to make real sense out of it, but it's quite useful to understand where it is failing, and I appreciate that you took the time to dig into it) Steffan, Arne, any ideas? This sounds very plausible, yes. I've seen situations in which an OpenSSL 2.3.3 client refuses to connect to a PolarSSL 1.2.10 server. I tried to reproduce that in a test setup last night, but did not manage make it break. So I'm still a bit in the dark on the real cause. For the 'fix the breaking asap', maybe we should add an --tls-version-max if that really resolves the problem. -Steffan Just to confirm that the issue is 1.2, not the negotiation: I added an unconditional sslopt |= SSL_OP_NO_TLSv1_2; in tls_ctx_set_options. With this (and the context initialized to SSL_v23_*_method, so we negotiate), the tunnel comes up. Without it, the tunnel does not come up. So it is the use of 1.2 that is the issue, not how it is selected. I generated a matching pair of traces of the failure (client and server) & posted a summary. Let me know if you would like the full traces. This is 100% reproducible here, so let me know if you need more instrumentation. (However, I can't build a windows client, so if that's necessary, you'll have to build it for me to run.) smime.p7s Description: S/MIME Cryptographic Signature
Re: [Openvpn-devel] Progress on Version negotiation
This is quite cool, thank you. You're welcome. I don't like unsolved mysteries, and since I have a solid reproducer, thought I should do what I can. Some more. I looked into building on Windows, but the doc says one needs commercial tools; I'm not going to buy them for this. However, I got verb 9 traces from the client and server side of a failure. They're a bit too large to post here, but I can provide the full traces off-list. I haven't backtracked to understanding what is being sent. Perhaps one of you has the magic decoder for the data in the messages... Here's a summary of what I see that seems interesting: Server: SSL state: Read client hello B Write server hello A Write certificate a Write key exchange A Write certificate request A Flush data This is the server sending the data that triggers the error, starting with the preceding read (that presumably triggers it): Wed Apr 23 09:26:25 2014 us=505743 GET INST BY REAL: 192.168.148.191:1194 [succeeded] Wed Apr 23 09:26:25 2014 us=507179 192.168.148.191:1194 UDP READ [50] from [AF_INET]192.168.148.191:1194: P_ACK_V1 kid=0 sid=87209bbf d7d12aeb tls_hmac=03a350ad c3c8bd68 06aacaa0 148a74fd b7731036 pid=[ #38 / time = (1398259583) Wed Apr 23 09:26:23 2014 ] [ 30 sid=adda013b a627c847 ] Wed Apr 23 09:26:25 2014 us=508665 192.168.148.191:1194 TLS: control channel, op=P_ACK_V1, IP=[AF_INET]192.168.148.191:1194 Wed Apr 23 09:26:25 2014 us=510704 192.168.148.191:1194 TLS: initial packet test, i=0 state=S_START, mysid=95747581 daf31aa7, rec-sid=87209bbf d7d12aeb, rec-ip=[AF_INET]192.168.148.191:1194, stored-sid=65fbeaf3 0672d359, stored-ip=[AF_INET]192.168.148.191:1194 Wed Apr 23 09:26:25 2014 us=511162 192.168.148.191:1194 TLS: initial packet test, i=1 state=S_START, mysid=adda013b a627c847, rec-sid=87209bbf d7d12aeb, rec-ip=[AF_INET]192.168.148.191:1194, stored-sid=87209bbf d7d12aeb, stored-ip=[AF_INET]192.168.148.191:1194 Wed Apr 23 09:26:25 2014 us=512181 192.168.148.191:1194 TLS: found match, session[1], sid=87209bbf d7d12aeb Wed Apr 23 09:26:25 2014 us=513617 192.168.148.191:1194 PID_TEST [0] [TLS_AUTH-0] [01222] 1398259583:37 1398259583:38 t=1398259585[0] r=[-2,64,15,0,1] sl=[27,37,64,272] Wed Apr 23 09:26:25 2014 us=514591 192.168.148.191:1194 TLS: received control channel packet s#=1 sid=87209bbf d7d12aeb Wed Apr 23 09:26:25 2014 us=515784 192.168.148.191:1194 ACK received for pid 30, deleting from send buffer Wed Apr 23 09:26:25 2014 us=516298 192.168.148.191:1194 TLS: tls_multi_process: i=0 state=S_START, mysid=95747581 daf31aa7, stored-sid=65fbeaf3 0672d359, stored-ip=[AF_INET]192.168.148.191:1194 Wed Apr 23 09:26:25 2014 us=517260 192.168.148.191:1194 TLS: tls_process: chg=0 ks=S_START lame=S_UNDEF to_link->len=0 wakeup=604800 Wed Apr 23 09:26:25 2014 us=518489 192.168.148.191:1194 ACK reliable_can_send active=1 current=0 : [35] 34 Wed Apr 23 09:26:25 2014 us=518920 192.168.148.191:1194 ACK reliable_send_timeout 1 [35] 34 Wed Apr 23 09:26:25 2014 us=519836 192.168.148.191:1194 TLS: tls_process: timeout set to 1 Wed Apr 23 09:26:25 2014 us=521129 192.168.148.191:1194 TLS: tls_multi_process: i=1 state=S_START, mysid=adda013b a627c847, stored-sid=87209bbf d7d12aeb, stored-ip=[AF_INET]192.168.148.191:1194 Wed Apr 23 09:26:25 2014 us=522055 192.168.148.191:1194 TLS: tls_process: chg=0 ks=S_START lame=S_UNDEF to_link->len=0 wakeup=1 Wed Apr 23 09:26:25 2014 us=523328 192.168.148.191:1194 ACK reliable_can_send active=3 current=0 : [34] 33 31 32 Wed Apr 23 09:26:25 2014 us=523743 192.168.148.191:1194 BIO read tls_read_ciphertext 95 bytes Wed Apr 23 09:26:25 2014 us=524649 192.168.148.191:1194 ACK mark active outgoing ID 34 Wed Apr 23 09:26:25 2014 us=525799 192.168.148.191:1194 Outgoing Ciphertext -> Reliable Wed Apr 23 09:26:25 2014 us=526116 192.168.148.191:1194 TLS: tls_process: chg=1 ks=S_START lame=S_UNDEF to_link->len=0 wakeup=1 Wed Apr 23 09:26:25 2014 us=527049 192.168.148.191:1194 ACK reliable_can_send active=4 current=1 : [35] 33 34 31 32 Wed Apr 23 09:26:25 2014 us=528217 192.168.148.191:1194 ACK reliable_send ID 34 (size=99 to=5) Wed Apr 23 09:26:25 2014 us=528542 192.168.148.191:1194 Reliable -> TCP/UDP Wed Apr 23 09:26:25 2014 us=529436 192.168.148.191:1194 ACK reliable_send_timeout 2 [35] 33 34 31 32 Wed Apr 23 09:26:25 2014 us=530612 192.168.148.191:1194 TLS: tls_process: timeout set to 1 Wed Apr 23 09:26:25 2014 us=531584 192.168.148.191:1194 TLS: tls_multi_process: i=2 state=S_UNDEF, mysid= , stored-sid= , stored-ip=[AF_UNSPEC] Wed Apr 23 09:26:25 2014 us=532955 PO_CTL rwflags=0x0002 ev=5 arg=0x0009377c Wed Apr 23 09:26:25 2014 us=533939 PO_CTL rwflags=0x ev=4 arg=0x000936f0 Wed Apr 23 09:26:25 2014 us=535162 PO_CTL rwflags=0x0001 ev=3 arg=0x000936f4 Wed Apr 23 09:26:25 2014 us=536073 I/O WAIT Tr|Tw|Sr|SW [1/14229] Wed Apr 23
Re: [Openvpn-devel] Progress on Version negotiation
Hi, On 04/23/2014 10:10 AM, Gert Doering wrote: > On Tue, Apr 22, 2014 at 10:58:22PM -0400, Timothe Litt wrote: >> It does not appear to be the negotiation, rather it's TLS1.2. > > This is quite cool, thank you. (I'm not enough of a crypto geek to > make real sense out of it, but it's quite useful to understand where > it is failing, and I appreciate that you took the time to dig into it) > > Steffan, Arne, any ideas? This sounds very plausible, yes. I've seen situations in which an OpenSSL 2.3.3 client refuses to connect to a PolarSSL 1.2.10 server. I tried to reproduce that in a test setup last night, but did not manage make it break. So I'm still a bit in the dark on the real cause. For the 'fix the breaking asap', maybe we should add an --tls-version-max if that really resolves the problem. -Steffan
Re: [Openvpn-devel] Progress on Version negotiation
Hi, On Tue, Apr 22, 2014 at 10:58:22PM -0400, Timothe Litt wrote: > It does not appear to be the negotiation, rather it's TLS1.2. This is quite cool, thank you. (I'm not enough of a crypto geek to make real sense out of it, but it's quite useful to understand where it is failing, and I appreciate that you took the time to dig into it) Steffan, Arne, any ideas? gert -- USENET is *not* the non-clickable part of WWW! //www.muc.de/~gert/ Gert Doering - Munich, Germany g...@greenie.muc.de fax: +49-89-35655025g...@net.informatik.tu-muenchen.de pgpE0xD5ca22z.pgp Description: PGP signature
[Openvpn-devel] Progress on Version negotiation
It does not appear to be the negotiation, rather it's TLS1.2.
I debugged the client hello in OpenSSL - a bit tricky due to the
timeouts, but I established that the server is picking TLS1.2.
I then switched the tls_ctx_{client,server}_new to use
TLSv1_2_{client,server}_method in the call to SSL_CTX_new.
The connection failed. So, we didn't negotiate, TLSv1.2 fails.
Next, I switched to TLSv1.1 (TLSv1_1_{client,server}method). The tunnel
comes up.
So, it appears that the issue has to do with TLSv1.2. (Also, it is 100%
reproducible here.)
Remember that the client log shows
Wed Apr 09 21:10:52 2014 us=748478 TLS_ERROR: BIO read
tls_read_plaintext error: error:04066083:rsa
routines:RSA_EAY_PRIVATE_ENCRYPT:invalid message length:
error:14099006:SSL routines:SSL3_SEND_CLIENT_VERIFY:EVP lib
Wed Apr 09 21:10:52 2014 us=748478 TLS Error: TLS object -> incoming
plaintext read error
Wed Apr 09 21:10:52 2014 us=748478 TLS Error: TLS handshake failed
I don't have development tools on my windows client, but perhaps someone
can build me an instrumented version. Here's what I think is happening:
The error is coming from ssl_openssl.c:key_state_read_plaintext, where
bio_read is failing.
bio_read (same file) fails if openssl/crypto/bio/bio_lib.c: BIO_read
fails with a hard error. There are several possibilities:
1) bio_read will reduce the read to 'maxlen', which I think is
TLS_CHANNEL_BUF_SIZE.
2) the data received is larger than the buffer.
3) the message length is corrupted.
According to the log, the client has received and verified the server
certificate. So the server hello has been processed successfully --
it's not the longer cipher suite in TLS1.2.
Interesting things to instrument:
Is bio_read reducing the read size? If so, from what to what?
What is the actual data received size?
What would happen if we built with larger TLS_CHANNEL_BUF_SIZE
(common.h defines it as 2048; suppose it was doubled...)
What is actually being read? Is it something else that we can
reduce on the server end?
Let me know if there's more I can do.
--
Timothe Litt
ACM Distinguished Engineer
--
This communication may not represent the ACM or my employer's views,
if any, on the matters discussed.
smime.p7s
Description: S/MIME Cryptographic Signature
