Re: [PATCH] Was: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
On 2018-08-03 12:46, Daniel Shahaf wrote: Folker Schamel wrote on Thu, 02 Aug 2018 15:34 +0200: On 2018-08-01 19:19, Daniel Shahaf wrote: Folker Schamel wrote on Wed, 01 Aug 2018 17:51 +0200: Hi Julian, Draft which may save you some time: First patch against trunk: [[[ * site/staging/faq.html: Add entry for "An error occurred during SSL communication" error. ]]] Second patch against trunk: [[[ * site/staging/docs/release-notes/1.10.html: Add entry for an OpenSSL upgrade causing "An error occurred during SSL communication" error. ]]] Thanks for the patch, Folker. Could we link to the OpenSSL upstream documentation --- their list of incompatible changes, or upgrade guide, or something along these lines? In our case, I'm not aware of any such documentation which would have helped us. Then why not ask the OpenSSL project to create such documentation? Again, this problem and its solution are entirely in the domain of OpenSSL. The Serf changes discussed are simply about having Serf marshal to its caller all the information it receives from its dependency. In our case, I don't see which additional OpenSSL documentation would have helped us. People never read any documentation, until they get an error. Then they google for that error. So getting a useful google hit would be great. This specific error "An error occurred during SSL communication" is from Subversion. So for all practical purposes it must be handled in the Subversion documentation, not in the OpenSSL documentation. I agree that there's room for some Subversion-specific documentation here, but I think it should focus on Subversion-specific parts (such as the ssl-authority-files knob, or failure modes that for whatever reason are more common with Subversion than with other SSL-using software) and refer to OpenSSL's docs for the lion's share of the content. For us the key issue was: What can I do if Subversion fails with "An error occurred during SSL communication"? The corresponding patches based on Philip's answer address this question. We lost a lot of time by wrongly suspecting problems between serf, mod_dav_svn and mod_ssl, for example https://code.google.com/archive/p/serf/issues/135. I appreciate that the system architecture involves several components (svn, serf, apr, openssl) and that it might not be obvious which component was responsible for which error message. That is knowledge I would love to see better transferred by our own documentation (the website or the svnbook). Documenting new releases of _our_ upstreams also makes sense, since we can read the upstream changelogs and summarize to our users what the impact of the upgrade on their Subversion deployments would be. What I disagree with is starting to document specific OpenSSL failure mode in our own documentation, when the failure modes are not Subversion- specific in any way. I hope you will spare me listing all the reasons why Project X shouldn't be documenting failure modes in Project Y [happy to do so but I think we are all professionals here and know these reasons]. I tried to write the patches in such way that they talk mainly about the Subversion specific aspects of the generic OpenSLL case, and mention the specific OpenSSL failure mode only as one particular sample. I will also do a commit review of your patch, in case it stays, but again, I think it should have been committed to OpenSSL's docs rather than ours. Hope you don't feel this as obstructions. I am merely trying to improve both our own docs and OpenSSL's, but part of improving our docs is to know what they should incorporate by value and what by reference. Very valid discussion, just different views ;-) Cheers, Folker Thanks for writing the docs patch! Daniel Cheers, Folker
Re: [PATCH] Was: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Folker Schamel wrote on Thu, 02 Aug 2018 15:34 +0200: > On 2018-08-01 19:19, Daniel Shahaf wrote: > > Folker Schamel wrote on Wed, 01 Aug 2018 17:51 +0200: > >> Hi Julian, > >> > >> Draft which may save you some time: > >> > >> First patch against trunk: > >> [[[ > >> * site/staging/faq.html: > >> Add entry for "An error occurred during SSL communication" error. > >> ]]] > >> > >> Second patch against trunk: > >> [[[ > >> * site/staging/docs/release-notes/1.10.html: > >> Add entry for an OpenSSL upgrade causing "An error occurred during > >> SSL communication" error. > >> ]]] > > Thanks for the patch, Folker. > > > > Could we link to the OpenSSL upstream documentation --- their list of > > incompatible changes, or upgrade guide, or something along these lines? > In our case, I'm not aware of any such documentation which would have > helped us. Then why not ask the OpenSSL project to create such documentation? Again, this problem and its solution are entirely in the domain of OpenSSL. The Serf changes discussed are simply about having Serf marshal to its caller all the information it receives from its dependency. > > I agree that there's room for some Subversion-specific documentation > > here, but I think it should focus on Subversion-specific parts (such as > > the ssl-authority-files knob, or failure modes that for whatever reason > > are more common with Subversion than with other SSL-using software) > > and refer to OpenSSL's docs for the lion's share of the content. > For us the key issue was: > What can I do if Subversion fails with "An error occurred during SSL > communication"? > The corresponding patches based on Philip's answer address this question. > We lost a lot of time by wrongly suspecting problems between serf, > mod_dav_svn and mod_ssl, for example > https://code.google.com/archive/p/serf/issues/135. > I appreciate that the system architecture involves several components (svn, serf, apr, openssl) and that it might not be obvious which component was responsible for which error message. That is knowledge I would love to see better transferred by our own documentation (the website or the svnbook). Documenting new releases of _our_ upstreams also makes sense, since we can read the upstream changelogs and summarize to our users what the impact of the upgrade on their Subversion deployments would be. What I disagree with is starting to document specific OpenSSL failure mode in our own documentation, when the failure modes are not Subversion- specific in any way. I hope you will spare me listing all the reasons why Project X shouldn't be documenting failure modes in Project Y [happy to do so but I think we are all professionals here and know these reasons]. I will also do a commit review of your patch, in case it stays, but again, I think it should have been committed to OpenSSL's docs rather than ours. Hope you don't feel this as obstructions. I am merely trying to improve both our own docs and OpenSSL's, but part of improving our docs is to know what they should incorporate by value and what by reference. Thanks for writing the docs patch! Daniel > Cheers, > Folker
Re: [PATCH] Was: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Folker Schamel wrote: > Daniel Shahaf wrote: >> Folker Schamel wrote: >>> * site/staging/faq.html: >>>Add entry for "An error occurred during SSL communication" error. >>> * site/staging/docs/release-notes/1.10.html: >>>Add entry for an OpenSSL upgrade causing "An error occurred during >>> SSL communication" error. >> >> Could we link to the OpenSSL upstream documentation --- their list of >> incompatible changes, or upgrade guide, or something along these lines? > > In our case, I'm not aware of any such documentation which would have helped > us. [...] I committed your patches in http://svn.apache.org/r1837321 (straight to the 'publish' tree, as I don't think there is anything there that could cause harm). We can adjust it further there if necessary. Thanks for contributing back the results from what you learnt the hard way. - Julian
Re: [PATCH] Was: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
On 2018-08-01 19:19, Daniel Shahaf wrote: Folker Schamel wrote on Wed, 01 Aug 2018 17:51 +0200: Hi Julian, Draft which may save you some time: First patch against trunk: [[[ * site/staging/faq.html: Add entry for "An error occurred during SSL communication" error. ]]] Second patch against trunk: [[[ * site/staging/docs/release-notes/1.10.html: Add entry for an OpenSSL upgrade causing "An error occurred during SSL communication" error. ]]] Thanks for the patch, Folker. Could we link to the OpenSSL upstream documentation --- their list of incompatible changes, or upgrade guide, or something along these lines? In our case, I'm not aware of any such documentation which would have helped us. I agree that there's room for some Subversion-specific documentation here, but I think it should focus on Subversion-specific parts (such as the ssl-authority-files knob, or failure modes that for whatever reason are more common with Subversion than with other SSL-using software) and refer to OpenSSL's docs for the lion's share of the content. For us the key issue was: What can I do if Subversion fails with "An error occurred during SSL communication"? The corresponding patches based on Philip's answer address this question. We lost a lot of time by wrongly suspecting problems between serf, mod_dav_svn and mod_ssl, for example https://code.google.com/archive/p/serf/issues/135. Cheers, Folker
Re: [PATCH] Was: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Folker Schamel wrote on Wed, 01 Aug 2018 17:51 +0200: > Hi Julian, > > Draft which may save you some time: > > First patch against trunk: > [[[ > * site/staging/faq.html: >Add entry for "An error occurred during SSL communication" error. > ]]] > > Second patch against trunk: > [[[ > * site/staging/docs/release-notes/1.10.html: >Add entry for an OpenSSL upgrade causing "An error occurred during > SSL communication" error. > ]]] Thanks for the patch, Folker. Could we link to the OpenSSL upstream documentation --- their list of incompatible changes, or upgrade guide, or something along these lines? I agree that there's room for some Subversion-specific documentation here, but I think it should focus on Subversion-specific parts (such as the ssl-authority-files knob, or failure modes that for whatever reason are more common with Subversion than with other SSL-using software) and refer to OpenSSL's docs for the lion's share of the content. HTH! Daniel
[PATCH] Was: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Hi Julian, Draft which may save you some time: First patch against trunk: [[[ * site/staging/faq.html: Add entry for "An error occurred during SSL communication" error. ]]] Second patch against trunk: [[[ * site/staging/docs/release-notes/1.10.html: Add entry for an OpenSSL upgrade causing "An error occurred during SSL communication" error. ]]] Cheers, Folker On 2018-08-01 15:03, Julian Foad wrote: Folker Schamel wrote: Hi Stefan, That's the catch here. Subversion does not ship with OpenSSL by itself. From Subversion's point of view this is a 3rd-party dependency. [...] It could be something worthwhile adding to the FAQ however, though then in a more general manner like: Troubleshooting Subversion SSL connection. Good point. The FAQ seems to be a good place. Nevertheless, in such situations we are probably not the only ones looking primarily into the Subversion release notes, not so much into the Debian documentation or Subversion FAQ, because the problem seemingly was caused - in simple terms - by the Subversion update. Also note that new releases of distributions of Subversion are usually strongly correlated with new Subversion releases. So I still suggest to also put a warning in the Subversion release notes, for example: "Your distribution may also upgrade OpenSSL along with the Subversion upgrade, which may cause trouble, see in the FAQ." At least us it would have spared a lot of time ;-) Even if you may insist that this logically the "wrong" place, sometimes a note in such a "wrong" place can be very helpful for users who are looking in that "wrong" place, ;-) I agree -- when something like this hits users in real life, we should add it to the release notes, either in total or as a pointer to a FAQ. And I want to say thank you for writing a helpful description for other users to diagnose the problem after being troubled by it yourself. If no-one else volunteers, I will try to do it in the next day or two (but I haven't much time). - Julian Index: staging/faq.html === --- staging/faq.html(revision 1837244) +++ staging/faq.html(working copy) @@ -1,4 +1,4 @@ -http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd;> http://www.w3.org/1999/xhtml;> @@ -256,6 +256,8 @@ characters do not seem to be working? Why does an HTTP(S) URL-to-URL copy or branch/tag operation take a long time? +When performing Subversion operations +over SSL, I get the error An error occurred during SSL communication Developer questions: @@ -4158,8 +4160,50 @@ + + +When performing Subversion operations +over SSL, I get the error An error occurred during SSL communication + + + +SSL communication errors can have various reasons. +You can use the openssl binary to debug the ssl connection. + +openssl s_client -connect example.com:443 -servername example.com + +If you use a client certificate, +then you need to convert Subversion's client certificate from pkcs12 to pem first: + +openssl pkcs12 -in path/to/svn/cert.p12 -out cert.pem + +Then you can use: + +openssl s_client -connect example.com:443 -servername example.com -cert cert.pem + +If you are using ssl-authority-files in .subversion/servers to verify +the server cert you can get s_client to do the same with the additional +parameter: + +openssl s_client ... -CAfile path/to/authority.pem + +The s_client output may indicate what problem is occurring. + + + +For example, if s_client reports + +error setting certificate +140258270184704:error:140AB18E:SSL routines:SSL_CTX_use_certificate:ca md too weak:../ssl/ssl_rsa.c:303: + +then creating new CA keys with sha256 instead of md5 should solve the problem. + + + + Developer questions: Index: staging/docs/release-notes/1.10.html === --- staging/docs/release-notes/1.10.html(revision 1837244) +++ staging/docs/release-notes/1.10.html(working copy) @@ -1,4 +1,4 @@ -http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd;> http://www.w3.org/1999/xhtml;> @@ -291,6 +291,43 @@ + +New CA keys may be required + + + + +Some binary distributions of this new Subversion version +may link to a newer OpenSSL version than previous distributions. +This may lead to different behavior. + + + +Especially, some distributions may link this Subversion release to OpenSSL 1.1 instead of OpenSSL 1.0. +OpenSSL 1.1 does not allow md5 hashes for CA keys anymore. +When using client certificates signed by such a CA, +the new Subversion client may fail with An error occurred during SSL communication. +You can analyze the underlying cause by first converting the client certificate from p12 to pem by + +openssl pkcs12 -in path/to/svn/cert.p12 -out cert.pem + +and then testing the SSL connection by + +openssl s_client -connect example.com:443 -servername example.com -cert cert.pem + +If this test connection fails with ca md
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Folker Schamel wrote: > Hi Stefan, >> That's the catch here. Subversion does not ship with OpenSSL by >> itself. From Subversion's point of view this is a 3rd-party >> dependency. [...] It could be something worthwhile adding to the FAQ >> however, though then in a more general manner like: Troubleshooting >> Subversion SSL connection.> Good point. The FAQ seems to be a good place. > > Nevertheless, in such situations we are probably not the only ones > looking primarily into the Subversion release notes, not so much into > the Debian documentation or Subversion FAQ, because the problem > seemingly was caused - in simple terms - by the Subversion update. > Also note that new releases of distributions of Subversion are > usually strongly correlated with new Subversion releases. > > So I still suggest to also put a warning in the Subversion release > notes, for example: "Your distribution may also upgrade OpenSSL along > with the Subversion upgrade, which may cause trouble, see in the > FAQ." At least us it would have spared a lot of time ;-) > > Even if you may insist that this logically the "wrong" place, > sometimes a note in such a "wrong" place can be very helpful for > users who are looking in that "wrong" place, ;-) I agree -- when something like this hits users in real life, we should add it to the release notes, either in total or as a pointer to a FAQ. And I want to say thank you for writing a helpful description for other users to diagnose the problem after being troubled by it yourself. If no-one else volunteers, I will try to do it in the next day or two (but I haven't much time). - Julian
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Hi Stefan, That's the catch here. Subversion does not ship with OpenSSL by itself. From Subversion's point of view this is a 3rd-party dependency. You can easily build Subversion 1.9.x/1.10.x with OpenSSL 1.0.x. Whether or not you run into this issue therefore is outside the scope of Subversion IMO. It's something the distribution of Subversion (in your case the Debian Subversion distribution) should document. Note that in principle you could very well run into the same situation with Subversion 1.8 or even 1.7, if you build one version with OpenSSL <= 1.0 and the other with OpenSSL >= 1.1 (or set certain OpenSSL configs which also would flag md5 digests as being too weak with older OpenSSL versions). It could be something worthwhile adding to the FAQ however, though then in a more general manner like: Troubleshooting Subversion SSL connection. Good point. The FAQ seems to be a good place. Nevertheless, in such situations we are probably not the only ones looking primarily into the Subversion release notes, not so much into the Debian documentation or Subversion FAQ, because the problem seemingly was caused - in simple terms - by the Subversion update. Also note that new releases of distributions of Subversion are usually strongly correlated with new Subversion releases. So I still suggest to also put a warning in the Subversion release notes, for example: "Your distribution may also upgrade OpenSSL along with the Subversion upgrade, which may cause trouble, see in the FAQ." At least us it would have spared a lot of time ;-) Even if you may insist that this logically the "wrong" place, sometimes a note in such a "wrong" place can be very helpful for users who are looking in that "wrong" place, ;-) Cheers, Folker
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
On 8/1/2018 9:25 AM, Folker Schamel wrote: On 2018-07-31 21:09, Philip Martin wrote: Daniel Shahaf writes: Subversion uses Serf, which uses OpenSSL, which talks to an SSL implementation on the server. The root cause of the error is known to the SSL implementation on the server (that's why you see it in the error log). It's not obvious that OpenSSL on the client side even knows what the root cause is. In this case the client knows exactly what is wrong, it's the one closing the connection because: 140258270184704:error:140AB18E:SSL routines:SSL_CTX_use_certificate:ca md too weak:../ssl/ssl_rsa.c:303: Could we get our client to show that error? We would need a new serf API to marshal the error message back to Subversion. This sounds like the best solution to me. Until then, some short note in the Subversion release notes would have saved us a lot of time, for example: Section title: Potential need for creating new CA keys and new client certificates Section text: This Subversion release upgrades OpenSSL from 1.0 to 1.1, which doesn't allow md5 hashes for CA keys anymore. That's the catch here. Subversion does not ship with OpenSSL by itself. From Subversion's point of view this is a 3rd-party dependency. You can easily build Subversion 1.9.x/1.10.x with OpenSSL 1.0.x. Whether or not you run into this issue therefore is outside the scope of Subversion IMO. It's something the distribution of Subversion (in your case the Debian Subversion distribution) should document. Note that in principle you could very well run into the same situation with Subversion 1.8 or even 1.7, if you build one version with OpenSSL <= 1.0 and the other with OpenSSL >= 1.1 (or set certain OpenSSL configs which also would flag md5 digests as being too weak with older OpenSSL versions). When using client certificates signed by such a CA, the new Subversion client now fails with "An error occurred during SSL communication". You can analyze the underlying cause by converting the client certificate from p12 to pem by"openssl pkcs12 -in path/to/svn/cert.p12 -out cert.pem" and then test the SSL connection by "openssl s_client -connect example.com:443 -servername example.com -cert cert.pem". If this test connection fails with "ca md too weak", then creating new CA keys using sha256 instead of md5 and corresponding new client certificates should solve the problem. See also https://lists.apache.org/thread.html/66b9bfa0a83693c3ccef34b29056c7e73a0d21cd4b70cd7f7519fa57@%3Cdev.subversion.apache.org%3E. Cheers, Folker It could be something worthwhile adding to the FAQ however, though then in a more general manner like: Troubleshooting Subversion SSL connection. -- Regards, Stefan Hett
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
On 2018-07-31 21:09, Philip Martin wrote: Daniel Shahaf writes: Subversion uses Serf, which uses OpenSSL, which talks to an SSL implementation on the server. The root cause of the error is known to the SSL implementation on the server (that's why you see it in the error log). It's not obvious that OpenSSL on the client side even knows what the root cause is. In this case the client knows exactly what is wrong, it's the one closing the connection because: 140258270184704:error:140AB18E:SSL routines:SSL_CTX_use_certificate:ca md too weak:../ssl/ssl_rsa.c:303: Could we get our client to show that error? We would need a new serf API to marshal the error message back to Subversion. This sounds like the best solution to me. Until then, some short note in the Subversion release notes would have saved us a lot of time, for example: Section title: Potential need for creating new CA keys and new client certificates Section text: This Subversion release upgrades OpenSSL from 1.0 to 1.1, which doesn't allow md5 hashes for CA keys anymore. When using client certificates signed by such a CA, the new Subversion client now fails with "An error occurred during SSL communication". You can analyze the underlying cause by converting the client certificate from p12 to pem by"openssl pkcs12 -in path/to/svn/cert.p12 -out cert.pem" and then test the SSL connection by "openssl s_client -connect example.com:443 -servername example.com -cert cert.pem". If this test connection fails with "ca md too weak", then creating new CA keys using sha256 instead of md5 and corresponding new client certificates should solve the problem. See also https://lists.apache.org/thread.html/66b9bfa0a83693c3ccef34b29056c7e73a0d21cd4b70cd7f7519fa57@%3Cdev.subversion.apache.org%3E. Cheers, Folker
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Daniel Shahaf writes: > Subversion uses Serf, which uses OpenSSL, which talks to an SSL implementation > on the server. The root cause of the error is known to the SSL implementation > on the server (that's why you see it in the error log). It's not obvious that > OpenSSL on the client side even knows what the root cause is. In this case the client knows exactly what is wrong, it's the one closing the connection because: 140258270184704:error:140AB18E:SSL routines:SSL_CTX_use_certificate:ca md too weak:../ssl/ssl_rsa.c:303: Could we get our client to show that error? We would need a new serf API to marshal the error message back to Subversion. -- Philip
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Folker Schamel wrote on Tue, Jul 31, 2018 at 17:42:10 +0200: > On 2018-07-31 17:04, Philip Martin wrote: > > Folker Schamel writes: > > > For the broken setup, the client reports: > > > svn: E120171: Error running context: An error occurred during SSL > > > communication > > > And the server Apache log reports: > > > ssl_engine_io.c(1308): (70014)End of file found: [client x:x] > > > AH02007: SSL handshake interrupted by system [Hint: Stop button > > > pressed in browser?!] > > Maybe a hint in the svn release notes could be useful, since the svn error > messages are not very useful. Subversion uses Serf, which uses OpenSSL, which talks to an SSL implementation on the server. The root cause of the error is known to the SSL implementation on the server (that's why you see it in the error log). It's not obvious that OpenSSL on the client side even knows what the root cause is. It could be that only the server-side SSL implementation knows the root cause; it could be that the client-side SSL implementation also knows the root cause, but the ball of relaying that information up the stack (openssl->serf->libsvn->stderr) was dropped. The error code in question (E120171, SERF_ERROR_SSL_COMM_FAILED) does appear to be somewhat of a catchall, i.e., a code to which several openssl errors are mapped; but nevertheless, I wouldn't be surprised if the openssl client-side error message were less detailed than the server-side one. For what it's worth, the only part of the quoted error message that Subversion controls is the text "Error running context". The remainder, both the number prefixed andthe error suffixed, is generated by the serf library, based on an error returned by the openssl library. That said, I do agree that "Error running context" isn't the best phrasing. "Context", here, is the name of an internal API. Something like "HTTP request failed" would be better, wouldn't it? > Thanks a lot! > And sorry for bothering on the dev list. I should have posted to user. > Also thanks for your other SNI and DEFLATE tips! Cheers, Daniel
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Hi Philip, this solved it! Using "openssl s_client" as you described it reported: error setting certificate 140258270184704:error:140AB18E:SSL routines:SSL_CTX_use_certificate:ca md too weak:../ssl/ssl_rsa.c:303: So we created new certificates with sha256 (default in openssl 1.1) instead of md5. This solved the problem. Maybe a hint in the svn release notes could be useful, since the svn error messages are not very useful. Thanks a lot! And sorry for bothering on the dev list. I should have posted to user. Also thanks for your other SNI and DEFLATE tips! Cheers, Folker On 2018-07-31 17:04, Philip Martin wrote: Folker Schamel writes: After upgrading, Subversion SSL connections with "SSLVerifyClient require" seem to be broken. Broken: SVN Client 1.9.5, Serf 1.3.9-3, Server "SSLVerifyClient require" Works: SVN Client 1.9.5, Serf 1.3.9-3, Server "SSLVerifyClient off" Works: SVN Client 1.9.5, Serf 1.3.8-1, Server "SSLVerifyClient require" I can use client certificates on my Debian machine using 1.3.9-3 with "SSLVerifyClient require" so it does work in some configurations. I think that one of the changes between 1.3.8-1 and 1.3.9-3 on Debian is that openssl was upgraded from 1.0 to 1.1. That's a major upgrade and some sort of openssl incompatibility may be the problem. You could use the openssl binary to debug the ssl connection. First you need to convert Subversion's client certificate from pkcs12 to pem: openssl pkcs12 -in path/to/svn/cert.p12 -out cert.pem then you can use: openssl s_client -connect example.com:443 -servername example.com -cert cert.pem If you are using ssl-authority-files in .subversion/servers to verify the server cert you can get s_client to do the same with the additional parameter: openssl s_client ... -CAfile path/to/authority.pem The s_client output may indicate what problem is occurring. For the broken setup, the client reports: svn: E120171: Error running context: An error occurred during SSL communication And the server Apache log reports: ssl_engine_io.c(1308): (70014)End of file found: [client x:x] AH02007: SSL handshake interrupted by system [Hint: Stop button pressed in browser?!] Using the latest TortoiseSVN client reports the same problem, presumably the same cause. TortoiseSVN is probably built with openssl 1.1 as well. [Tue Jul 31 15:30:43.886183 2018] [ssl:debug] [pid x:tid x] ssl_engine_kernel.c(2122): [client x:x] AH02044: No matching SSL virtual host for servername x found (using default/first virtual host) That looks like SNI isn't working but I don't know if that is relevant for your problem. Some sort of vhost/servername problem in the apache config? SetOutputFilter DEFLATE SetInputFilter DEFLATE Header append Vary User-Agent env=!dont-vary DEFLATE combined with mod_dav_svn had problems in the past but I think they are all fixed. Note that when mod_dav_svn and Subversion clients communicate they do so using compressed deltas, so adding DEFLATE doesn't result in very much additional compression. The combination has a more significant compression effect if users are using non-Subversion tools: web browsers, curl, etc. This is probably not relevant to your problem.
Re: Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Folker Schamel writes: > After upgrading, Subversion SSL connections with "SSLVerifyClient > require" seem to be broken. > > Broken: SVN Client 1.9.5, Serf 1.3.9-3, Server "SSLVerifyClient require" > Works: SVN Client 1.9.5, Serf 1.3.9-3, Server "SSLVerifyClient off" > Works: SVN Client 1.9.5, Serf 1.3.8-1, Server "SSLVerifyClient require" I can use client certificates on my Debian machine using 1.3.9-3 with "SSLVerifyClient require" so it does work in some configurations. I think that one of the changes between 1.3.8-1 and 1.3.9-3 on Debian is that openssl was upgraded from 1.0 to 1.1. That's a major upgrade and some sort of openssl incompatibility may be the problem. You could use the openssl binary to debug the ssl connection. First you need to convert Subversion's client certificate from pkcs12 to pem: openssl pkcs12 -in path/to/svn/cert.p12 -out cert.pem then you can use: openssl s_client -connect example.com:443 -servername example.com -cert cert.pem If you are using ssl-authority-files in .subversion/servers to verify the server cert you can get s_client to do the same with the additional parameter: openssl s_client ... -CAfile path/to/authority.pem The s_client output may indicate what problem is occurring. > For the broken setup, the client reports: > svn: E120171: Error running context: An error occurred during SSL > communication > And the server Apache log reports: > ssl_engine_io.c(1308): (70014)End of file found: [client x:x] > AH02007: SSL handshake interrupted by system [Hint: Stop button > pressed in browser?!] > > Using the latest TortoiseSVN client reports the same problem, > presumably the same cause. TortoiseSVN is probably built with openssl 1.1 as well. > [Tue Jul 31 15:30:43.886183 2018] [ssl:debug] [pid x:tid x] > ssl_engine_kernel.c(2122): [client x:x] AH02044: No matching > SSL virtual host for servername x found (using default/first > virtual host) That looks like SNI isn't working but I don't know if that is relevant for your problem. Some sort of vhost/servername problem in the apache config? > > SetOutputFilter DEFLATE > SetInputFilter DEFLATE > Header append Vary User-Agent env=!dont-vary > DEFLATE combined with mod_dav_svn had problems in the past but I think they are all fixed. Note that when mod_dav_svn and Subversion clients communicate they do so using compressed deltas, so adding DEFLATE doesn't result in very much additional compression. The combination has a more significant compression effect if users are using non-Subversion tools: web browsers, curl, etc. This is probably not relevant to your problem. -- Philip
Bug report: Regression SVN Client, SSL, Serf 1.3.9-3, SSLVerifyClient require
Hello everyone, After upgrading, Subversion SSL connections with "SSLVerifyClient require" seem to be broken. Broken: SVN Client 1.9.5, Serf 1.3.9-3, Server "SSLVerifyClient require" Works: SVN Client 1.9.5, Serf 1.3.9-3, Server "SSLVerifyClient off" Works: SVN Client 1.9.5, Serf 1.3.8-1, Server "SSLVerifyClient require" For the broken setup, the client reports: svn: E120171: Error running context: An error occurred during SSL communication And the server Apache log reports: ssl_engine_io.c(1308): (70014)End of file found: [client x:x] AH02007: SSL handshake interrupted by system [Hint: Stop button pressed in browser?!] Using the latest TortoiseSVN client reports the same problem, presumably the same cause. Additional details below. Can I help with additional information? Btw, thanks a lot to all Subversion developers and contributors for the awesome work!!! Cheers, Folker * Client-side recipt (latest Debian stretch): root@x:/# apt-get install libserf-1-1=1.3.8-1 . root@x:/# svn --version svn, version 1.9.5 (r1770682) compiled Jun 30 2018, 13:44:22 on x86_64-pc-linux-gnu Copyright (C) 2016 The Apache Software Foundation. This software consists of contributions made by many people; see the NOTICE file for more information. Subversion is open source software, see http://subversion.apache.org/ The following repository access (RA) modules are available: * ra_svn : Module for accessing a repository using the svn network protocol. - with Cyrus SASL authentication - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - using serf 1.3.8 (compiled with 1.3.9) - handles 'http' scheme - handles 'https' scheme The following authentication credential caches are available: * Plaintext cache in /root/.subversion * Gnome Keyring * GPG-Agent * KWallet (KDE) root@x:/# svn update Updating '.': At revision 828. root@x:/# apt-get install libserf-1-1=1.3.9-3 . root@x:/# svn --version svn, version 1.9.5 (r1770682) compiled Jun 30 2018, 13:44:22 on x86_64-pc-linux-gnu Copyright (C) 2016 The Apache Software Foundation. This software consists of contributions made by many people; see the NOTICE file for more information. Subversion is open source software, see http://subversion.apache.org/ The following repository access (RA) modules are available: * ra_svn : Module for accessing a repository using the svn network protocol. - with Cyrus SASL authentication - handles 'svn' scheme * ra_local : Module for accessing a repository on local disk. - handles 'file' scheme * ra_serf : Module for accessing a repository via WebDAV protocol using serf. - using serf 1.3.9 (compiled with 1.3.9) - handles 'http' scheme - handles 'https' scheme The following authentication credential caches are available: * Plaintext cache in /root/.subversion * Gnome Keyring * GPG-Agent * KWallet (KDE) root@x:/# svn update Updating '.': svn: E170013: Unable to connect to a repository at URL 'https://x/x/x' svn: E120171: Error running context: An error occurred during SSL communication root@x:/# * Client-side recipt continuation after SSLVerifyClient require -> off root@x:/# svn update Updating '.': At revision 828. root@x:/# * Server-side ssl-error.log: ... [Tue Jul 31 15:30:43.885515 2018] [ssl:info] [pid x:tid x] [client x:x] AH01964: Connection to child 68 established (server localhost:443) [Tue Jul 31 15:30:43.885795 2018] [ssl:trace2] [pid x:tid x] ssl_engine_rand.c(126): Seeding PRNG with 656 bytes of entropy [Tue Jul 31 15:30:43.885983 2018] [ssl:trace3] [pid x:tid x] ssl_engine_kernel.c(1989): [client x:x] OpenSSL: Handshake: start [Tue Jul 31 15:30:43.886064 2018] [ssl:trace3] [pid x:tid x] ssl_engine_kernel.c(1998): [client x:x] OpenSSL: Loop: before/accept initialization [Tue Jul 31 15:30:43.886114 2018] [ssl:trace4] [pid x:tid x] ssl_engine_io.c(2135): [client x:x] OpenSSL: read 5/5 bytes from BIO#7fcef0001580 [mem: 7fcef0006dc3] (BIO dump follows) [Tue Jul 31 15:30:43.886134 2018] [ssl:trace4] [pid x:tid x] ssl_engine_io.c(2135): [client x:x] OpenSSL: read 191/191 bytes from BIO#7fcef0001580 [mem: 7fcef0006dc8] (BIO dump follows) [Tue Jul 31 15:30:43.886183 2018] [ssl:debug] [pid x:tid x] ssl_engine_kernel.c(2122): [client x:x] AH02044: No matching SSL virtual host for servername x found (using default/first virtual host) [Tue Jul 31 15:30:43.886258 2018] [ssl:trace3] [pid x:tid x] ssl_engine_kernel.c(1998): [client x:x] OpenSSL: Loop: unknown state [Tue Jul 31 15:30:43.886294 2018] [ssl:trace3] [pid x:tid x] ssl_engine_kernel.c(1998): [client x:x] OpenSSL: Loop: unknown state [Tue Jul 31 15:30:43.886419 2018]
