Re: [lftp] Make certificate verification great again
Just an update to close the thread: the hosting company has changed their server setup to include the intermediate CA certificates in addition to the server certificates. I can now confirm that the certificate chain is indeed visible when connecting with lftp (in debug mode), and certificate verification succeeds. Thanks again for your help with this issue! Naël On Wed, Mar 22, 2017 at 1:03 AM, Nathanaël Naeri wrote: > I've contacted the hosting company. Thank you so much for > troubleshooting this issue, and helping me understand certificate > verification better! > > Naël > > On Tue, Mar 21, 2017 at 1:37 PM, Alexander V. Lukyanov wrote: >> On Mon, Mar 20, 2017 at 11:49:46PM +0100, Daniel Fazekas wrote: >>> On Mar 20, 2017, at 14:55, Nathanaël Naeri >>> wrote: >>> > Is that an issue that this hosting company could do something about? I >>> > can ask their sysadmins for help. >>> >>> It's a common setup mistake to make for server admins that they only add >>> the server certificate to their configuration. Normally you also need to >>> add one or more CA intermediate certs so that clients, which only normally >>> carry and trust a bundle of root certs, could successfully verify the whole >>> chain. >>> It's generally as simple as concatenating the intermediate cert(s) after >>> your server certificate, for the server admin. >>> >>> This could be the issue causing your problems, and something only they can >>> fix, short of you manually adding that missing intermediate cert on all >>> your client systems, working around their mistake. >> >> This seems to be the issue. The certificate chain of the ftp server is not >> a chain, but rather a single link. It's necessary either change the server's >> certificate to the full chain to the root CA, or add the "next link" to the >> local CA storage. ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
Re: [lftp] Make certificate verification great again
I've contacted the hosting company. Thank you so much for troubleshooting this issue, and helping me understand certificate verification better! Naël On Tue, Mar 21, 2017 at 1:37 PM, Alexander V. Lukyanov wrote: > On Mon, Mar 20, 2017 at 11:49:46PM +0100, Daniel Fazekas wrote: >> On Mar 20, 2017, at 14:55, Nathanaël Naeri wrote: >> > Is that an issue that this hosting company could do something about? I >> > can ask their sysadmins for help. >> >> It's a common setup mistake to make for server admins that they only add the >> server certificate to their configuration. Normally you also need to add one >> or more CA intermediate certs so that clients, which only normally carry and >> trust a bundle of root certs, could successfully verify the whole chain. >> It's generally as simple as concatenating the intermediate cert(s) after >> your server certificate, for the server admin. >> >> This could be the issue causing your problems, and something only they can >> fix, short of you manually adding that missing intermediate cert on all >> your client systems, working around their mistake. > > This seems to be the issue. The certificate chain of the ftp server is not > a chain, but rather a single link. It's necessary either change the server's > certificate to the full chain to the root CA, or add the "next link" to the > local CA storage. > > -- >Alexander. ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
Re: [lftp] Make certificate verification great again
@Alexander: Sure, SERVER=pool222, and other numbers would probably work I suppose. I originally assumed it was irrelevant since CN=*.seedbox.fr but apparently it's not without importance. The error happens when I run the first "ls" command (lftp 4.7.7 w/ GnuTLS 3.5.10): $ ./lftp lftp :~> debug lftp :~> set ssl:ca-file /etc/ssl/certs/ca-certificates.crt lftp :~> open -p 21 -u USER,PASS pool222.seedbox.fr Resolving host address... 1 address found: 51.254.45.220 lftp u...@pool222.seedbox.fr:~> ls Connecting to pool222.seedbox.fr (51.254.45.220) port 21 <--- 220-- Welcome to Pure-FTPd [privsep] [TLS] -- <--- [other 220 info] ---> FEAT <--- [feat reply] ---> AUTH TLS <--- 234 AUTH TLS OK. ---> OPTS UTF8 ON Certificate: C=FR,postalCode=77310,ST=Seine-et-Marne,L=PRINGY,street=IMPASSE DU BREAU,O=SDBX FRANCE,OU=0002 529997199,CN=*.seedbox.fr Issued by: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA Limited,CN=COMODO RSA Organization Validation Secure Server CA ERROR: Certificate verification: Not trusted (FA:D3:C7:E9:E8:42:54:BD:4D:AC:15:48:5B:17:65:E4:D0:F9:CF:63) Certificate verification: Not trusted (FA:D3:C7:E9:E8:42:54:BD:4D:AC:15:48:5B:17:65:E4:D0:F9:CF:63) Closing control socket ls: Fatal error: Certificate verification: Not trusted (FA:D3:C7:E9:E8:42:54:BD:4D:AC:15:48:5B:17:65:E4:D0:F9:CF:63) @Daniel: I thought clients followed certificate chains themselves, by downloading the intermediate CA certificates from the URI in the "Authority Information Access" field? If that's not what happens, I understand having only the server certificate on the server is not enough indeed. In this case, the intermediate CA certificates would be missing from the FTP host but present on the HTTP host? This would explain why verification fails for the first one but succeeds for the second one. I'm going to contact the hosting company's sysadmins, thanks. On Mon, Mar 20, 2017 at 11:49 PM, Daniel Fazekas wrote: > On Mar 20, 2017, at 14:55, Nathanaël Naeri wrote: >> Is that an issue that this hosting company could do something about? I >> can ask their sysadmins for help. > > It's a common setup mistake to make for server admins that they only add the > server certificate to their configuration. Normally you also need to add one > or more CA intermediate certs so that clients, which only normally carry and > trust a bundle of root certs, could successfully verify the whole chain. > It's generally as simple as concatenating the intermediate cert(s) after your > server certificate, for the server admin. > > This could be the issue causing your problems, and something only they can > fix, short of you manually adding that missing intermediate cert on all your > client systems, working around their mistake. > ___ > lftp mailing list > lftp@uniyar.ac.ru > http://univ.uniyar.ac.ru/mailman/listinfo/lftp ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
Re: [lftp] Make certificate verification great again
It appears that "open -d https://www.seedbox.fr"; works indeed ("Trusted", certificate chain printed out as in your previous message), but "open -d -p 21 -u USER,PASS SERVER.seedbox.fr" doesn't ("Certificate verification: Not trusted", same output as reported in my first message). Using lftp 4.7.7 with GnuTLS 3.5.10 and my CA bundle. I also checked manually that both your CA bundle and mine: * don't include COMODORSAOrganizationValidationSecureServerCA.pem * include COMODORSACertificationAuthority.pem * don't include COMODORSAAddTrustCA.pem So they're not different in this respect. It's not clear to me, which one is the root CA certificate. Only the AddTrust one is self-signed, but the certificate chain printed by lftp with GnuTLS stops at the second one, while that using OpenSSL includes the last one. The server certificates coming from the HTTP and FTP servers are the same: I downloaded one from https://www.seedbox.fr using Firefox 52 > Page Info and the other from SERVER.seedbox.fr using "openssl s_client -connect SERVER.seedbox.fr:21 -starttls ftp": they're the same except for the end-of-line characters, and apply both to *.seedbox.fr. Is that an issue that this hosting company could do something about? I can ask their sysadmins for help. On Mon, Mar 20, 2017 at 12:52 PM, Alexander V. Lukyanov wrote: > On Sat, Mar 18, 2017 at 09:13:27PM +0100, Nathanaël Naeri wrote: >> Thank you for your answer. I have updated my version of GnuTLS to >> 3.5.10 and compiled lftp 4.7.7 against it. The resulting "./lftp >> --version" shows "Libraries used: Readline 6.3, Expat 2.1.0, GnuTLS >> 3.5.10, zlib 1.2.8". Yet the error I reported in my first message >> remains: "Certificate verification: Not trusted". >> >> What commands did you use in your last message to verify certificate >> chains? The output I get with openssl verify and certtool is quite >> different. > > I did "open -d https://www.seedbox.fr";. My CA bundle is attached. > > -- >Alexander. ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
Re: [lftp] Make certificate verification great again
Thank you for your answer. I have updated my version of GnuTLS to 3.5.10 and compiled lftp 4.7.7 against it. The resulting "./lftp --version" shows "Libraries used: Readline 6.3, Expat 2.1.0, GnuTLS 3.5.10, zlib 1.2.8". Yet the error I reported in my first message remains: "Certificate verification: Not trusted". What commands did you use in your last message to verify certificate chains? The output I get with openssl verify and certtool is quite different. On Tue, Mar 14, 2017 at 7:13 AM, Alexander V. Lukyanov wrote: > I can't reproduce the problem. Here is what I get with OpenSSL 1.0.2k: > > Certificate depth: 3; subject: /C=SE/O=AddTrust AB/OU=AddTrust External TTP > Network/CN=AddTrust External CA Root; issuer: /C=SE/O=AddTrust AB/OU=AddTrust > External TTP Network/CN=AddTrust External CA Root > Certificate depth: 2; subject: /C=GB/ST=Greater Manchester/L=Salford/O=COMODO > CA Limited/CN=COMODO RSA Certification Authority; issuer: /C=SE/O=AddTrust > AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root > Certificate depth: 1; subject: /C=GB/ST=Greater Manchester/L=Salford/O=COMODO > CA Limited/CN=COMODO RSA Organization Validation Secure Server CA; issuer: > /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA > Certification Authority > Certificate depth: 0; subject: > /C=FR/postalCode=77310/ST=Seine-et-Marne/L=PRINGY/street=IMPASSE DU > BREAU/O=SDBX FRANCE/OU=0002 529997199/CN=*.seedbox.fr; issuer: > /C=GB/ST=Greater Manchester/L=Salford/O=COMODO CA Limited/CN=COMODO RSA > Organization Validation Secure Server CA > Certificate verification: subjectAltName: ‘www.seedbox.fr’ matched > > And with GnuTLS 3.5.10: > > Certificate: C=FR,postalCode=77310,ST=Seine-et-Marne,L=PRINGY,street=IMPASSE > DU BREAU,O=SDBX FRANCE,OU=0002 529997199,CN=*.seedbox.fr > Issued by:C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA > Limited,CN=COMODO RSA Organization Validation Secure Server CA > Checking against: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA > Limited,CN=COMODO RSA Organization Validation Secure Server CA > Trusted > Certificate: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA > Limited,CN=COMODO RSA Organization Validation Secure Server CA > Issued by:C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA > Limited,CN=COMODO RSA Certification Authority > Checking against: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA > Limited,CN=COMODO RSA Certification Authority > Trusted > Certificate: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA > Limited,CN=COMODO RSA Certification Authority > Issued by: C=SE,O=AddTrust AB,OU=AddTrust External TTP Network,CN=AddTrust > External CA Root > Trusted > > -- >Alexander. ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
Re: [lftp] Make certificate verification great again
PS: The certificate chain that I can follow manually using OpenSSL is different than that shown by my browser (Firefox > Page Info). I don't know why that is. It goes as follow: AddTrust External CA Root COMODO RSA AddTrust CA COMODO RSA Organization Validation Secure Server CA *.seedbox.fr instead of: COMODO RSA Certification Authority COMODO RSA Organization Validation Secure Server CA *.seedbox.fr But in any case, I have certificates for both roots in my /etc/ssl/certs/ca-certificates.crt bundle file, so I don't understand why verification fails. Thanks in advance for your help Naël On Sun, Mar 12, 2017 at 4:33 PM, Nathanaël Naeri wrote: > Thanks for your answer. I have checked that Comodo's root CA > certificate is present in the certificate bundle file, however > Comodo's intermediate CA certificate (that signed the server's > certificate) isn't, as is normal if I understand correctly. > > The certificate hierarchy is as follow (as shown by Firefox > Page > Info when connecting to https://www.seedbox.fr): > > COMODO RSA Certification Authority > COMODO RSA Organization Validation Secure Server CA > *.seedbox.fr > > Certificate validation fails as described whether I point to the root > CA certificate bundle file or to the relevant root certificate only: > > set ssl:ca-file /etc/ssl/certs/ca-certificates.crt > set ssl:ca-file /etc/ssl/certs/COMODO_RSA_Certification_Authority.pem > set ssl:ca-file > /path/to/manually/downloaded/COMODORSACertificationAuthority.crt > > It succeeds if I point to the intermediate CA certificate, but I > shouldn't have to do that, should I? > > set ssl:ca-file > /path/to/manually/downloaded/COMODORSAOrganizationValidationSecureServerCA.crt > > I checked using lftp 4.6.3a (Ubuntu 16.04) and 4.7.7 (latest stable) BTW. > > On Sun, Mar 12, 2017 at 8:29 AM, Alexander Lukyanov wrote: >> Your understanding of CA is correct. The Comodo certificate should be >> present in the CA bundle for the verification to succeed. >> >> >> вс, 12 марта 2017, 5:16 Nathanaël Naeri : >>> >>> I'm trying to connect to a FTP server that supports explicit FTPS >>> using TLS, but I can't get certificate verification working. Most of >>> the online help I find advises disabling certificate verification with >>> "ssl:verify-certificate no", and I assume this is not good advice. >>> >>> My current understanding of the process is that lftp downloads the >>> server's certificate when it negotiates TLS, then follows the >>> certificate chain up to the certificate of a root CA, and trusts that >>> root CA because it is in my list of trusted third parties, that I >>> indicate to lftp using "ssl:ca-file >>> /etc/ssl/certs/ca-certificates.crt" (the root CA certificates bundle >>> file). This is, as far as I know, what web browsers do when they >>> connect to HTTPS hosts (isn't it?). >>> >>> This doesn't appear to work so I guess I don't understand right. The >>> debug output is: >>> >>> $ lftp -d -p 21 -u USER,PASS SERVER.seedbox.fr >>> lftp u...@server.seedbox.fr:~> set ssl:ca-file >>> /etc/ssl/certs/ca-certificates.crt >>> lftp u...@server.seedbox.fr:~> ls >>> Connecting to SERVER.seedbox.fr (IPADDRESS) port 21 >>> <--- 220-- Welcome to Pure-FTPd [privsep] [TLS] -- >>> <--- [other 220 info] >>> ---> FEAT >>> <--- [feat reply] >>> ---> AUTH TLS >>> <--- 234 AUTH TLS OK. >>> ---> OPTS UTF8 ON >>> Certificate: >>> C=FR,postalCode=77310,ST=Seine-et-Marne,L=PRINGY,street=IMPASSE >>> DU BREAU,O=SDBX FRANCE,OU=0002 529997199,CN=*.seedbox.fr >>> Issued by: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA >>> Limited,CN=COMODO RSA Organization Validation Secure Server CA >>> ERROR: Certificate verification: Not trusted >>> Certificate verification: Not trusted >>> Closing control socket >>> ls: Fatal error: Certificate verification: Not trusted >>> >>> However, if I download the server's certificate beforehand, using a >>> web browser or the OpenSSL CLI (openssl s_client -connect >>> SERVER.seedbox.fr:21 -starttls ftp), and then points lftp to this >>> certificate using "ssl ca-file >>> /path/to/manually/downloaded/server/certificate.crt", the certificate >>> verification succeeds. Why? What is verified in this case, precisely? >>> That the certificate lftp downloads from the server during TLS >>> negotiation is the same as one that was previously downloaded? How >>> does that authenticates the server? >>> >>> And does this mean that the user has to maintain a certificate >>> database of the servers they connect to? I thought the point of >>> certificate hierarchies was that the user would only have to maintain >>> a short list of trusted third party certificates (the root CA >>> certificates). >>> >>> Thanks in advance for your help >>> Naël ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
Re: [lftp] Make certificate verification great again
Thanks for your answer. I have checked that Comodo's root CA certificate is present in the certificate bundle file, however Comodo's intermediate CA certificate (that signed the server's certificate) isn't, as is normal if I understand correctly. The certificate hierarchy is as follow (as shown by Firefox > Page Info when connecting to https://www.seedbox.fr): COMODO RSA Certification Authority COMODO RSA Organization Validation Secure Server CA *.seedbox.fr Certificate validation fails as described whether I point to the root CA certificate bundle file or to the relevant root certificate only: set ssl:ca-file /etc/ssl/certs/ca-certificates.crt set ssl:ca-file /etc/ssl/certs/COMODO_RSA_Certification_Authority.pem set ssl:ca-file /path/to/manually/downloaded/COMODORSACertificationAuthority.crt It succeeds if I point to the intermediate CA certificate, but I shouldn't have to do that, should I? set ssl:ca-file /path/to/manually/downloaded/COMODORSAOrganizationValidationSecureServerCA.crt I checked using lftp 4.6.3a (Ubuntu 16.04) and 4.7.7 (latest stable) BTW. On Sun, Mar 12, 2017 at 8:29 AM, Alexander Lukyanov wrote: > Your understanding of CA is correct. The Comodo certificate should be > present in the CA bundle for the verification to succeed. > > > вс, 12 марта 2017, 5:16 Nathanaël Naeri : >> >> I'm trying to connect to a FTP server that supports explicit FTPS >> using TLS, but I can't get certificate verification working. Most of >> the online help I find advises disabling certificate verification with >> "ssl:verify-certificate no", and I assume this is not good advice. >> >> My current understanding of the process is that lftp downloads the >> server's certificate when it negotiates TLS, then follows the >> certificate chain up to the certificate of a root CA, and trusts that >> root CA because it is in my list of trusted third parties, that I >> indicate to lftp using "ssl:ca-file >> /etc/ssl/certs/ca-certificates.crt" (the root CA certificates bundle >> file). This is, as far as I know, what web browsers do when they >> connect to HTTPS hosts (isn't it?). >> >> This doesn't appear to work so I guess I don't understand right. The >> debug output is: >> >> $ lftp -d -p 21 -u USER,PASS SERVER.seedbox.fr >> lftp u...@server.seedbox.fr:~> set ssl:ca-file >> /etc/ssl/certs/ca-certificates.crt >> lftp u...@server.seedbox.fr:~> ls >> Connecting to SERVER.seedbox.fr (IPADDRESS) port 21 >> <--- 220-- Welcome to Pure-FTPd [privsep] [TLS] -- >> <--- [other 220 info] >> ---> FEAT >> <--- [feat reply] >> ---> AUTH TLS >> <--- 234 AUTH TLS OK. >> ---> OPTS UTF8 ON >> Certificate: >> C=FR,postalCode=77310,ST=Seine-et-Marne,L=PRINGY,street=IMPASSE >> DU BREAU,O=SDBX FRANCE,OU=0002 529997199,CN=*.seedbox.fr >> Issued by: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA >> Limited,CN=COMODO RSA Organization Validation Secure Server CA >> ERROR: Certificate verification: Not trusted >> Certificate verification: Not trusted >> Closing control socket >> ls: Fatal error: Certificate verification: Not trusted >> >> However, if I download the server's certificate beforehand, using a >> web browser or the OpenSSL CLI (openssl s_client -connect >> SERVER.seedbox.fr:21 -starttls ftp), and then points lftp to this >> certificate using "ssl ca-file >> /path/to/manually/downloaded/server/certificate.crt", the certificate >> verification succeeds. Why? What is verified in this case, precisely? >> That the certificate lftp downloads from the server during TLS >> negotiation is the same as one that was previously downloaded? How >> does that authenticates the server? >> >> And does this mean that the user has to maintain a certificate >> database of the servers they connect to? I thought the point of >> certificate hierarchies was that the user would only have to maintain >> a short list of trusted third party certificates (the root CA >> certificates). >> >> Thanks in advance for your help >> Naël ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp
[lftp] Make certificate verification great again
I'm trying to connect to a FTP server that supports explicit FTPS using TLS, but I can't get certificate verification working. Most of the online help I find advises disabling certificate verification with "ssl:verify-certificate no", and I assume this is not good advice. My current understanding of the process is that lftp downloads the server's certificate when it negotiates TLS, then follows the certificate chain up to the certificate of a root CA, and trusts that root CA because it is in my list of trusted third parties, that I indicate to lftp using "ssl:ca-file /etc/ssl/certs/ca-certificates.crt" (the root CA certificates bundle file). This is, as far as I know, what web browsers do when they connect to HTTPS hosts (isn't it?). This doesn't appear to work so I guess I don't understand right. The debug output is: $ lftp -d -p 21 -u USER,PASS SERVER.seedbox.fr lftp u...@server.seedbox.fr:~> set ssl:ca-file /etc/ssl/certs/ca-certificates.crt lftp u...@server.seedbox.fr:~> ls Connecting to SERVER.seedbox.fr (IPADDRESS) port 21 <--- 220-- Welcome to Pure-FTPd [privsep] [TLS] -- <--- [other 220 info] ---> FEAT <--- [feat reply] ---> AUTH TLS <--- 234 AUTH TLS OK. ---> OPTS UTF8 ON Certificate: C=FR,postalCode=77310,ST=Seine-et-Marne,L=PRINGY,street=IMPASSE DU BREAU,O=SDBX FRANCE,OU=0002 529997199,CN=*.seedbox.fr Issued by: C=GB,ST=Greater Manchester,L=Salford,O=COMODO CA Limited,CN=COMODO RSA Organization Validation Secure Server CA ERROR: Certificate verification: Not trusted Certificate verification: Not trusted Closing control socket ls: Fatal error: Certificate verification: Not trusted However, if I download the server's certificate beforehand, using a web browser or the OpenSSL CLI (openssl s_client -connect SERVER.seedbox.fr:21 -starttls ftp), and then points lftp to this certificate using "ssl ca-file /path/to/manually/downloaded/server/certificate.crt", the certificate verification succeeds. Why? What is verified in this case, precisely? That the certificate lftp downloads from the server during TLS negotiation is the same as one that was previously downloaded? How does that authenticates the server? And does this mean that the user has to maintain a certificate database of the servers they connect to? I thought the point of certificate hierarchies was that the user would only have to maintain a short list of trusted third party certificates (the root CA certificates). Thanks in advance for your help Naël ___ lftp mailing list lftp@uniyar.ac.ru http://univ.uniyar.ac.ru/mailman/listinfo/lftp