Re: How to ascertain that the connection is secure? Re: Cannot connect via FTPES; lftp hangs
Hello Daniel and all, On Thursday 02 June 2011 12:06:19 am you wrote: You can conveniently set settings to apply only to specific servers, for example set ftp:ssl-force/ftp.example.com on set ftp:use-feat/example.com off set ftp:ssl-force/example.com on This feature is s cool and s undocumented anywhere I looked (man page as well as online lftp tutorials). Also the exact syntax to use on configuration files is not clearly mentioned anywhere. Luckily, I found some forum posts where people had posted their config files, which was helpful. I have added this in the new wiki page: http://linux.overshoot.tv/wiki/networking/lftp#Configuration_files Let me know if I have anything wrong or if I missed something potentially useful. Looks like a misbehaving server. A friendlier server would advertise AUTH TLS in the FEAT reply so that clients connecting know it's supported. That was useful. Apparently, in my case, this was the source of the difficulty. I documented this possibility in the wiki, using the examples from this thread. Earlier in the thread, you wrote: don't even use ftps:// with lftp since that is for implicit ftps, [...] For explicit TLS just open it like ftp://; or you don't even need to specify a protocol since ftp is the default. Using an encrypted control connection when available is also turned on by default in lftp (set ftp:use-feat yes, set ftp:ssl-allow yes). This was very helpful as I was trying to complete the table with the various protocols and their URI schemes. Let me know if you see and glaring mistakes: http://linux.overshoot.tv/wiki/networking/lftp#A_multitude_of_protocols I hope that this table alone will make it easier for future users to understand when to use what. In my case: the server uses FTPES which is really FTPS (explicit) except that I cannot use the FTPS:// URI scheme (used for FTPS implicit) but rather the FTP:// scheme, except that it won't connect securely because the sever won't acknowledge using FTPES in the first place even though it really does! I think that's a nice summary of the whole thread!! LOL! No wonder I was confused. :) I hope to be the last one to be confused on this specific issue. Keep in mind I'm just a fellow user hanging around on this mailing list, my only qualification being a long-time satisified user of lftp. :) That's already a lot. See my blog on the whole experience: http://linux.overshoot.tv/blogs/augustin/best_combination_linux_users_lftp_example There are two additional things to note in regards to using TLS with ftp. First is certificate verification, same as when you'd visit an https web site. It's of little comfort that your password was sent with strong encryption if you sent it to the wrong guy. TLS uses certificates to help ensure you are connected to who you intented to. A basic setup is to make sure certificate verification is turned on (these too are on by default in the current version): set ssl:check-hostname yes set ssl:verify-certificate yes set ssl:ca-file path to your a certificate bundle file, containing the certificate authorities you choose to trust An easy answer to what bundle of certificates authorities to trust is to just take what your browser vendor (eg. Mozilla) or operating system vendor supplies. Then you'll be generally as safe as you'd be accessing https web site in your browser. More paranoid users might hand-pick what certificate authorities to trust on their own. The second important part for ftp with TLS is unique for ftp's peculiarity of using multiple connections, one as a control channel and a separate one for transferring data. lftp by default is set to encrypt only the control channel and leave the data channel in the clear. I find computers and Internet connections plenty fast enough nowadays to afford encrypting everything, so just turn it all on: set ftp:ssl-protect-data yes set ftp:ssl-protect-list yes As you can see from all this, everything is a whole lot simpler if you just connect with sftp to an ssh2 server instead. Everything is always encrypted no matter what, no separate control and data channels to worry about, no certificate authorities to trust (a host fingerprint is verified instead). Thank you for this explanation. It is, at very long last, starting to make sense to me. I couldn't have said it better, so, as per your authorization, I have added it almost verbatim to the wiki. Many, many, many, thanks for your tremendous help. :) Blessings, Augustin. -- Friends: http://www.reuniting.info/ My projects: http://astralcity.org/ http://3enjeux.overshoot.tv/ http://linux.overshoot.tv/ http://overshoot.tv/ http://charityware.info/ http://masquilier.org/ http://openteacher.info/ http://minguo.info/ http://www.wechange.org/ http://searching911.info/ .
Re: How to ascertain that the connection is secure? Re: Cannot connect via FTPES; lftp hangs
On Jun 7, 2011, at 09:59, augustin wrote:
set ftp:use-feat/example.com off
set ftp:ssl-force/example.com on
This feature is s cool and s undocumented anywhere I looked (man page
as well as online lftp tutorials).
It's there in the man page, though perhaps not easy to understand what it's
talking about on first read:
”You can set one variable several times for different closures, and thus you
can get a particular settings for particular state. The closure is to be
specified after variable name separated with slash `/'.
The closure for `dns:', `net:', `ftp:', `http:', `hftp:' domain variables is
currently just the host name as you specify it in the `open' command (with some
exceptions where closure is meaningless, e.g. dns:cache-size). For some `cmd:'
domain variables the closure is current URL without path. For other variables
it is not currently used.
See examples in the sample lftp.conf.“
Also the exact syntax to use on configuration files is not clearly mentioned
anywhere. Luckily, I found some forum posts where people had posted their
config files, which was helpful.
As it said above, that sample system-wide lftp.conf file which is normally
installed as {prefix}/etc/lftp.conf is a good place to start.
Re: How to ascertain that the connection is secure?
On Wednesday 01 June 2011 10:44:29 pm augustin wrote: When searching the web earlier, I found other people asking very similar questions. There is a lot of confusion surrounding these topics; that's why I am offering, with your help, to clearly document all of this. This is a 2006 post which is almost exactly the same scenario as mine, and the poster is asking the very same questions: http://www.webmasterworld.com/forum40/1638.htm Does this mean that when I did lftp ftp://username... it actually connected through ftps? Augustin. -- Friends: http://www.reuniting.info/ My projects: http://astralcity.org/ http://3enjeux.overshoot.tv/ http://linux.overshoot.tv/ http://overshoot.tv/ http://charityware.info/ http://masquilier.org/ http://openteacher.info/ http://minguo.info/ http://www.wechange.org/ http://searching911.info/ .
Re: How to ascertain that the connection is secure? Re: Cannot connect via FTPES; lftp hangs
On Jun 1, 2011, at 16:44, augustin wrote: With the settings you suggest, I did not really suggest changing any settings, I just noted that those are already the defaults so you shouldn't need to do anything and lftp will automatically use the secure connection if the server tells it that it's supported. how do I know I have a secure connection? You are already on the right track, just turn on ssl-force for at least that server and then lftp will refuse to proceed without encryption. You can conveniently set settings to apply only to specific servers, for example set ftp:ssl-force/ftp.example.com on Then you can just leave that forever in your ~/.lftp/rc file and not have to think about it ever again. Again, the man page does not differentiate between the various protocols, so I am double-plus unclear as to what options are available for which protocol. Actually the settings are all nicely prefixed by the protocol they apply to, or a different prefix to note that it applies in a more generic way to all protocols. It's ftp:ssl-force so it applies to ftp. http:user-agent is for http, and so on. I tried: set ftp:ssl-force true but I get: Login failed: ftp:ssl-force is set and server does not support or allow SSL Looks like a misbehaving server. What's odd is the ftp:use-feat option. The following lines are only present with: set ftp:use-feat true --- FEAT --- 211-Extensions supported: --- EPRT --- IDLE --- MDTM --- SIZE --- REST STREAM --- MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*; --- MLSD --- ESTP --- PASV --- EPSV --- SPSV --- 211 End. --- OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid; --- 200 MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique A friendlier server would advertise AUTH TLS in the FEAT reply so that clients connecting know it's supported. Something like this: --- FEAT --- 211-Features: --- AUTH SSL --- AUTH TLS --- EPRT --- EPSV --- MDTM --- PASV --- PBSZ --- PROT --- REST STREAM --- SIZE --- TVFS --- UTF8 --- 211 End The following lines are only present with: set ftp:use-feat false --- AUTH TLS --- 234 AUTH TLS OK. Good, so it looks like your server supports AUTH TLS despite not admitting to it in the FEAT reply. Just turn use-feat off for that server then. set ftp:use-feat/example.com off set ftp:ssl-force/example.com on When searching the web earlier, I found other people asking very similar questions. There is a lot of confusion surrounding these topics; that's why I am offering, with your help, to clearly document all of this. Keep in mind I'm just a fellow user hanging around on this mailing list, my only qualification being a long-time satisified user of lftp. :) There are two additional things to note in regards to using TLS with ftp. First is certificate verification, same as when you'd visit an https web site. It's of little comfort that your password was sent with strong encryption if you sent it to the wrong guy. TLS uses certificates to help ensure you are connected to who you intented to. A basic setup is to make sure certificate verification is turned on (these too are on by default in the current version): set ssl:check-hostname yes set ssl:verify-certificate yes set ssl:ca-file path to your a certificate bundle file, containing the certificate authorities you choose to trust An easy answer to what bundle of certificates authorities to trust is to just take what your browser vendor (eg. Mozilla) or operating system vendor supplies. Then you'll be generally as safe as you'd be accessing https web site in your browser. More paranoid users might hand-pick what certificate authorities to trust on their own. The second important part for ftp with TLS is unique for ftp's peculiarity of using multiple connections, one as a control channel and a separate one for transferring data. lftp by default is set to encrypt only the control channel and leave the data channel in the clear. I find computers and Internet connections plenty fast enough nowadays to afford encrypting everything, so just turn it all on: set ftp:ssl-protect-data yes set ftp:ssl-protect-list yes As you can see from all this, everything is a whole lot simpler if you just connect with sftp to an ssh2 server instead. Everything is always encrypted no matter what, no separate control and data channels to worry about, no certificate authorities to trust (a host fingerprint is verified instead).
Re: How to ascertain that the connection is secure? Re: Cannot connect via FTPES; lftp hangs
On Thursday 02 June 2011 12:04:07 am you wrote: Good, so it looks like your server supports AUTH TLS despite not admitting to it in the FEAT reply. Just turn use-feat off for that server then. set ftp:use-feat/example.com off set ftp:ssl-force/example.com on Great. That seems to do the trick. :) how do I know I have a secure connection? --- AUTH TLS --- 234 AUTH TLS OK. Is that it? Keep in mind I'm just a fellow user hanging around on this mailing list, my only qualification being a long-time satisified user of lftp. :) I repeat what i just told you privately: I appreciate all the more all the time you spend trying to help me. I am very grateful. http://linux.overshoot.tv/ticket/176#comment-225 I need to go to bed (I'll sleep better tonight ;)). I'll try to digest all the information tomorrow and summarize the important bits in the docs. Thank you Daniel again for all your help so far :) Blessings, Augustin. -- Friends: http://www.reuniting.info/ My projects: http://astralcity.org/ http://3enjeux.overshoot.tv/ http://linux.overshoot.tv/ http://overshoot.tv/ http://charityware.info/ http://masquilier.org/ http://openteacher.info/ http://minguo.info/ http://www.wechange.org/ http://searching911.info/ .
