Re: proto_ftp.c
On Fri, Feb 25, 2011 at 03:52:56PM -0500, Ben Timby wrote: 2011/2/25 Krzysztof Ol??dzki o...@ans.pl: Proxing FTP is much more complicated than simply providing one additional command for passing client's IP address. Please note that FTP is based on two independent TCP connections: control and data. You need to analyze a control stream and modify on-fly data (port numbers and ip addresses) and set up additional sockets and initiate additional connections to handle data stream. To do this you also need to handle both PASV/EPSV (passive) and PORT/EPRT (active) modes. It is of course doable but the amount of work is quite big. I even was recently asked to implement such function as a sponsored feature. After a short conversation with my possible employer we decided that it would took too much time to be profitable and cost effective. Instead another solution was chosen - LVS DR. I have all of that figured out. I simply would like to have the client's IP address. I only use HAProxy for the command channel. Data channel is handled simply by choosing a different PASV port range for each backend server, and NATing the right range to the right server. Outbound Active connections are similarly S-NAT'd to the appropriate outbound address. I just want the last piece of the puzzle. I must admit I'm not sure I really understand how your whole thing will work. Would you please show us the sequence of commands and exchanges you'd get between the client, haproxy and the server for both the command and data connection ? My point is simple : if this hack makes it possible to simply provide FTP load balancing to users who can't easily reconfigure routing, client or server settings, we could consider it. But if this only works in your own environment, you'd better have it as a specific patch, because we can't implement and support everyone's specific use case. Regards, Willy
Re: proto_ftp.c
OK, first off, the FTP SITE command is reserved for specific FTP server extensions. It is commonly used for banning IP addresses. So that the user can, via their FTP client issue a command such as: SITE ADDIP XXX.XXX.XXX.XXX The server knows what to do with this IP address because it has an extension loaded that stores the provided IP into a ban list. This is of course implementation specific, some servers will handle this extension, some don't. SITE Command description: http://www.nsftools.com/tips/RawFTP.htm#SITE Apache FTP Server SITE command: http://incubator.terra-intl.com/projects/ftpserver/site_cmd.html Relevant RFC: http://www.faqs.org/rfcs/rfc959.html -- SITE PARAMETERS (SITE) This command is used by the server to provide services specific to his system that are essential to file transfer but not sufficiently universal to be included as commands in the protocol. The nature of these services and the specification of their syntax can be stated in a reply to the HELP SITE command. -- With that in mind, the sequence I am thinking of would be: Client HAProxy Backend ** connect | | connect --| | SITE IP --| | --250 OK | *===* USER ---| | USER -| | --250 OK | -250 OK | ** Everything below the horizontal (==) line is as usual, HAProxy just sends an initial SITE command to the backend FTP server to let it know the client's real IP address. It then starts shoveling data from the client to the backend as usual. The fly in the ointment is that the backend FTP server will need to be able to handle this SITE command. I maintain my own FTP server daemon, so mine will of course support this. I will contribute patches back to the community for it. Other FTP daemons like proftpd can easily support this SITE extension using add-on modules. The module simply looks for the client IP provided by the SITE command, then overwrites the variable containing the remote IP address so that the server can make active FTP connections to the right place. Also the logs would then contain the correct client IP address. It is kinda like the X-Forwarded-For header on HTTP, but using the SITE command on FTP (which is the right place for this according to the RFCs involved). I am investigating the feasibility and interest in a feature such as this at this point.
Re: proto_ftp.c
On Sat, Feb 26, 2011 at 09:11:42AM -0500, Ben Timby wrote: OK, first off, the FTP SITE command is reserved for specific FTP server extensions. It is commonly used for banning IP addresses. So that the user can, via their FTP client issue a command such as: SITE ADDIP XXX.XXX.XXX.XXX The server knows what to do with this IP address because it has an extension loaded that stores the provided IP into a ban list. This is of course implementation specific, some servers will handle this extension, some don't. It looks like the command is only accepted for users with sufficient privileges. In my case, vsftpd rejects the SITE command before the user authenticates, which seems to be in line with Apache's description. (...) -- With that in mind, the sequence I am thinking of would be: Client HAProxy Backend ** connect | | connect --| | SITE IP --| | --250 OK | *===* USER ---| | USER -| | --250 OK | -250 OK | ** Everything below the horizontal (==) line is as usual, HAProxy just sends an initial SITE command to the backend FTP server to let it know the client's real IP address. It then starts shoveling data from the client to the backend as usual. That looks close to the way we make the PROXY protocol work, except haproxy here will have to consume one response line. What happens if the client issues a SITE IP command here ? It looks dangerous to me to let him change its apparent IP address. Or the server will have to ensure it accepts the SITE command only once, but that's dangerous. The fly in the ointment is that the backend FTP server will need to be able to handle this SITE command. I maintain my own FTP server daemon, so mine will of course support this. I will contribute patches back to the community for it. If you maintain your own servers, wouldn't you be interested in making them support the proxy protocol we've added between stunnel and haproxy ? It provides the server with a first line containing the protocol (TCPv4, TCPv6), source and destination addresses and ports, and does not require a state to consume a response. Also since by definition it can only appear on the first line of the connection, there is no risk a client would send it. It would work like this : Client HAProxy Backend ** connect | | connect -| | PROXY TCP4 IP ...--| *===* | -- 220 Ready | USER --- | | USER --- | | -250 OK | -250 OK | ** I'm just checking how we could implement something simple, reliable and durable. Regards, Wlily
Re: proto_ftp.c
On Sat, Feb 26, 2011 at 9:34 AM, Willy Tarreau w...@1wt.eu wrote: If you maintain your own servers, wouldn't you be interested in making them support the proxy protocol we've added between stunnel and haproxy ? It provides the server with a first line containing the protocol (TCPv4, TCPv6), source and destination addresses and ports, and does not require a state to consume a response. Also since by definition it can only appear on the first line of the connection, there is no risk a client would send it. It would work like this : Client HAProxy Backend ** connect | | connect -| | PROXY TCP4 IP ...--| *===* | -- 220 Ready | USER --- | | USER --- | | -250 OK | -250 OK | ** I'm just checking how we could implement something simple, reliable and durable. As am I. I was not aware of that protocol, but that sounds like it would fit the bill. Is there any other information about that? Is HAProxy able to insert that protocol line or is that an extension to stunnel?
Re: proto_ftp.c
On Sat, Feb 26, 2011 at 11:09:28AM -0500, Ben Timby wrote: On Sat, Feb 26, 2011 at 9:34 AM, Willy Tarreau w...@1wt.eu wrote: If you maintain your own servers, wouldn't you be interested in making them support the proxy protocol we've added between stunnel and haproxy ? It provides the server with a first line containing the protocol (TCPv4, TCPv6), source and destination addresses and ports, and does not require a state to consume a response. Also since by definition it can only appear on the first line of the connection, there is no risk a client would send it. It would work like this : Client HAProxy Backend ** connect | | connect -| | PROXY TCP4 IP ...--| *===* | -- 220 Ready | USER --- | | USER --- | | -250 OK | -250 OK | ** I'm just checking how we could implement something simple, reliable and durable. As am I. I was not aware of that protocol, but that sounds like it would fit the bill. Is there any other information about that? Is HAProxy able to insert that protocol line or is that an extension to stunnel? It has been implement on the client side in haproxy but not yet on the server side, though it should not be difficult at all. You can find information on the protocol here : http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt The goal was to make it very strict and simple to parse in order to encourage a broader adoption than just the stunnel+haproxy tandem. Regards, Willy
Re: proto_ftp.c
On Sat, Feb 26, 2011 at 12:04 PM, Willy Tarreau w...@1wt.eu wrote: It has been implement on the client side in haproxy but not yet on the server side, though it should not be difficult at all. You can find information on the protocol here : http://haproxy.1wt.eu/download/1.5/doc/proxy-protocol.txt The goal was to make it very strict and simple to parse in order to encourage a broader adoption than just the stunnel+haproxy tandem. Thanks Willy, I will look into this on Monday.
proto_ftp.c
First of all, sorry for the previous list spam. I pasted the wrong address while subscribing. I am setting up FTP load balancing using HAProxy. The rub is that I want something similar to the X-Forwarded-For header supported in HTTP. I am aware of TPROXY, but I don't wish to maintain my own packages for the kernel, xen and all the dependencies this entails. A simpler user-space solution would suit me much better. I would like to patch HAProxy so that it provides specialized FTP handling in the form of an FTP SITE command. Such that when optionally enabled, it will inject the following FTP command at the beginning of the TCP stream. SITE IP=XXX.XXX.XXX.XXX My backend FTP server will know how to deal with this site command and store the IP address for use internally. This would negate the need for TPROXY and seems fairly straightforward. Any feedback or thoughts on this topic? Thanks.
Re: proto_ftp.c
On 2011-02-25 18:29, Ben Timby wrote: First of all, sorry for the previous list spam. I pasted the wrong address while subscribing. I am setting up FTP load balancing using HAProxy. The rub is that I want something similar to the X-Forwarded-For header supported in HTTP. I am aware of TPROXY, but I don't wish to maintain my own packages for the kernel, xen and all the dependencies this entails. TPROXY is now included in the mainline. Howevwe, I'm not sure if it is the solution you are interested in. A simpler user-space solution would suit me much better. I would like to patch HAProxy so that it provides specialized FTP handling in the form of an FTP SITE command. Such that when optionally enabled, it will inject the following FTP command at the beginning of the TCP stream. SITE IP=XXX.XXX.XXX.XXX My backend FTP server will know how to deal with this site command and store the IP address for use internally. This would negate the need for TPROXY and seems fairly straightforward. Any feedback or thoughts on this topic? Proxing FTP is much more complicated than simply providing one additional command for passing client's IP address. Please note that FTP is based on two independent TCP connections: control and data. You need to analyze a control stream and modify on-fly data (port numbers and ip addresses) and set up additional sockets and initiate additional connections to handle data stream. To do this you also need to handle both PASV/EPSV (passive) and PORT/EPRT (active) modes. It is of course doable but the amount of work is quite big. I even was recently asked to implement such function as a sponsored feature. After a short conversation with my possible employer we decided that it would took too much time to be profitable and cost effective. Instead another solution was chosen - LVS DR. Best regards, Krzysztof Olędzki
