Re: [RFC] ignore ftp_epsv off for IPv6
[I may have figured out why we could not make progress before, and we may be finally converging on a solution. P4 at the bottom. ] On 01/29/2014 08:51 PM, Amos Jeffries wrote: On 30/01/2014 1:44 p.m., Alex Rousskov wrote: P1: ignore ftp_epsv off for IPv6 servers. What I was meaning was this nasty beast: Squid requests EPSV 2 (IPv6 data connection), server responds 522 ... (1) indicating only accepting IPv4 data connections. ftp_epsv off disabling IPv4 ... EPSV blocked to IPv6 server. I suspect we are using a different definition of IPv6 server. For this discussion, I am defining IPv6 server as a server that Squid [intends to] send FTP commands using an IPv6 destination address. The data connection protocol is irrelevant for this definition. Given my definition, proposal P1 above does not result in ftp_epsv off disabling IPv4 for the IPv6 server you are describing. With P1, ftp_epsv setting would have no effect on IPv6 servers at all. I define IPv4 server the same way (s/6/4/g), of course. The main point being that in FTP with two TCP connections to a Dual-stack server there is no simple distinction IPv4 server or IPv6 server. I am using the definition above. It is not affected by the data connections. I am sorry if I have not been clear about this from the beginning and have not realized that you actually do not know what I mean by IPv6 server. More importantly perhapse these changes to IPv6 behaviour do nothing for your described problem case which is entirely in IPv4-only traffic. Perhaps now that we have resolved the terminology problem, you will see how P1-3 work to address the case I am after, using my definition of the IPv6 server. P2: add ftp_epsv ipv6 that will disable EPSV use with IPv4 servers. P3: rename ftp_epsv to ftp_epsv_for_ipv4 (essentially) and naturally ignore the renamed version for IPv6 servers. Take a longer view. We will face matching breakage cases with all the commands we implement. Using this approach will explode the currently 4 directives out to 12 on/off settings when the other currently cases are dealt with. I want something simpler and more intuitive for admin than directives_documented_with_a_sentence_about_what_is_does_in_its_own_name on/off I would certainly welcome simpler and more intuitive alternatives! ftp_epsv with a set of flag options about when and how it can be used seems simpler to me than a bunch of separate on/off directives. Which is what my P2 proposes, but my understanding is that you objected to that as not longer view enough and/or exploding too much. Simple: ... leave the (working) attempt-based algorithm from RFC 2428 P1: extend the incomplete ftp_epsv squid.conf setting to configure per-protocol skipping of EPSV 1|2 Can you detail that a little please? I do not understand what you are proposing. What is protocol and skipping in this context, exactly? It may help if you can give a configuration example addressing the use case in question. protocol: IPv4 or IPv6 data connection skipping ... like the dictionary says. Skip over one step in the attempt-based algorithm. OK. I cannot think of a configuration approach that would allow Squid to make the right decision in my use case based on the data connection protocol. I do not want to tell Squid to disable support for IPv4 data connections or IPv6 data connections! Can you give an example of how this long view configuration could look like when addressing my use case? The ftp_epsv off solves yoru problem case but currently skips over both IPv4 and IPv6 connections without considering their protocol type. ftp_epsv off (as currently implemented) solves my use case but disables support for IPv6 servers that cannot use IPv4 data connections. That is why I am looking for a better solution. As you say that can be annoying *if* only one type is broken. Your problem case is also solved by adding a flag ftp_epsv ipv6 that means only IPV6 (EPSV 2) is to be used. No, my P2 means that EPSV is only sent to IPv6 servers (my definition). Those IPv6 servers are still free to use IPv4 and IPv6 data connections. P2 is less restrictive than you think. Needless to say, we may need to add more knobs to address more cases. Still focusing on just EPSV, Squid can make two decisions: * What flavor of EPSV to send: none, EPSV, EPSV 1, EPSV 2. * Where to send it: Any server, IPv4 server, IPv6 server. (using my control connection-based definition of IPvN server) To gain full control, we would need to support something like this: * P4: ftp_epsv flavor [for destination] For example: # do not send EPSV do any servers: ftp_epsv off # do not send EPSV to IPv4 servers: ftp_epsv off for ipv4_server # send EPSV 1 to IPv4 servers: ftp_epsv 1 for ipv4_server # send parameterless EPSV to IPv6 servers: ftp_epsv on for ipv6_server with on as the default for not explicitly covered cases (we can discuss whether on is the right
Re: [RFC] ignore ftp_epsv off for IPv6
On 2014-01-31 07:35, Alex Rousskov wrote: [I may have figured out why we could not make progress before, and we may be finally converging on a solution. P4 at the bottom. ] On 01/29/2014 08:51 PM, Amos Jeffries wrote: On 30/01/2014 1:44 p.m., Alex Rousskov wrote: P1: ignore ftp_epsv off for IPv6 servers. What I was meaning was this nasty beast: Squid requests EPSV 2 (IPv6 data connection), server responds 522 ... (1) indicating only accepting IPv4 data connections. ftp_epsv off disabling IPv4 ... EPSV blocked to IPv6 server. I suspect we are using a different definition of IPv6 server. For this discussion, I am defining IPv6 server as a server that Squid [intends to] send FTP commands using an IPv6 destination address. The data connection protocol is irrelevant for this definition. Given my definition, proposal P1 above does not result in ftp_epsv off disabling IPv4 for the IPv6 server you are describing. With P1, ftp_epsv setting would have no effect on IPv6 servers at all. I define IPv4 server the same way (s/6/4/g), of course. The main point being that in FTP with two TCP connections to a Dual-stack server there is no simple distinction IPv4 server or IPv6 server. I am using the definition above. It is not affected by the data connections. I am sorry if I have not been clear about this from the beginning and have not realized that you actually do not know what I mean by IPv6 server. More importantly perhapse these changes to IPv6 behaviour do nothing for your described problem case which is entirely in IPv4-only traffic. Perhaps now that we have resolved the terminology problem, you will see how P1-3 work to address the case I am after, using my definition of the IPv6 server. P2: add ftp_epsv ipv6 that will disable EPSV use with IPv4 servers. P3: rename ftp_epsv to ftp_epsv_for_ipv4 (essentially) and naturally ignore the renamed version for IPv6 servers. Take a longer view. We will face matching breakage cases with all the commands we implement. Using this approach will explode the currently 4 directives out to 12 on/off settings when the other currently cases are dealt with. I want something simpler and more intuitive for admin than directives_documented_with_a_sentence_about_what_is_does_in_its_own_name on/off I would certainly welcome simpler and more intuitive alternatives! ftp_epsv with a set of flag options about when and how it can be used seems simpler to me than a bunch of separate on/off directives. Which is what my P2 proposes, but my understanding is that you objected to that as not longer view enough and/or exploding too much. Simple: ... leave the (working) attempt-based algorithm from RFC 2428 P1: extend the incomplete ftp_epsv squid.conf setting to configure per-protocol skipping of EPSV 1|2 Can you detail that a little please? I do not understand what you are proposing. What is protocol and skipping in this context, exactly? It may help if you can give a configuration example addressing the use case in question. protocol: IPv4 or IPv6 data connection skipping ... like the dictionary says. Skip over one step in the attempt-based algorithm. OK. I cannot think of a configuration approach that would allow Squid to make the right decision in my use case based on the data connection protocol. I do not want to tell Squid to disable support for IPv4 data connections or IPv6 data connections! Can you give an example of how this long view configuration could look like when addressing my use case? The ftp_epsv off solves yoru problem case but currently skips over both IPv4 and IPv6 connections without considering their protocol type. ftp_epsv off (as currently implemented) solves my use case but disables support for IPv6 servers that cannot use IPv4 data connections. That is why I am looking for a better solution. As you say that can be annoying *if* only one type is broken. Your problem case is also solved by adding a flag ftp_epsv ipv6 that means only IPV6 (EPSV 2) is to be used. No, my P2 means that EPSV is only sent to IPv6 servers (my definition). Those IPv6 servers are still free to use IPv4 and IPv6 data connections. P2 is less restrictive than you think. Needless to say, we may need to add more knobs to address more cases. Still focusing on just EPSV, Squid can make two decisions: * What flavor of EPSV to send: none, EPSV, EPSV 1, EPSV 2. * Where to send it: Any server, IPv4 server, IPv6 server. (using my control connection-based definition of IPvN server) To gain full control, we would need to support something like this: * P4: ftp_epsv flavor [for destination] For example: # do not send EPSV do any servers: ftp_epsv off # do not send EPSV to IPv4 servers: ftp_epsv off for ipv4_server # send EPSV 1 to IPv4 servers: ftp_epsv 1 for ipv4_server # send parameterless EPSV to IPv6 servers: ftp_epsv on for ipv6_server s/ on for ipv6_server/ ipv6/ s/ off for
Re: [RFC] ignore ftp_epsv off for IPv6
On 01/30/2014 03:35 PM, Amos Jeffries wrote: P4-b: Shall we skip the arguing and go straight to ACL driven in that format? I think it may be faster to simply write up a patch for ACLs with a default allow all and simply allow/deny action choice than to continue discussions around the on/off scoping. We are clearly focusing on different use-cases and error conditions being more or less subjectively important. The admin on the ground can probably get that right far better than we can anyway. Do you want me to add an ipv4_server and ipv6_server hard-coded ACLs? They would work in contexts where the server address is known (any origin server: HTTP, FTP, Gopher, etc.). I fear opening another big can of worms with this! If we do not add those ACLs, how will an admin know that Squid is going to talk to an IPv6 server (my definition)? We should still keep on and off keywords for backward compatibility, right? So, we will have: ftp_epsv on ftp_epsv off ftp_epsv deny ipv4_server ftp_epsv deny ipv6_server but folks can use other, customer ACLs instead of ipv4_server and ipv6_server. The action on the first matching ftp_epsv line is applied. Anything else for ftp_epsv? Thank you, Alex.
Re: [RFC] ignore ftp_epsv off for IPv6
On 31/01/2014 12:17 p.m., Alex Rousskov wrote: On 01/30/2014 03:35 PM, Amos Jeffries wrote: P4-b: Shall we skip the arguing and go straight to ACL driven in that format? I think it may be faster to simply write up a patch for ACLs with a default allow all and simply allow/deny action choice than to continue discussions around the on/off scoping. We are clearly focusing on different use-cases and error conditions being more or less subjectively important. The admin on the ground can probably get that right far better than we can anyway. Do you want me to add an ipv4_server and ipv6_server hard-coded ACLs? They would work in contexts where the server address is known (any origin server: HTTP, FTP, Gopher, etc.). I fear opening another big can of worms with this! If we do not add those ACLs, how will an admin know that Squid is going to talk to an IPv6 server (my definition)? Hmm. Up to you, but I think admin are at least smart enough to figure out the necessary ACL definition if we leave it to them. I would put an example in the squid.conf documentation though, just like the tcp_outgoing_address etc have. We should still keep on and off keywords for backward compatibility, right? Yes. So, we will have: ftp_epsv on ftp_epsv off ftp_epsv deny ipv4_server ftp_epsv deny ipv6_server but folks can use other, customer ACLs instead of ipv4_server and ipv6_server. The action on the first matching ftp_epsv line is applied. Anything else for ftp_epsv? Not that I can think of. Amos
Re: [RFC] ignore ftp_epsv off for IPv6
On 31/01/2014 5:35 p.m., Amos Jeffries wrote: On 31/01/2014 12:17 p.m., Alex Rousskov wrote: On 01/30/2014 03:35 PM, Amos Jeffries wrote: P4-b: Shall we skip the arguing and go straight to ACL driven in that format? I think it may be faster to simply write up a patch for ACLs with a default allow all and simply allow/deny action choice than to continue discussions around the on/off scoping. We are clearly focusing on different use-cases and error conditions being more or less subjectively important. The admin on the ground can probably get that right far better than we can anyway. Do you want me to add an ipv4_server and ipv6_server hard-coded ACLs? They would work in contexts where the server address is known (any origin server: HTTP, FTP, Gopher, etc.). I fear opening another big can of worms with this! If we do not add those ACLs, how will an admin know that Squid is going to talk to an IPv6 server (my definition)? Sorry just realized I can lay our mind to rest that but did not. Any time after peer selection the destination server name/FQDN/IP are known. These ACLs for FTP are being run after the server has been connected and traffic already exchanged. So the IP is most certainly known. Amos
Re: [RFC] ignore ftp_epsv off for IPv6
On 01/29/2014 12:30 AM, Amos Jeffries wrote: off should never be abused to mean half-off. The problem here is that the directive itself was misnamed IMO. It is named correctly for its scope OK, then its scope is wrong. The scope is right. For IPv4, it is right. For IPv6, it is right. For a world where the two co-exist, the approach does not work. It is probably pointless to argue whether it is the name, the scope, the implementation, or something else that went wrong. The fact remains: There is no way for Squid to operate well with any of the currently supported EPSV option settings. Let's focus on that problem. So this is one of those instances you mentioned in the -n discussion where Squid can be badly configured but a smart admin does not do so and we should be allowing configuration rather than denying it. It is the opposite of that: In this case, a smart admin cannot configure Squid to work well, no matter how hard she tries. NP: a smart admin may actually disable both if the network was gatewayed over software which dies on both. The clients could not contact IPv6-only servers, but that is the smart admins intent. Fail-fast on IPv6 rather than trying slow-ish things and killing the gateway router on the way. The use case is different: IPv6 servers work fine. IPv4 servers work fine with standard FTP commands. Transactions stall when Squid attempts to use EPSV with some IPv4 servers. To avoid that breakage, the admin needs a way to tell Squid do not try EPSV with IPv4 servers while still using EPSV with IPv6 servers. If ftp_epsv is on (default): contacting an IPv4 server may break because the network breaks EPSV. If ftp_epsv is off: contacting IPv6 servers breaks because they need EPSV. (I am omitting the cases of ESPV ALL and EPRT for clarity, they do not add value to the current discussion). They *do* add value. Since EPRT may be working perfectly and nullify your second example of when ftp_epsv off. To make sure we are making progress, please do not through in other use cases until we agree on how cover the current one. In the current use case, EPRT setting is irrelevant because transaction stalls before Squid tries to use EPRT (and may stall even if Squid does try to use EPRT, but I do not really know). To avoid misunderstanding, I do agree that any change should not make other important use cases [much] worse, at least not without a discussion about trade-offs. So other use cases are important. I just want to make sure we find a solution for my use case first since it appears to be surprisingly difficult even without adding more use cases into the mix. Also note that by break you mean: IPv6 connectivity to IPv6-only servers over a faulty router is not possible. I do not mean that. IPv6 servers work fine if they are sent an EPSV command. A faulty network (outside of the admin control) breaks IPv4 servers that support EPSV, tricking Squid into sending EPSV to them. So, what can we possibly do that would not make that broken? IPv4 connectivity to IPv4-enabled servers is still possible. Which is the scope of your change, yes? to selectively disable the EPSV command being sent depending on IPv4 or IPv6. Yes, treating IPv4 and IPv6 servers differently is at the root of all my proposals on this thread. I suspect the initial attempt-based algorithm was developed for PASV and PORT only, where it makes sense. It was wrong to add similar EPSV and EPRT configuration to that IPv4-specific algorithm because a single ftp_epsv option setting does work for a mix of IPv4 and IPv6 servers. The algorithm is baked into RFC 2428. See section 2 about the meaning of that 1-digit net-prt parameteron EPSV; 1 = IPv4 wanted, 2 = IPv6. The server accepts or rejects with 522 and a list of supported protocols which can be tried on the followup attempt. Server actions are irrelevant in this use case. It is the network that breaks things. The servers are working fine and comply with FTP RFCs. The data connection to an FTP server is not necessarily going to be the same protocol as the ctrl channel connection due to dual-stack servers in IPv6. The use case under consideration assumes the same IP address for both data and control connections. In theory only EPSV/EPRT should ever be needed. In practice, for IPv4-enabled servers all 4 commands should still work fine ... but do not. In practice, faulty firewalls prevent EPSV from working fine with IPv4 servers that _do_ support EPSV. We can, of course, declare that we are not going to do anything about it. I do not think the Project should take such a position in this case because the fix appears to be relatively straightforward [to me], despite this painful discussion. The only known [to me] negative side-effect is that some IPv4 servers on good networks will not use EPSV if Squid admin choses to disable EPSV for for IPv4 servers. Those servers will continue to use PASV, of course. PASV/PORT
Re: [RFC] ignore ftp_epsv off for IPv6
[if you don't want the point-by-point skip to the end ] On 2014-01-30 05:57, Alex Rousskov wrote: On 01/29/2014 12:30 AM, Amos Jeffries wrote: off should never be abused to mean half-off. The problem here is that the directive itself was misnamed IMO. It is named correctly for its scope OK, then its scope is wrong. The scope is right. For IPv4, it is right. For IPv6, it is right. For a world where the two co-exist, the approach does not work. It is probably pointless to argue whether it is the name, the scope, the implementation, or something else that went wrong. The fact remains: There is no way for Squid to operate well with any of the currently supported EPSV option settings. Let's focus on that problem. So this is one of those instances you mentioned in the -n discussion where Squid can be badly configured but a smart admin does not do so and we should be allowing configuration rather than denying it. It is the opposite of that: In this case, a smart admin cannot configure Squid to work well, no matter how hard she tries. NP: a smart admin may actually disable both if the network was gatewayed over software which dies on both. The clients could not contact IPv6-only servers, but that is the smart admins intent. Fail-fast on IPv6 rather than trying slow-ish things and killing the gateway router on the way. The use case is different: IPv6 servers work fine. IPv4 servers work fine with standard FTP commands. Transactions stall when Squid attempts to use EPSV with some IPv4 servers. To avoid that breakage, the admin needs a way to tell Squid do not try EPSV with IPv4 servers while still using EPSV with IPv6 servers. If ftp_epsv is on (default): contacting an IPv4 server may break because the network breaks EPSV. If ftp_epsv is off: contacting IPv6 servers breaks because they need EPSV. (I am omitting the cases of ESPV ALL and EPRT for clarity, they do not add value to the current discussion). They *do* add value. Since EPRT may be working perfectly and nullify your second example of when ftp_epsv off. To make sure we are making progress, please do not through in other use cases until we agree on how cover the current one. In the current use case, EPRT setting is irrelevant because transaction stalls before Squid tries to use EPRT (and may stall even if Squid does try to use EPRT, but I do not really know). You argued IPv6 servers breaks against disabling *only* EPSV and letting the overall algorithm complete the whole sequence. I provided the case why only-EPSV disabling is required for the sequence to work by reaching a later EPRT, proving that a false assertion. may break sure, but not always. And certainly not in the common case of a working IPv6 server set to a active-FTP because the NAT gateway has problems with EPSV. To avoid misunderstanding, I do agree that any change should not make other important use cases [much] worse, at least not without a discussion about trade-offs. So other use cases are important. I just want to make sure we find a solution for my use case first since it appears to be surprisingly difficult even without adding more use cases into the mix. Your single failure use-case is more important than keeping working the multiple cases which work today? no. We had a handful of complaints about EPSV crashing Cisco NAT routers regardless of IPv4/IPv6. So far one (you) with this new case. Which seems more common/important? Also note that by break you mean: IPv6 connectivity to IPv6-only servers over a faulty router is not possible. I do not mean that. IPv6 servers work fine if they are sent an EPSV command. A faulty network (outside of the admin control) breaks IPv4 servers that support EPSV, tricking Squid into sending EPSV to them. If anything breaks on the server itself or the pathway there the method of connection is broken and cannot be used. Normally we have a direct connection to FTP servers with any middleware *supposed* to be transparent. So, what can we possibly do that would not make that broken? IPv4 connectivity to IPv4-enabled servers is still possible. Which is the scope of your change, yes? to selectively disable the EPSV command being sent depending on IPv4 or IPv6. Yes, treating IPv4 and IPv6 servers differently is at the root of all my proposals on this thread. I am in agreement with doing it. I am in disagreement with a) exploding the config options into a large set, and/or b) dropping the existing algorithm which has been working for other breakage cases just to fix this one case. I suspect the initial attempt-based algorithm was developed for PASV and PORT only, where it makes sense. It was wrong to add similar EPSV and EPRT configuration to that IPv4-specific algorithm because a single ftp_epsv option setting does work for a mix of IPv4 and IPv6 servers. The algorithm is baked into RFC 2428. See section 2 about the meaning of that 1-digit net-prt
Re: [RFC] ignore ftp_epsv off for IPv6
On 01/29/2014 03:19 PM, Amos Jeffries wrote: [if you don't want the point-by-point skip to the end ] I skipped discussion of other use cases. I want to focus on my simple use case before considering others (and no, I did not say my case is more important than others; just that I want to focus on it first. If we cannot address my case, I will let others deal with other cases they care about). Perhapse we should go back to the reason you are proposing this RFC. How exactly is the breakage appearing in the case of EPSV both ctrl channel details and TCP actions? AFAIK, Squid sends EPSV to an IPv4 origin server. The server responds with an IPv4 address. Squid tries to connect to that address. A broken firewall in front of the origin server stalls that data connection. Users get upset. Without Squid, everything works because the client sends PASV command that the firewall understands and allows the data connection through. AFAIK, this happens with several FTP servers around the world, not just some unlucky one. The data connection to an FTP server is not necessarily going to be the same protocol as the ctrl channel connection due to dual-stack servers in IPv6. The use case under consideration assumes the same IP address for both data and control connections. This is then assuming that: 1) EPSV got through to the server, and 2) the server reply got back to Squid with protocol:IP:port details, and 3) breakage occurs when opening a normal TCP connection to that IP:port Yes. And all addresses are IPv4. Details on how this is possible without also assuming misconfiguration of the FTP servers gateway would be useful. Note that the CTRL channel was setup in the first place by opening just such a TCP connection through apparently the same pathway. See above. The control channel works fine because the firewall allows control connections to the FTP server port. The data connection is not going to the same port, of course. The firewall is supposed to let that data connection in based on the previous EPSV response, but, evidently, there is a bug with handling/recognizing that response and so the data connection stalls. Are you comfortable with any of my proposals for that specific use case, as a starting point? Here is a brief summary: P1: ignore ftp_epsv off for IPv6 servers. No. That breaks the already fixed group of breakage cases. Plenty of FTP servers are listening on and operating on IPv6 and configured only to accept data connections via IPv4. How will ignoring ftp_epsv off for IPv6 servers break FTP servers are listening on and operating on IPv6 and configured only to accept data connections via IPv4? AFAICT, those servers will receive an EPSV command, respond with an IPv4 address, and the data will flow over an IPv4 connection, just like today. No change for those servers! Confusing the admin with: 'off' does not mean OFF Agreed (hence the renaming proposals). P2: add ftp_epsv ipv6 that will disable EPSV use with IPv4 servers. P3: rename ftp_epsv to ftp_epsv_for_ipv4 (essentially) and naturally ignore the renamed version for IPv6 servers. Take a longer view. We will face matching breakage cases with all the commands we implement. Using this approach will explode the currently 4 directives out to 12 on/off settings when the other currently cases are dealt with. I want something simpler and more intuitive for admin than directives_documented_with_a_sentence_about_what_is_does_in_its_own_name on/off I would certainly welcome simpler and more intuitive alternatives! Simple: ... leave the (working) attempt-based algorithm from RFC 2428 P1: extend the incomplete ftp_epsv squid.conf setting to configure per-protocol skipping of EPSV 1|2 Can you detail that a little please? I do not understand what you are proposing. What is protocol and skipping in this context, exactly? It may help if you can give a configuration example addressing the use case in question. Thank you, Alex.
Re: [RFC] ignore ftp_epsv off for IPv6
On 30/01/2014 1:44 p.m., Alex Rousskov wrote: On 01/29/2014 03:19 PM, Amos Jeffries wrote: [if you don't want the point-by-point skip to the end ] I skipped discussion of other use cases. I want to focus on my simple use case before considering others (and no, I did not say my case is more important than others; just that I want to focus on it first. If we cannot address my case, I will let others deal with other cases they care about). Perhapse we should go back to the reason you are proposing this RFC. How exactly is the breakage appearing in the case of EPSV both ctrl channel details and TCP actions? AFAIK, Squid sends EPSV to an IPv4 origin server. The server responds with an IPv4 address. Squid tries to connect to that address. A broken firewall in front of the origin server stalls that data connection. Users get upset. Without Squid, everything works because the client sends PASV command that the firewall understands and allows the data connection through. AFAIK, this happens with several FTP servers around the world, not just some unlucky one. The data connection to an FTP server is not necessarily going to be the same protocol as the ctrl channel connection due to dual-stack servers in IPv6. The use case under consideration assumes the same IP address for both data and control connections. This is then assuming that: 1) EPSV got through to the server, and 2) the server reply got back to Squid with protocol:IP:port details, and 3) breakage occurs when opening a normal TCP connection to that IP:port Yes. And all addresses are IPv4. Details on how this is possible without also assuming misconfiguration of the FTP servers gateway would be useful. Note that the CTRL channel was setup in the first place by opening just such a TCP connection through apparently the same pathway. See above. The control channel works fine because the firewall allows control connections to the FTP server port. The data connection is not going to the same port, of course. The firewall is supposed to let that data connection in based on the previous EPSV response, but, evidently, there is a bug with handling/recognizing that response and so the data connection stalls. Why do you see any difference in this broken firewall software for IPv4 or IPv6 based A) The same bug is very likely to exist in IPv6 traffic handling on the same broken firewall. B) EPSV is not the only method available to Squid. Ergo, if EPSV is known not to work. Skip it entirely and let those other mechanisms have a shot. Are you comfortable with any of my proposals for that specific use case, as a starting point? Here is a brief summary: P1: ignore ftp_epsv off for IPv6 servers. No. That breaks the already fixed group of breakage cases. Plenty of FTP servers are listening on and operating on IPv6 and configured only to accept data connections via IPv4. How will ignoring ftp_epsv off for IPv6 servers break FTP servers are listening on and operating on IPv6 and configured only to accept data connections via IPv4? AFAICT, those servers will receive an EPSV command, respond with an IPv4 address, and the data will flow over an IPv4 connection, just like today. No change for those servers! Checking the code I see we already explicitly break them by not allowing EPSV 1 to be sent in response to a 522 message with (1) hint from IPv6 server (and vice-versa on (2) from IPv4 server). What I was meaning was this nasty beast: Squid requests EPSV 2 (IPv6 data connection), server responds 522 ... (1) indicating only accepting IPv4 data connections. ftp_epsv off disabling IPv4 ... EPSV blocked to IPv6 server. Both of these are very rare IME, and I see we are already preventing Squid working in those cases. But are valid behaviour according to the RFC. The main point being that in FTP with two TCP connections to a Dual-stack server there is no simple distinction IPv4 server or IPv6 server. You have to tie the version decision to either the ctrl or data connection. I'm just saying off should mean OFF completely. More importantly perhapse these changes to IPv6 behaviour do nothing for your described problem case which is entirely in IPv4-only traffic. Confusing the admin with: 'off' does not mean OFF Agreed (hence the renaming proposals). P2: add ftp_epsv ipv6 that will disable EPSV use with IPv4 servers. P3: rename ftp_epsv to ftp_epsv_for_ipv4 (essentially) and naturally ignore the renamed version for IPv6 servers. Take a longer view. We will face matching breakage cases with all the commands we implement. Using this approach will explode the currently 4 directives out to 12 on/off settings when the other currently cases are dealt with. I want something simpler and more intuitive for admin than directives_documented_with_a_sentence_about_what_is_does_in_its_own_name on/off I would certainly welcome simpler and more intuitive
Re: [RFC] ignore ftp_epsv off for IPv6
On 01/25/2014 06:05 PM, Amos Jeffries wrote: On 25/01/2014 9:27 a.m., Alex Rousskov wrote: I propose to limit squid.conf ftp_epsv off prohibition to IPv4 FTP servers. ... Do you think it would be OK to allow the use of EPSV commands with IPv6 servers even if ftp_epsv is off? off should never be abused to mean half-off. We are having enough trouble with forwarded_for off historically meaning something other than disable XFF feature. The problem here is that the directive itself was misnamed IMO. It should have been ftp_epsv_for_ipv4 or similar. I think extending the directive to allow selective disabling with no-ipv6 or no-ipv4 values would be better. I do not like negative names so I would suggest ipv6 instead of no-ipv4. However, what should Squid do when it is talking to an IPv6 server and ftp_epsv is off or ipv4? Does it really make sense to write more code to handle that essentially misconfigured (but inherited from the old configs) case? I doubt... How about this alternative: 1. Add ftp_epsv_for_ipv4 on/off. 2. Deprecate ftp_epsv in favor of the newly added ftp_epsv_for_ipv4. 3. Treat ftp_epsv on/off as ftp_epsv_for_ipv4 on/off. This would avoid writing useless code to handle misconfigurations because it would be impossible to misconfigure Squid in this area. Thank you, Alex.
Re: [RFC] ignore ftp_epsv off for IPv6
On 29/01/2014 9:24 a.m., Alex Rousskov wrote: On 01/25/2014 06:05 PM, Amos Jeffries wrote: On 25/01/2014 9:27 a.m., Alex Rousskov wrote: I propose to limit squid.conf ftp_epsv off prohibition to IPv4 FTP servers. ... Do you think it would be OK to allow the use of EPSV commands with IPv6 servers even if ftp_epsv is off? off should never be abused to mean half-off. We are having enough trouble with forwarded_for off historically meaning something other than disable XFF feature. The problem here is that the directive itself was misnamed IMO. It should have been ftp_epsv_for_ipv4 or similar. It is named correctly for its scope of whether to send EPSV command in FTP. I think extending the directive to allow selective disabling with no-ipv6 or no-ipv4 values would be better. I do not like negative names so I would suggest ipv6 instead of no-ipv4. Fine. Up to you as implementer. Though these extensions are supposed to be protocol agnostic FTP commands and that means future addressing types will put admin who use ipv6 at a disadvantage by default-off instead of default-attempt for any new connection types we add. However, what should Squid do when it is talking to an IPv6 server and ftp_epsv is off or ipv4? Does it really make sense to write more code to handle that essentially misconfigured (but inherited from the old configs) case? I doubt... When talking to an IPv6 server with ftp_epsv off or ipv4 the expected behaviour is that Squid uses active-FTP immediately (EPRT command) without even trying the passive connection first. How about this alternative: 1. Add ftp_epsv_for_ipv4 on/off. 2. Deprecate ftp_epsv in favor of the newly added ftp_epsv_for_ipv4. 3. Treat ftp_epsv on/off as ftp_epsv_for_ipv4 on/off. This would avoid writing useless code to handle misconfigurations because it would be impossible to misconfigure Squid in this area. It is not an isolated command, but one of a set of 5 connection possibilities. The commands permutations are attempted as follows, any one command may be skipped by a matching squid.conf directive, first to succeed wins: * contacting IPv4 server EPSV ALL - skipped when ftp_passive off, ftp_epsv_all off, ftp_epsv_all ipv6 - default is skip due to broken NATs and lack of failover possibilities. EPSV port - skipped when ftp_passive off, ftp_epsv off, ftp_epsv ipv6 - default to attempt since failover is easy and IPv6 benefits outweight the RTT cost. PASV - skipped when ftp_passive off - default to attempt due to RFC requirements EPRT ipv4 - skipped when ftp_eprt off - default to attempt since failover is easy and IPv6 benefits outweight the RTT cost. PORT ipv4 Send Failed connection error to client * contacting IPv6 server EPSV ALL - skipped when ftp_passive off, ftp_epsv_all off, ftp_epsv_all ipv4 - default is skip due to broken NATs and lack of failover possibilities. EPSV port - skipped when ftp_passive off, ftp_epsv off, ftp_epsv ipv4 - default to attempt PASV - not possible. always skipped. EPRT ipv6 - skipped when ftp_eprt off - default to attempt (unless patched by RHEL to send EPSV again) PORT - not possible. always skipped. Send Failed connection error to client PS. ftp_eprt and ftp_epsv_all could do with matching updates. HTH Amos
Re: [RFC] ignore ftp_epsv off for IPv6
On 01/28/2014 09:29 PM, Amos Jeffries wrote: On 29/01/2014 9:24 a.m., Alex Rousskov wrote: On 01/25/2014 06:05 PM, Amos Jeffries wrote: off should never be abused to mean half-off. The problem here is that the directive itself was misnamed IMO. It should have been ftp_epsv_for_ipv4 or similar. It is named correctly for its scope of whether to send EPSV command in FTP. OK, then its scope is wrong. There should not be an option to enable or disable a required command; EPSV (or EPRT) are required for IPv6 servers to work. I think extending the directive to allow selective disabling with no-ipv6 or no-ipv4 values would be better. I do not like negative names so I would suggest ipv6 instead of no-ipv4. Fine. Up to you as implementer. Though these extensions are supposed to be protocol agnostic FTP commands and that means future addressing types will put admin who use ipv6 at a disadvantage by default-off instead of default-attempt for any new connection types we add. Whether default-off is more advantageous than default-attempt depends on the consequences of those attempts. Today, the consequences of default-attempt are already bad for IPv4 in many cases. However, this thread is not about the arguably wrong defaults. If future addressing types require more complex expressions, we should switch to ACLs that can express both positive and negative conditions equally well. However, what should Squid do when it is talking to an IPv6 server and ftp_epsv is off or ipv4? Does it really make sense to write more code to handle that essentially misconfigured (but inherited from the old configs) case? I doubt... When talking to an IPv6 server with ftp_epsv off or ipv4 the expected behaviour is that Squid uses active-FTP immediately (EPRT command) without even trying the passive connection first. I assume EPRT is similarly disabled, of course. The two options ought to be treated the same for the purpose of this discussion. How about this alternative: 1. Add ftp_epsv_for_ipv4 on/off. 2. Deprecate ftp_epsv in favor of the newly added ftp_epsv_for_ipv4. 3. Treat ftp_epsv on/off as ftp_epsv_for_ipv4 on/off. This would avoid writing useless code to handle misconfigurations because it would be impossible to misconfigure Squid in this area. It is not an isolated command, but one of a set of 5 connection possibilities. The commands permutations are attempted as follows, any one command may be skipped by a matching squid.conf directive, first to succeed wins: * contacting IPv4 server EPSV ALL - skipped when ftp_passive off, ftp_epsv_all off, ftp_epsv_all ipv6 - default is skip due to broken NATs and lack of failover possibilities. EPSV port - skipped when ftp_passive off, ftp_epsv off, ftp_epsv ipv6 - default to attempt since failover is easy and IPv6 benefits outweight the RTT cost. PASV - skipped when ftp_passive off - default to attempt due to RFC requirements EPRT ipv4 - skipped when ftp_eprt off - default to attempt since failover is easy and IPv6 benefits outweight the RTT cost. PORT ipv4 Send Failed connection error to client * contacting IPv6 server EPSV ALL - skipped when ftp_passive off, ftp_epsv_all off, ftp_epsv_all ipv4 - default is skip due to broken NATs and lack of failover possibilities. EPSV port - skipped when ftp_passive off, ftp_epsv off, ftp_epsv ipv4 - default to attempt PASV - not possible. always skipped. EPRT ipv6 - skipped when ftp_eprt off - default to attempt (unless patched by RHEL to send EPSV again) PORT - not possible. always skipped. Send Failed connection error to client The current approach does not work well in practice because, while it looks reasonable when IPv4 and IPv6 actions are documented separately, the [necessarily single] configuration breaks either IPv4 or IPv6 servers: If ftp_epsv is on (default): contacting an IPv4 server may break because the network breaks EPSV. If ftp_epsv is off: contacting IPv6 servers breaks because they need EPSV. (I am omitting the cases of ESPV ALL and EPRT for clarity, they do not add value to the current discussion). I suspect the initial attempt-based algorithm was developed for PASV and PORT only, where it makes sense. It was wrong to add similar EPSV and EPRT configuration to that IPv4-specific algorithm because a single ftp_epsv option setting does work for a mix of IPv4 and IPv6 servers. Alex.
Re: [RFC] ignore ftp_epsv off for IPv6
On 29/01/2014 6:40 p.m., Alex Rousskov wrote: On 01/28/2014 09:29 PM, Amos Jeffries wrote: On 29/01/2014 9:24 a.m., Alex Rousskov wrote: On 01/25/2014 06:05 PM, Amos Jeffries wrote: off should never be abused to mean half-off. The problem here is that the directive itself was misnamed IMO. It should have been ftp_epsv_for_ipv4 or similar. It is named correctly for its scope of whether to send EPSV command in FTP. OK, then its scope is wrong. There should not be an option to enable or disable a required command; EPSV (or EPRT) are required for IPv6 servers to work. The (or EPRT) is important. The scope is right. However, what should Squid do when it is talking to an IPv6 server and ftp_epsv is off or ipv4? Does it really make sense to write more code to handle that essentially misconfigured (but inherited from the old configs) case? I doubt... When talking to an IPv6 server with ftp_epsv off or ipv4 the expected behaviour is that Squid uses active-FTP immediately (EPRT command) without even trying the passive connection first. I assume EPRT is similarly disabled, of course. The two options ought to be treated the same for the purpose of this discussion. So this is one of those instances you mentioned in the -n discussion where Squid can be badly configured but a smart admin does not do so and we should be allowing configuration rather than denying it. NP: a smart admin may actually disable both if the network was gatewayed over software which dies on both. The clients could not contact IPv6-only servers, but that is the smart admins intent. Fail-fast on IPv6 rather than trying slow-ish things and killing the gateway router on the way. How about this alternative: 1. Add ftp_epsv_for_ipv4 on/off. 2. Deprecate ftp_epsv in favor of the newly added ftp_epsv_for_ipv4. 3. Treat ftp_epsv on/off as ftp_epsv_for_ipv4 on/off. This would avoid writing useless code to handle misconfigurations because it would be impossible to misconfigure Squid in this area. It is not an isolated command, but one of a set of 5 connection possibilities. The commands permutations are attempted as follows, any one command may be skipped by a matching squid.conf directive, first to succeed wins: * contacting IPv4 server EPSV ALL - skipped when ftp_passive off, ftp_epsv_all off, ftp_epsv_all ipv6 - default is skip due to broken NATs and lack of failover possibilities. EPSV port - skipped when ftp_passive off, ftp_epsv off, ftp_epsv ipv6 - default to attempt since failover is easy and IPv6 benefits outweight the RTT cost. PASV - skipped when ftp_passive off - default to attempt due to RFC requirements EPRT ipv4 - skipped when ftp_eprt off - default to attempt since failover is easy and IPv6 benefits outweight the RTT cost. PORT ipv4 Send Failed connection error to client * contacting IPv6 server EPSV ALL - skipped when ftp_passive off, ftp_epsv_all off, ftp_epsv_all ipv4 - default is skip due to broken NATs and lack of failover possibilities. EPSV port - skipped when ftp_passive off, ftp_epsv off, ftp_epsv ipv4 - default to attempt PASV - not possible. always skipped. EPRT ipv6 - skipped when ftp_eprt off - default to attempt (unless patched by RHEL to send EPSV again) PORT - not possible. always skipped. Send Failed connection error to client The current approach does not work well in practice because, while it looks reasonable when IPv4 and IPv6 actions are documented separately, the [necessarily single] configuration breaks either IPv4 or IPv6 servers: If ftp_epsv is on (default): contacting an IPv4 server may break because the network breaks EPSV. If ftp_epsv is off: contacting IPv6 servers breaks because they need EPSV. (I am omitting the cases of ESPV ALL and EPRT for clarity, they do not add value to the current discussion). They *do* add value. Since EPRT may be working perfectly and nullify your second example of when ftp_epsv off. Also note that by break you mean: IPv6 connectivity to IPv6-only servers over a faulty router is not possible. So, what can we possibly do that would not make that broken? IPv4 connectivity to IPv4-enabled servers is still possible. Which is the scope of your change, yes? to selectively disable the EPSV command being sent depending on IPv4 or IPv6. I suspect the initial attempt-based algorithm was developed for PASV and PORT only, where it makes sense. It was wrong to add similar EPSV and EPRT configuration to that IPv4-specific algorithm because a single ftp_epsv option setting does work for a mix of IPv4 and IPv6 servers. The algorithm is baked into RFC 2428. See section 2 about the meaning of that 1-digit net-prt parameteron EPSV; 1 = IPv4 wanted, 2 = IPv6. The server accepts or rejects with 522 and a list of supported protocols which can be tried on the followup attempt. Only
Re: [RFC] ignore ftp_epsv off for IPv6
On 25/01/2014 9:27 a.m., Alex Rousskov wrote: Hello, I propose to limit squid.conf ftp_epsv off prohibition to IPv4 FTP servers. Setting ftp_epsv to off is often necessary to correctly handle real-world cases where an IPv4 FTP server correctly responds to an EPSV command but is located behind a firewall that does not understand EPSV responses and, hence, does not allow the subsequent data connection through. This combination forces Squid admins to turn ftp_epsv off. However, turning ftp_epsv off to handle a few broken IPv4 FTP servers immediately breaks *all* IPv6 FTP servers because EPSV is required for any IPv6 FTP server to exchange data. The old PASV command is not flexible enough to serve IPv6 needs. See RFC 2428 for details. Since using EPSV with IPv6 servers cannot make matters worse and will make them better in many cases, I suggest ignoring ftp_epsv off when Squid has to talk to an IPv6 FTP server. Do you think it would be OK to allow the use of EPSV commands with IPv6 servers even if ftp_epsv is off? off should never be abused to mean half-off. We are having enough trouble with forwarded_for off historically meaning something other than disable XFF feature. I think extending the directive to allow selective disabling with no-ipv6 or no-ipv4 values would be better. Amos
[RFC] ignore ftp_epsv off for IPv6
Hello, I propose to limit squid.conf ftp_epsv off prohibition to IPv4 FTP servers. Setting ftp_epsv to off is often necessary to correctly handle real-world cases where an IPv4 FTP server correctly responds to an EPSV command but is located behind a firewall that does not understand EPSV responses and, hence, does not allow the subsequent data connection through. This combination forces Squid admins to turn ftp_epsv off. However, turning ftp_epsv off to handle a few broken IPv4 FTP servers immediately breaks *all* IPv6 FTP servers because EPSV is required for any IPv6 FTP server to exchange data. The old PASV command is not flexible enough to serve IPv6 needs. See RFC 2428 for details. Since using EPSV with IPv6 servers cannot make matters worse and will make them better in many cases, I suggest ignoring ftp_epsv off when Squid has to talk to an IPv6 FTP server. Do you think it would be OK to allow the use of EPSV commands with IPv6 servers even if ftp_epsv is off? Thank you, Alex. P.S. This problem was discovered in early Native FTP proxy feature deployments, but it will be relevant to the FTP gateway code as well. P.P.S. The same logic would apply to EPRT, I guess.
