Re: [squid-dev] Alternate origin server selection
On 10/29/21 9:57 AM, Steve Hill wrote: > Ok, I've gone back and looked over my old debug logs. It appears what > was actually happening was: > > - Client sends "CONNECT www.google.com:443". > - Connection with TLS made to forcesafesearch.google.com. > - Client sends "GET / HTTP/1.1\r\nHost: www.google.com" > - Squid runs the peer selector to find peers for www.google.com (i.e. > the host contained in the GET request). > - It finds the appropriate pinned connection: > client_side.cc(3872) borrowPinnedConnection: conn28 > local=81.187.83.66:52488 remote=216.239.38.120:443 HIER_DIRECT FD 18 > flags=1 > - Squid then logs: > FwdState.cc(472) fail: ERR_ZERO_SIZE_OBJECT "Bad Gateway" > https://www.google.com/ > FwdState.cc(484) fail: pconn race happened > FwdState.cc(494) fail: zero reply on pinned connection The above looks like a persistent connection race, way after SslBump bumped the connections. This kind of races seem unrelated to everything we have discussed on this thread so far, but I may be missing something. > Unfortunately, I cannot reproduce this problem now. Well, if you want to reproduce it, you probably can do that using a custom origin server, but I do not see the point: There is nothing particularly "wrong" in the above sequence AFAICT; races happen. However, it is possible that the above log is lying or misleading, and there was actually a different problem, a problem related to previous discussions, but it manifested itself as ERR_ZERO_SIZE_OBJECT "pconn race happened" diagnostic. > I can remove the unpinning code and submit a new pull request, which now > works ok for me. But I'm very wary that I did originally have problems > with that which I can no longer reproduce. Unfortunately, I did not have a chance to study the rejected pull request before it was rejected, so I cannot advise you on whether subtracting something from that pull request will result in changes that should be, in principle, accepted. I suggest to imagine that the rejected pull request did not exist (instead of using it as a starting/reference point) and then describe the problem and the proposed solution from scratch. You can do that here or via a new pull request. The best avenue may depend on whether your code changes alter some key Squid principles. If they do, it might be best to get an agreement on those changes here, but GitHub discussions have better access to code and better formatting abilities. Cheers, Alex. ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] Alternate origin server selection
On 10/28/21 12:39 PM, Steve Hill wrote: > On 28/10/2021 16:41, Alex Rousskov wrote: >> AFAICT, the primary obstacle here is that Squid pins the connection >> while obtaining the origin server certificate. > Well, I can't see why Squid needs the origin certificate - it should be > able to make a decision off the SNI before connecting to the origin server. Squid does not "need" any of this, of course. Configuration and/or bugs force Squid to do what it does. If your decision-making process does not involve the certificate, then you should be able to rewrite the fake CONNECT request during SslBump step2, without (or before) telling Squid to stare at the certificate (and pin the resulting connection). There are bugs in this area, including bugs that may prevent certain CONNECT adaptations from happening. We are fixing one of those bugs right now. For details, you can see an unpolished/unofficial pull request at https://github.com/measurement-factory/squid/pull/108 > I didn't seem to be able to make the decision prior to the connection > being pinned though. I'm not sure why - I will go back and investigate > further. Sounds like a plan! Cheers, Alex. ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] Alternate origin server selection
On 10/28/21 9:24 AM, Steve Hill wrote: > For transparently proxied traffic, the client makes a connection to > www.google.com's IP address, which Squid intercepts. Squid must then > SSL-peek the request to figure out that it is connecting to > www.google.com. The onward connection can then be redirected to the > virtual IP. IIRC, Google has recommended (to my surprise) something like that as well, for environments where DNS modifications are inappropriate and bumping is possible. I cannot find that recommendation now, unfortunately. > There is code to do this: > https://github.com/squid-cache/squid/pull/924 > This allows an external ACL to record an alt-host note, or an ICAP > server to return an X-Alt-Host header, specifying a new origin server to > connect to. > > The pull request was rejected, as it adds CVE-2009-0801 vulnerabilities. > > I'm hoping for some guidance on the best way to achieve this. While I disagree with some of the assertions made in that PR review and the unilateral approach to closing that PR, I hope we can find a positive way forward here. Your use case deserves Squid support IMO. AFAICT, the primary obstacle here is that Squid pins the connection while obtaining the origin server certificate. Please confirm that without that origin server certificate you cannot make the decision whether to redirect the HTTP request. In other words, the client-intended destination IP address and the client-supplied SNI are not sufficient to correctly determine whether the connection must be bumped (in relevant cases). Also, if you need the server certificate indeed, please confirm that when the origin server uses TLS v1.3, the proposed scheme will have to fully bump the Squid-server connection before redirecting the request. Just staring at the TLS ServerHello will not be enough because the origin certificate comes later, in the encrypted records. I hope to suggest the right solution based on your feedback. Thank you, Alex. ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
