Re: [squid-users] intermittent TCP_MISS on file specified in refresh_pattern
On 01.11.2012 04:46, Mike Pentecost wrote: We are running Squid 3.1 on Debian Squeeze. We are using squid as a reverse proxy/cache for a Django backend. The cache is working well, but there is one file that keeps getting by. It has a "?" in its URL, which is needed because it has a license key parameter in it. I put a refresh pattern to try to catch it, but it is not cached in a consistent manner. It returns a HIT maybe 50% of the time, sometimes requests seconds apart will show different results. You mean the URL has parameters which are not shown in your log? Or do you mean that you append '?' without anything following to the URL in order to make the network infrastructure treat it as dynamic content? (default for dynamic content in a lot of places is not to cache, or not for long) The difference is important. It could be correct behaviour, or not. * When a URL parameter changes a single byte it is a whole different URL. MISS is expected if any URL is not cached already. versus * Depending on URL octets to determine traffic caching behaviour is a major FAIL. - Squid's old behaviour of not caching URLs with '?' was solely due to a default config workaround for old broken CGI scripts, which has never been followed by many non-Squid caches, and is no longer followed by Squid-3 either. - The only thing you can rely on is the above detail about URLs with different exact-string values being considered different URLs by HTTP-compliant caches. Here is the refresh pattern that I was hoping would catch it, this is above any other patterns (this is a static file, and we want squid to cache it for at least an hour): refresh_pattern -i http://foo.bar/static/floatbox/options.js? 60 100%60 override-expire override-lastmod That is supposed to be a regular expression pattern. '\.' and '\?' are required to match '.' and '?' characters in the input value. Here are some logs showing the weird caching: 62.189.22.21 [31/Oct/2012:06:15:29 -0700] "GET http://foo.bar/static/floatbox/options.js? HTTP/1.1" 200 TCP_MEM_HIT:NONE 208.101.141.24 [31/Oct/2012:06:15:55 -0700] "GET http://foo.bar/static/floatbox/options.js? HTTP/1.1" 200 TCP_MISS:FIRST_UP_PARENT I was hoping it was a staleness issue, and setting the override-expire and lastmod options would help enforce the min/max in the refresh pattern. I'm sure I have missed something. You mentioned there is a license key transferred. In which case you absolutely do not want to override those two cache controls. Occasional unnecessary checks with the backend are better than leaving obsolete security/license keys responding with 'allow' type actions. Speaking of headers, what *are* the response headers being produced by the backend server for Squid to work with? Amos
Re: [squid-users] RE: : [squid-users] Squid and SSL interception (ssl-bump)
Date: Thu, 1 Nov 2012 00:59:15 +0800 From: ammdispose-squid -- On Wed 31 Oct, 2012 9:03 PM IST Heinrich Hirtzel wrote: >http_port 10.0.1.1.:3128 intercept >https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm > you have forgotten intercept on https line Amm On 01.11.2012 06:07, Heinrich Hirtzel wrote: you have forgotten intercept on https line Amm Already tired that before, doesn't work either. You are intercepting packets. That option is required on the port. - instructs Squid to lookup the NAT table and find TCP details from before REDIRECT erased things in the packet. - instructs Squid the URL is a partial and to lookup the Host: HTTP header to find the domain;port details. Any errors after adding intercept flag properly will be due to some other problem than URL "/" being received. Which is the only problem you have described so far (other than "dont work", "doesnt work either" which I assume are all the same error message appearing due to URL parsing being one of the very first things Squid does for new traffic). Amos
RE: [squid-users] Squid and SSL interception (ssl-bump)
On 01.11.2012 04:49, Heinrich Hirtzel wrote: Hi Eliezer what iptables rules have you used? also you better use squid 3.2 for ssl-bump. iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 81 -j REDIRECT --to-port 3128 iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 443 -j REDIRECT --to-port 443 also you better use squid 3.2 for ssl-bump. K, will try that. Stay tuned :-) take a look at: http://wiki.squid-cache.org/Features/SslBump and http://wiki.squid-cache.org/Features/DynamicSslCert I've read through them for at least 10 times (I'm not kidding) and tried various different configurations without finding any solution. Maybe I simply missed something :-/ Do I need to compile squid with '--enable-ssl-crtd' or is '--enable-ssl' enough? For HTTPS interception ssl-crtd is better. server-first feature and certificate-mimic are even better. Squid-3.3 which has these is needed for anything close to useful HTTPS port 443 interception. Amos
Re: [squid-users] Squid and SSL interception (ssl-bump)
On 10/31/2012 6:43 PM, Heinrich Hirtzel wrote: I've just updated to the most recent squid version (compiled with --enable-ssl *and* --enable-ssl-crtd): ./squid -v Squid Cache: Version 3.2.3-20121031-r11695 what are all the compilation options from -v (curios) try the latest stable at: http://www.squid-cache.org/Versions/v3/3.2/squid-3.2.3.tar.bz2 I can start squid without any errors, but when I try to connect to the daemon the connection gets terminated immediately (using the same cfg as before): $ telnet 10.0.1.1 3128 Trying 10.0.1.1... Connected to 10.0.1.1. Escape character is '^]'. Connection closed by foreign host. While cache.log shows: kid1| BUG: Orphan Comm::Connection: local=10.0.1.1:3128 remote=10.0.1.1:51288 FD 10 flags=33 kid1| NOTE: 1 Orphan since last started Unfortunately I have no idea what Squid tries to tell me In fact HTTP and HTTPS are not working now :-( Ideas? what are your config now? what about: sslcrtd_program ? as I mentioned before you use https only for reverse proxy.(3.2) did you took your time to look at "Limitations" section in http://wiki.squid-cache.org/Features/DynamicSslCert ? if you need to intercept all https connections(seems to be your case) I would suggest you to try use the 3.3 beta. Regards, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il
Re: [squid-users] Squid and SSL interception (ssl-bump)
On 01.11.2012 04:33, Heinrich Hirtzel wrote: Hello For a school project I'm trying to intercept SSL connections by using Squid (client -> squid (transparent) -> server). I'm running Squid 3.1.20 on Ubuntu server 12.10 (64 bit) using the following configuration: * http_port 10.0.1.1.:3128 intercept https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm acl our_networks src 10.0.1.0/24 http_access allow our_networks forwarded_for off ssl_bump allow all sslproxy_cert_error allow all sslproxy_flags DONT_VERIFY_PEER * I've complied squid with SSL support (--enable-ssl). When starting Squid I do not get any error message. Also, proxying http traffic works without any problems. However, when I try to establish a HTTPS session through squid, the client retrieves the SSL certificate from squid, but after accepting it the browser displays an error message from squid that the URL is invalid: "The following error was encountered while trying to retrieve the URL: /. Invalid URL" In the Squid access.log I see the following line: " 0 10.0.1.5 NONE/440 3503 GET / - NONE/- text/html" It appears that squid does strips away the hostname / domain name of the URL the client tries to access, which causes the error message mentioned above. I've already spent hours in finding a solution for this problem and went through dozens of tutorials, unfortunately I wasn't able to find a solution so far. Any ideas what could be wrong? You are missing the intercept flag on https_port. That is what tells Squid how to interpret the URL and TCP layer differences in the port 80 and 443 syntax traffic. Amos
Re: [squid-users] Reverse Proxy for Virtual Hosts
On 01.11.2012 10:48, Karl Hill wrote: Hello, I'm having some issues setting up a reverse proxy in my environment for some URL's running on the same Apache server. I'm using squid 3.1.10. My configuration is below, but basically I have two sites on one Apache server and that server is in an isolated environment. The Apache sites are vhost1 and vhost2.example.com, but I need to get to them via the outside.example.com domain. So effectively: client -> vhost1.outside.example.com -> squid -> vhost1.example.com -> apache server for both. The problem is that only the vhost1 is working. When I go to vhost2.outside.example.com I go to vhost1.example.com. Any help would be appreciated. Virtual HTTPS hosts have not had much use in Squid and thus not much testing or feature development. The main problem as I understand it is that the certificate is a static cert for only one of the domains, possibly a wildcard cert can make it kind of useful for sub-domains. The 3.1 and older Squid also do not support the TLS SNS feature necessary for proxies to identify the virtual domain FQDN up front. Squid-3.2 and later provide a certificate generation feature that it should be very useful for virtual hosting with HTTPS. Are you able to upgrade and test it out? There are a few wrinkles due to it being quite new and not much tested in reverse-proxy, but if it works for you it would be good to know that - or any issues you can find. Amos
[squid-users] Reverse Proxy for Virtual Hosts
Hello, I'm having some issues setting up a reverse proxy in my environment for some URL's running on the same Apache server. I'm using squid 3.1.10. My configuration is below, but basically I have two sites on one Apache server and that server is in an isolated environment. The Apache sites are vhost1 and vhost2.example.com, but I need to get to them via the outside.example.com domain. So effectively: client -> vhost1.outside.example.com -> squid -> vhost1.example.com -> apache server for both. The problem is that only the vhost1 is working. When I go to vhost2.outside.example.com I go to vhost1.example.com. Any help would be appreciated. https_port 443 cert=/etc/pki/tls/CertAuth/testcert.cert key=/etc/pki/tls/CertAuth/testkey.pem vhost cache_peer vhost1.example.com parent 443 0 no-query originserver ssl sslflags=DONT_VERIFY_PEER name=vsite1 cache_peer vhost2.example.com parent 443 0 no-query originserver ssl sslflags=DONT_VERIFY_PEER login=PASS name=vsite2 cache_peer host3.example.com parent 443 0 no-query originserver ssl sslflags=DONT_VERIFY_PEER name=site3 acl d_vhost1dstdomain vhost1.outside.example.com acl d_vhost2dstdomain vhost2.outside.example.com acl d_host3 dstdomain host3.outside.example.com http_access allow d_vhost1 http_access allow d_vhost2 http_access allow d_host3 cache_peer_access vsite1 allow d_vhost1 cache_peer_access vsite1 deny all cache_peer_access vsite2 allow d_vhost2 cache_peer_access vsite2 deny all cache_peer_access site3 allow d_host3 cache_peer_access site3 deny all http_access deny all
[squid-users] Re: No Kerberos Auth
Hi Ralph, If you use NTLM and Kerberos make sure you do NOT use the sam AD account for both. The samba daemon will change the password on a regular basis which will bring the keytab out of sync with the AD acccount. Your proxy will not need any kerberos cache (except if you use my squid_kerb_ldap module but it is not the root user cache as you show below). Markus "Jarosch, Ralph" wrote in message news:c644cb972edfa3488cfd140b498136231b5e9...@justizcembx14.justiz.niedersachsen.de... I've found this today. why is the last ticket not renewed ?? Could that point the problem [root@http-proxy ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de Valid starting ExpiresService principal 10/30/12 14:47:38 10/31/12 00:47:37 krbtgt/justiz.niedersachsen...@justiz.niedersachsen.de renew until 10/31/12 14:47:38 10/30/12 15:24:49 10/31/12 00:47:37 ldap/justizhadc01.justiz.niedersachsen...@justiz.niedersachsen.de renew until 10/31/12 14:47:38 10/30/12 15:24:49 10/30/12 15:26:49 kadmin/chang...@justiz.niedersachsen.de renew until 10/30/12 15:26:49 -Ursprüngliche Nachricht- Von: Jarosch, Ralph [mailto:ralph.jaro...@justiz.niedersachsen.de] Gesendet: Dienstag, 30. Oktober 2012 15:27 An: Bastien Ceriani Cc: squid-users@squid-cache.org Betreff: AW: [squid-users] No Kerberos Auth I think encrypte Type is already 28. This is the output with -- encrypt 28 -- ldap_set_supportedEncryptionTypes: No need to change msDs-supportedEncryptionTypes they are 28 Von: Jarosch, Ralph Gesendet: Dienstag, 30. Oktober 2012 15:24 An: 'Bastien Ceriani' Cc: squid-users@squid-cache.org Betreff: AW: [squid-users] No Kerberos Auth Oh ok.. yes it work fine until ten minute i wrote the mail. There it crashed from one minute to the other I'am just troubleshoot the problem.. Von: Bastien Ceriani [mailto:bastien.ceri...@bulkypix.com] Gesendet: Dienstag, 30. Oktober 2012 15:16 An: Jarosch, Ralph Cc: squid-users@squid-cache.org Betreff: Re: [squid-users] No Kerberos Auth Ok Thx, With Windows Server 2008 you should use --enctypes 28 parameter with msktutils command. Did your ntlm authentification work fine ? How did you configure it ? With Samba/Winbind ? On Tue, Oct 30, 2012 at 3:08 PM, Jarosch, Ralph wrote: OK for wireshark i must wait for tonight because no one here can work If enable authentication My keytab Keytab name: WRFILE:/etc/squid/HTTP.keytab KVNO Timestamp Principal - 6 10/30/12 09:47:42 http-proxy$@JUSTIZ.NIEDERSACHSEN.DE (arcfour-hmac) 6 10/30/12 09:47:42 http-proxy$@JUSTIZ.NIEDERSACHSEN.DE (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 http-proxy$@JUSTIZ.NIEDERSACHSEN.DE (aes256-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de (arcfour-hmac) 6 10/30/12 09:47:42 HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de (aes256-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-pr...@justiz.niedersachsen.de (arcfour-hmac) 6 10/30/12 09:47:42 HTTP/http-pr...@justiz.niedersachsen.de (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-pr...@justiz.niedersachsen.de (aes256-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HOST/http-pr...@justiz.niedersachsen.de (arcfour-hmac) 6 10/30/12 09:47:42 HOST/http-pr...@justiz.niedersachsen.de (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HOST/http-pr...@justiz.niedersachsen.de (aes256-cts-hmac-sha1-96) My Squid.conf auth_param negotiate program /usr/lib64/squid/squid_kerb_auth -d -i -s HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de auth_param negotiate children 100 auth_param negotiate keep_alive on auth_param ntlm keep_alive on auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp auth_param ntlm children 200 #auth_param ntlm max_challenge_reuses 0 #auth_param ntlm max_challenge_lifetime 2 minutes auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic auth_param basic children 200 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 5 hours and my msktutil msktutil -c -b "OU=Sonstige Server,OU=Globale Dienste,DC=justiz,DC=niedersachsen,DC=de" -s HTTP/http-proxy.justiz.niedersachsen.de -h http-proxy.justiz.niedersachsen.de -k /etc/HTTP.keytab --computer-name http-proxy --upn HTTP/http-proxy.justiz.niedersachsen.de --server justizhadc01.justiz.niedersachsen.de --verbose We use Windows 2008 R2 Server Von: Bastien Ceriani [mailto:bastien.ceri...@bulkypix.com] Gesendet: Dienstag, 30. Oktober 2012 15:00 An: Jarosch, Ralph Betreff: Re: [squid-users] No Kerberos Auth I'm in the same case.. Try to check kerberos TGS REQ and TGS REP with wireshark ? Can you display : - your
[squid-users] RE: : [squid-users] Squid and SSL interception (ssl-bump)
> you have forgotten intercept on https line > > Amm Already tired that before, doesn't work either. > Date: Thu, 1 Nov 2012 00:59:15 +0800 > From: ammdispose-sq...@yahoo.com > To: heinrichhirtze...@hotmail.com; squid-users@squid-cache.org > Subject: Re:: [squid-users] Squid and SSL interception (ssl-bump) > > > > > > > -- > On Wed 31 Oct, 2012 9:03 PM IST Heinrich Hirtzel wrote: > > > >http_port 10.0.1.1.:3128 intercept > >https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm > > > > you have forgotten intercept on https line > > Amm
Re:: [squid-users] Squid and SSL interception (ssl-bump)
-- On Wed 31 Oct, 2012 9:03 PM IST Heinrich Hirtzel wrote: >http_port 10.0.1.1.:3128 intercept >https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm > you have forgotten intercept on https line Amm
RE: [squid-users] Squid and SSL interception (ssl-bump)
I've just updated to the most recent squid version (compiled with --enable-ssl *and* --enable-ssl-crtd): ./squid -v Squid Cache: Version 3.2.3-20121031-r11695 I can start squid without any errors, but when I try to connect to the daemon the connection gets terminated immediately (using the same cfg as before): $ telnet 10.0.1.1 3128 Trying 10.0.1.1... Connected to 10.0.1.1. Escape character is '^]'. Connection closed by foreign host. While cache.log shows: kid1| BUG: Orphan Comm::Connection: local=10.0.1.1:3128 remote=10.0.1.1:51288 FD 10 flags=33 kid1| NOTE: 1 Orphan since last started Unfortunately I have no idea what Squid tries to tell me In fact HTTP and HTTPS are not working now :-( Ideas? > From: heinrichhirtze...@hotmail.com > To: squid-users@squid-cache.org > Date: Wed, 31 Oct 2012 16:49:37 +0100 > Subject: RE: [squid-users] Squid and SSL interception (ssl-bump) > > > Hi Eliezer > > > what iptables rules have you used? > > also you better use squid 3.2 for ssl-bump. > iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 81 -j REDIRECT --to-port > 3128 > iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 443 -j REDIRECT > --to-port 443 > > > also you better use squid 3.2 for ssl-bump. > K, will try that. Stay tuned :-) > > > take a look at: > > http://wiki.squid-cache.org/Features/SslBump > > and > > http://wiki.squid-cache.org/Features/DynamicSslCert > > I've read through them for at least 10 times (I'm not kidding) and tried > various different configurations without finding any solution. Maybe I simply > missed something :-/ > > Do I need to compile squid with '--enable-ssl-crtd' or is '--enable-ssl' > enough? > > Regards, > Heinrich > > > > Date: Wed, 31 Oct 2012 17:40:38 +0200 > > From: elie...@ngtech.co.il > > To: squid-users@squid-cache.org > > Subject: Re: [squid-users] Squid and SSL interception (ssl-bump) > > > > On 10/31/2012 5:33 PM, Heinrich Hirtzel wrote: > > > Hello > > > > > > For a school project I'm trying to intercept SSL connections by using > > > Squid (client -> squid (transparent) -> server). > > > I'm running Squid 3.1.20 on Ubuntu server 12.10 (64 bit) using the > > > following configuration: > > > > > > * > > > http_port 10.0.1.1.:3128 intercept > > > https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm > > If i remeber right you shoudl use http and not https > > > > > > > > acl our_networks src 10.0.1.0/24 > > > http_access allow our_networks > > > forwarded_for off > > > ssl_bump allow all > > > sslproxy_cert_error allow all > > > sslproxy_flags DONT_VERIFY_PEER > > > * > > what iptables rules have you used? > > also you better use squid 3.2 for ssl-bump. > > > > what were you reading about ssl-bump? > > > > take a look at: > > http://wiki.squid-cache.org/Features/SslBump > > and > > http://wiki.squid-cache.org/Features/DynamicSslCert > > > > Regards, > > Eliezer > > > > -- > > Eliezer Croitoru > > https://www1.ngtech.co.il > > IT consulting for Nonprofit organizations > > eliezer ngtech.co.il >
RE: [squid-users] Squid and SSL interception (ssl-bump)
Hi Eliezer > what iptables rules have you used? > also you better use squid 3.2 for ssl-bump. iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 81 -j REDIRECT --to-port 3128 iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 443 -j REDIRECT --to-port 443 > also you better use squid 3.2 for ssl-bump. K, will try that. Stay tuned :-) > take a look at: > http://wiki.squid-cache.org/Features/SslBump > and > http://wiki.squid-cache.org/Features/DynamicSslCert I've read through them for at least 10 times (I'm not kidding) and tried various different configurations without finding any solution. Maybe I simply missed something :-/ Do I need to compile squid with '--enable-ssl-crtd' or is '--enable-ssl' enough? Regards, Heinrich > Date: Wed, 31 Oct 2012 17:40:38 +0200 > From: elie...@ngtech.co.il > To: squid-users@squid-cache.org > Subject: Re: [squid-users] Squid and SSL interception (ssl-bump) > > On 10/31/2012 5:33 PM, Heinrich Hirtzel wrote: > > Hello > > > > For a school project I'm trying to intercept SSL connections by using Squid > > (client -> squid (transparent) -> server). > > I'm running Squid 3.1.20 on Ubuntu server 12.10 (64 bit) using the > > following configuration: > > > > * > > http_port 10.0.1.1.:3128 intercept > > https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm > If i remeber right you shoudl use http and not https > > > > > acl our_networks src 10.0.1.0/24 > > http_access allow our_networks > > forwarded_for off > > ssl_bump allow all > > sslproxy_cert_error allow all > > sslproxy_flags DONT_VERIFY_PEER > > * > what iptables rules have you used? > also you better use squid 3.2 for ssl-bump. > > what were you reading about ssl-bump? > > take a look at: > http://wiki.squid-cache.org/Features/SslBump > and > http://wiki.squid-cache.org/Features/DynamicSslCert > > Regards, > Eliezer > > -- > Eliezer Croitoru > https://www1.ngtech.co.il > IT consulting for Nonprofit organizations > eliezer ngtech.co.il
[squid-users] intermittent TCP_MISS on file specified in refresh_pattern
We are running Squid 3.1 on Debian Squeeze. We are using squid as a reverse proxy/cache for a Django backend. The cache is working well, but there is one file that keeps getting by. It has a "?" in its URL, which is needed because it has a license key parameter in it. I put a refresh pattern to try to catch it, but it is not cached in a consistent manner. It returns a HIT maybe 50% of the time, sometimes requests seconds apart will show different results. Here is the refresh pattern that I was hoping would catch it, this is above any other patterns (this is a static file, and we want squid to cache it for at least an hour): refresh_pattern -i http://foo.bar/static/floatbox/options.js? 60 100%60 override-expire override-lastmod Here are some logs showing the weird caching: 62.189.22.21 [31/Oct/2012:06:15:29 -0700] "GET http://foo.bar/static/floatbox/options.js? HTTP/1.1" 200 TCP_MEM_HIT:NONE 208.101.141.24 [31/Oct/2012:06:15:55 -0700] "GET http://foo.bar/static/floatbox/options.js? HTTP/1.1" 200 TCP_MISS:FIRST_UP_PARENT I was hoping it was a staleness issue, and setting the override-expire and lastmod options would help enforce the min/max in the refresh pattern. I'm sure I have missed something. Thanks in advance for any help. - Michael Penetcost
Re: [squid-users] Squid and SSL interception (ssl-bump)
On 10/31/2012 5:33 PM, Heinrich Hirtzel wrote: Hello For a school project I'm trying to intercept SSL connections by using Squid (client -> squid (transparent) -> server). I'm running Squid 3.1.20 on Ubuntu server 12.10 (64 bit) using the following configuration: * http_port 10.0.1.1.:3128 intercept https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm If i remeber right you shoudl use http and not https acl our_networks src 10.0.1.0/24 http_access allow our_networks forwarded_for off ssl_bump allow all sslproxy_cert_error allow all sslproxy_flags DONT_VERIFY_PEER * what iptables rules have you used? also you better use squid 3.2 for ssl-bump. what were you reading about ssl-bump? take a look at: http://wiki.squid-cache.org/Features/SslBump and http://wiki.squid-cache.org/Features/DynamicSslCert Regards, Eliezer -- Eliezer Croitoru https://www1.ngtech.co.il IT consulting for Nonprofit organizations eliezer ngtech.co.il
[squid-users] Squid and SSL interception (ssl-bump)
Hello For a school project I'm trying to intercept SSL connections by using Squid (client -> squid (transparent) -> server). I'm running Squid 3.1.20 on Ubuntu server 12.10 (64 bit) using the following configuration: * http_port 10.0.1.1.:3128 intercept https_port 10.0.1.1.:443 ssl-bump cert=/user/local/squid3/ssl_cert/myCA.pm acl our_networks src 10.0.1.0/24 http_access allow our_networks forwarded_for off ssl_bump allow all sslproxy_cert_error allow all sslproxy_flags DONT_VERIFY_PEER * I've complied squid with SSL support (--enable-ssl). When starting Squid I do not get any error message. Also, proxying http traffic works without any problems. However, when I try to establish a HTTPS session through squid, the client retrieves the SSL certificate from squid, but after accepting it the browser displays an error message from squid that the URL is invalid: "The following error was encountered while trying to retrieve the URL: /. Invalid URL" In the Squid access.log I see the following line: " 0 10.0.1.5 NONE/440 3503 GET / - NONE/- text/html" It appears that squid does strips away the hostname / domain name of the URL the client tries to access, which causes the error message mentioned above. I've already spent hours in finding a solution for this problem and went through dozens of tutorials, unfortunately I wasn't able to find a solution so far. Any ideas what could be wrong? Regards, Heinrich
Re: [squid-users] add DENIED tag by redirector for easy identification in logfile
- Original Message - > From: Amos Jeffries > If you are interested in sponsoring any code development towards that > please > contact me off-list about payment details. Hi Amos, First of all thanks for replying immediately. But sorry to say that its a very small company. Bosses will not approve. > NOTE: redirectors do not "block" anything. They redirect. Possibly to > a location which does not exist, or a page containing the word > "blocked". Yes you are right if you consider a literal meaning and what it actually does. But I suppose most of the people use redirector only for blocking hence I used word "block". But technically you are right. > Um, REDIRECT tag is documented 6 lines above DENIED. Please upgrade to > Squid-3.2 > where this logging is available by default already. Or re-build your Squid > with > the -DLOG_TCP_REDIRECTS compiler flag. I am already using 3.2. > In all Squid whether they use that tag or not Squid will log a 301, 302, 303, > or > 307 status code along with "NONE/-" as the server contacted if > url_rewrite_program redirected the request. If there is anything else in the > upstream server field it means the 3xx status logged was generated by that > server, not by Squid. I am doing URL rewrite instead of redirect. The reason I am doing a rewrite instead of redirect is to avoid additional lookup by client. It also maintains original URL of the page in browser. Redirect otherwise changes the URL in location bar of the browser. And people get confused. And if I recall right then I have also seen some browser complaining about XSS or something, because URL domains do not match. I suppose as of now there is no solution. But thanks again. Regards, Amm
Re: [squid-users] add DENIED tag by redirector for easy identification in logfile
On 1/11/2012 12:07 a.m., Amm wrote: Hi I wanted to know if url_rewrite_program can add a TAG for logging. Not for tags output by the helper program itself. I am currently working on a project to clean up and combining the Squid helper interfaces such that for example; url_rewrite_program and external_acl_type helpers can return the same set of key=value details and have them do/mean the same things. If you are interested in sponsoring any code development towards that please contact me off-list about payment details. I have a redirector which blocks certain sites. But in squid logs there is no way to indicate if redirector blocked it. NOTE: redirectors do not "block" anything. They redirect. Possibly to a location which does not exist, or a page containing the word "blocked". As per this, there is already a tag called DENIED when request is rejected by acl. http://wiki.squid-cache.org/SquidFaq/SquidLogs#access.log I would like that redirector should also have ability to add a tag, say same one, DENIED. Um, REDIRECT tag is documented 6 lines above DENIED. Please upgrade to Squid-3.2 where this logging is available by default already. Or re-build your Squid with the -DLOG_TCP_REDIRECTS compiler flag. In all Squid whether they use that tag or not Squid will log a 301, 302, 303, or 307 status code along with "NONE/-" as the server contacted if url_rewrite_program redirected the request. If there is anything else in the upstream server field it means the 3xx status logged was generated by that server, not by Squid. If you are confusing re-write operation with "redirect". Then of course nothing is logged, because all Squid did was alter the *text* of the URL before fetching it. No redirect was done. So that its easy to identify the blocked requests (either by acl or by redirector) access controls (ACLs): * deny permission to access a requested resource * block the request from accessing the resource redirectors: * redirect to an alternative resource. * do not block the response Similar feature already exists for external_acl_type: http://www.squid-cache.org/Doc/config/external_acl_type/ which says: tag =Apply a tag to a request (for both ERR and OK results) So can redirector do the same? Not yet. see above about sponsorship to get early access to that ability and a faster time on the implementation. Amos
[squid-users] add DENIED tag by redirector for easy identification in logfile
Hi I wanted to know if url_rewrite_program can add a TAG for logging. I have a redirector which blocks certain sites. But in squid logs there is no way to indicate if redirector blocked it. As per this, there is already a tag called DENIED when request is rejected by acl. http://wiki.squid-cache.org/SquidFaq/SquidLogs#access.log I would like that redirector should also have ability to add a tag, say same one, DENIED. So that its easy to identify the blocked requests (either by acl or by redirector) Similar feature already exists for external_acl_type: http://www.squid-cache.org/Doc/config/external_acl_type/ which says: tag =Apply a tag to a request (for both ERR and OK results) So can redirector do the same? Thanks in advance, Amm
Re: [squid-users] feature request: setting location of coordinator.ipc and kidx.ipc during runtime?
> -Ursprüngliche Nachricht-
> Von: Amos Jeffries [mailto:squ...@treenet.co.nz]
> Gesendet: Mittwoch, 24. Oktober 2012 01:08
> An: squid-users@squid-cache.org
> Betreff: Re: [squid-users] feature request: setting location of
> coordinator.ipc and kidx.ipc during runtime?
>
> On 24.10.2012 03:38, Rietzler, Markus (RZF, SG 324 /
> ) wrote:
> > hi,
> >
> > we want to use squid with smp workers.
> > workers are running fine. now also logroate works (although not as
> > expected. see my other mail "[squid-users] question of understanding:
> > squid smp/workers and logfiles", works only with access_log for each
> > worker not one single one).
> >
> > now there is only one problem.
> >
> > when we compile squid we use
> >
> > ./configure --prefix /default/path/to/squid
> >
> > in our production environment squid lies under a different path (eg.
> > /path/to/squid). we also use several instances of squid, etc. one
> > internet, one intranet, one extranet etc. each one with its own
> > directory structure like etc, run, log, cache etc.
> >
> > via squid.conf we can set every required path (log, log_file_daemon,
> > icons, error, unlinkd etc) but not for the ipc-location.
> >
> > in src/ipc/Port.cc the location is hardcoded:
> >
> > const char Ipc::coordinatorAddr[] = DEFAULT_STATEDIR
> > "/coordinator.ipc";
> > const char Ipc::strandAddrPfx[] = DEFAULT_STATEDIR "/kid";
> >
> > I can patch src/ipc/Makefile to have localstatedir point to a other
> > dir then /default/path/to/squid/var (that's how localstatedir will be
> > expanded in the Makefile). but this is not really what we want. we
> > want to be able to have the location set via squid.conf or
> > environment
> > var during runtime.
> >
> > we tried to use something like
> >
> > const char Ipc::coordinatorAddr[] = Config.coredump_dir
> > "/coordinator.ipc";
> >
> > but then we get compile erros.
> >
> > is it possible to create some patch to have to set the location of
> > ipc-files during runtime.
>
> Yes and no.
>
> These are network sockets needing to be accessed by all instances of
> the multiple processes which form Squid. There is no reason to touch or
> change them.
> If we allow reconfiguration of where one is placed, anyone could
> accidentally place that inside if...else conditions and will be unable
> to operate their Squid reliably when the internal communication channels
> to the coordinator become disconnected.
> If we allowed you to register multiple "/some/shared/kid1.ipc" then
> start several differently configured Squid you could face the second
> instance crashing with unable to open socket errors or you could zombie
> the existing process, or you could cause crossover between the two
> coordinators or the two workers.
> We really do not want to have to assist with debugging that type of
> problem needlessly
>
sounds reasonable
>
> The SMP support in Squid is designed to remove any reason why you
> should need to operate multiple different Squid installations on one
> box. It is almost but not quite complete, if you find a particular
> feature (like that logs bug) you need to segment but are unable to do so
> please pint out. The UDS channel sockets notwithstanding as they are the
> mechanism by which segmentation is coordinated and enforced.
>
>
> To operate Squid with multiple segregated run-time environments for
> different clients I suggest you look at re-designing your squid.conf
> along these lines:
>
> squid.conf:
> workers 3
> /etc/squid/squid.conf.${process_id}
>
>
> With squid.conf.1, squid.conf.2, squid.conf.3 containing a complete
> copy of what would have been squid.conf for the environment you want to
> present to your client base that process is serving.
> When you need to guarantee a per-worker resource like log files use
> ${process_id} as art of the path or filename like the above example. You
> can also use ${process_name} the same way.
>
> FUN: If you need two workers to both present one shared environment you
> can use symlinks to point squid.conf.4 at squid.conf.5 for example and
> the coordinator will ensure they share resources as well as config
> files.
> * clashes with using the ${process_id} macro in paths
>
> MORE FUN: to share resources between environments, just configure the
> same lines for the cache location etc in multiple per-worker squid.conf.
> Again the coordinator will link the processes together with the shared
> resource.
>
> PS: we currently only provide one shared memory cache. So segmenting
> that is not possible the old style local caches can be used instead. TMF
> have a project cleaning up the cache systems underway to make things
> more flexible, get in touch if you need any changes there.
>
> Amos
ok, this sounds like a good idea. at the moment we have 3 squids running for
internet, intranet and extranet. so each one have a own squid.conf and on
acl-rules.
we could use the trick with squid.conf.{$process_id} etc. but there is one
small thing that d
AW: [squid-users] No Kerberos Auth
I've found this today. why is the last ticket not renewed ?? Could that point the problem [root@http-proxy ~]# klist Ticket cache: FILE:/tmp/krb5cc_0 Default principal: HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de Valid starting ExpiresService principal 10/30/12 14:47:38 10/31/12 00:47:37 krbtgt/justiz.niedersachsen...@justiz.niedersachsen.de renew until 10/31/12 14:47:38 10/30/12 15:24:49 10/31/12 00:47:37 ldap/justizhadc01.justiz.niedersachsen...@justiz.niedersachsen.de renew until 10/31/12 14:47:38 10/30/12 15:24:49 10/30/12 15:26:49 kadmin/chang...@justiz.niedersachsen.de renew until 10/30/12 15:26:49 -Ursprüngliche Nachricht- Von: Jarosch, Ralph [mailto:ralph.jaro...@justiz.niedersachsen.de] Gesendet: Dienstag, 30. Oktober 2012 15:27 An: Bastien Ceriani Cc: squid-users@squid-cache.org Betreff: AW: [squid-users] No Kerberos Auth I think encrypte Type is already 28. This is the output with -- encrypt 28 -- ldap_set_supportedEncryptionTypes: No need to change msDs-supportedEncryptionTypes they are 28 Von: Jarosch, Ralph Gesendet: Dienstag, 30. Oktober 2012 15:24 An: 'Bastien Ceriani' Cc: squid-users@squid-cache.org Betreff: AW: [squid-users] No Kerberos Auth Oh ok.. yes it work fine until ten minute i wrote the mail. There it crashed from one minute to the other I'am just troubleshoot the problem.. Von: Bastien Ceriani [mailto:bastien.ceri...@bulkypix.com] Gesendet: Dienstag, 30. Oktober 2012 15:16 An: Jarosch, Ralph Cc: squid-users@squid-cache.org Betreff: Re: [squid-users] No Kerberos Auth Ok Thx, With Windows Server 2008 you should use --enctypes 28 parameter with msktutils command. Did your ntlm authentification work fine ? How did you configure it ? With Samba/Winbind ? On Tue, Oct 30, 2012 at 3:08 PM, Jarosch, Ralph wrote: OK for wireshark i must wait for tonight because no one here can work If enable authentication My keytab Keytab name: WRFILE:/etc/squid/HTTP.keytab KVNO Timestamp Principal - 6 10/30/12 09:47:42 http-proxy$@JUSTIZ.NIEDERSACHSEN.DE (arcfour-hmac) 6 10/30/12 09:47:42 http-proxy$@JUSTIZ.NIEDERSACHSEN.DE (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 http-proxy$@JUSTIZ.NIEDERSACHSEN.DE (aes256-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de (arcfour-hmac) 6 10/30/12 09:47:42 HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de (aes256-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-pr...@justiz.niedersachsen.de (arcfour-hmac) 6 10/30/12 09:47:42 HTTP/http-pr...@justiz.niedersachsen.de (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HTTP/http-pr...@justiz.niedersachsen.de (aes256-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HOST/http-pr...@justiz.niedersachsen.de (arcfour-hmac) 6 10/30/12 09:47:42 HOST/http-pr...@justiz.niedersachsen.de (aes128-cts-hmac-sha1-96) 6 10/30/12 09:47:42 HOST/http-pr...@justiz.niedersachsen.de (aes256-cts-hmac-sha1-96) My Squid.conf auth_param negotiate program /usr/lib64/squid/squid_kerb_auth -d -i -s HTTP/http-proxy.justiz.niedersachsen...@justiz.niedersachsen.de auth_param negotiate children 100 auth_param negotiate keep_alive on auth_param ntlm keep_alive on auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp auth_param ntlm children 200 #auth_param ntlm max_challenge_reuses 0 #auth_param ntlm max_challenge_lifetime 2 minutes auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic auth_param basic children 200 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 5 hours and my msktutil msktutil -c -b "OU=Sonstige Server,OU=Globale Dienste,DC=justiz,DC=niedersachsen,DC=de" -s HTTP/http-proxy.justiz.niedersachsen.de -h http-proxy.justiz.niedersachsen.de -k /etc/HTTP.keytab --computer-name http-proxy --upn HTTP/http-proxy.justiz.niedersachsen.de --server justizhadc01.justiz.niedersachsen.de --verbose We use Windows 2008 R2 Server Von: Bastien Ceriani [mailto:bastien.ceri...@bulkypix.com] Gesendet: Dienstag, 30. Oktober 2012 15:00 An: Jarosch, Ralph Betreff: Re: [squid-users] No Kerberos Auth I'm in the same case.. Try to check kerberos TGS REQ and TGS REP with wireshark ? Can you display : - your keytab ? (klist -ekt HTTP.keytab) - your auth_param squid config - your mskutils command What version of windows server is running ? Regards, On Tue, Oct 30, 2012 at 2:49 PM, Jarosch, Ralph wrote: Hi, i have some trouble to authenticate our web browser over Kerberos. I Always get the following error message. 2012/10/30 14:27:55| squid_kerb_auth: DEBUG: Decode 'YIIJsQYGKwYBBQUCoIIJpTCCCaGgMDAuBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHgYKKwYBBAGCNwIC
