[squid-users] Squid Cache NEVER HIT's Only get TCP_MISS/200 and TCP_MISS/304
I am struggling with my Squid Reverse proxy cache. I have been all round the forums with no success in getting my Squid Proxy Cache to actually do any caching. I am running Squid 3.1 on Debian 6 Can anyone suggest what might be wrong with my Squid.conf file? I have verified that the correct permissions exist on the cache folder, cache folders are initialized, no errors are returned from running squid3 -k parse Regards, Robin --Squid.conf-- http_port 80 accel ignore-cc defaultsite=richmedia.mydomain.com cache_mem 500 MB maximum_object_size_in_memory 5 KB cache_dir ufs /var/spool/squid3 1 32 512 max-size=10485760 minimum_object_size 2 KB maximum_object_size 5000 MB refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp|xml)$ 26 90% 260009 refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv|mpg|wma|ogg|wmv|asx|asf)$ 26 90% 260009 refresh_pattern . 26 90% 260009 acl xdomain urlpath_regex ^/crossdomain.xml cache_peer 94.125.16.13 parent 80 0 no-query no-digest originserver name=server1 cache_peer_access server1 deny xdomain cache_peer 162.13.17.12 parent 8080 0 no-query no-digest originserver name=server2 cache_peer_access server2 allow xdomain cache_peer_access server2 deny all cache allow all http_access allow all cache_effective_user proxy cache_effective_group proxy --Access.log output-- 1394187754.972 108 195.157.14.29 TCP_MISS/200 118376 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187754.992 30 62.232.36.16 TCP_MISS/200 1004 GET http://richmedia.mydomain.com/favicon.ico - FIRST_UP_PARENT/server1 image/x-icon 1394187755.163 94 62.232.36.16 TCP_MISS/200 68954 GET http://richmedia.mydomain.com/media/webinar/supplier/I-Holland-2013Webinar/slides/Slide00029.swf - FIRST_UP_PARENT/server1 application/x-shockwave-flash 1394187765.378 9794 195.157.14.29 TCP_MISS/200 1696587 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187768.885 136 195.157.14.29 TCP_MISS/200 169077 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187782.779 38 62.232.3.16 TCP_MISS/200 1611 GET http://richmedia.mydomain.com/media/webinar/supplier/I-Holland-2013Webinar/slides/Slide624911.htm - FIRST_UP_PARENT/server1 text/html 1394187783.461 35 79.171.8.14 TCP_MISS/200 8811 GET http://richmedia.mydomain.com/media/webinar/supplier/Kampffmeyer-14Nov13/index.htm - FIRST_UP_PARENT/server1 text/html 1394187788.851 19370 195.157.14.29 TCP_MISS/200 3110156 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187792.101 66784 195.157.14.29 TCP_MISS/206 3961057 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187793.415 100 195.157.14.29 TCP_MISS/200 154126 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187807.537 13461 195.157.14.29 TCP_MISS/200 2109420 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187819.670 3 95.131.10.18 TCP_MISS/200 607 GET http://richmedia.mydomain.com/crossdomain.xml - FIRST_UP_PARENT/server2 application/xml 1394187838.664 144 195.157.14.29 TCP_MISS/200 115568 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187855.303 35596 95.131.10.18 TCP_MISS/200 75550871 GET http://richmedia.mydomain.com/content/download/424921/8844388/file/Apprenticeships.mp4 - FIRST_UP_PARENT/server1 video/mp4 1394187867.488 28168 195.157.14.29 TCP_MISS/200 3961100 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg
[squid-users] RE: Squid Cache NEVER HIT's Only get TCP_MISS/200 and TCP_MISS/304
My copy and paste was not correct in the original post. I have corrected my conf file below. Robin -Original Message- From: Robin Gwynne [mailto:robin.gwy...@wrbm.com] Sent: 07 March 2014 10:46 To: squid-users@squid-cache.org Subject: [squid-users] Squid Cache NEVER HIT's Only get TCP_MISS/200 and TCP_MISS/304 I am struggling with my Squid Reverse proxy cache. I have been all round the forums with no success in getting my Squid Proxy Cache to actually do any caching. I am running Squid 3.1 on Debian 6 Can anyone suggest what might be wrong with my Squid.conf file? I have verified that the correct permissions exist on the cache folder, cache folders are initialized, no errors are returned from running squid3 -k parse Regards, Robin --Squid.conf-- http_port 80 accel ignore-cc defaultsite=richmedia.mydomain.com cache_mem 500 MB maximum_object_size_in_memory 5 KB cache_dir ufs /var/spool/squid3 1 32 512 max-size=10485760 minimum_object_size 2 KB maximum_object_size 5000 MB refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp|xml)$ 26 90% 260009 refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv|mpg|wma|ogg|wmv|asx|asf)$ 26 90% 260009 refresh_pattern . 26 90% 260009 acl xdomain urlpath_regex ^/crossdomain.xml cache_peer 94.125.16.13 parent 80 0 no-query no-digest originserver name=server1 cache_peer_access server1 deny xdomain cache_peer 162.13.17.12 parent 8080 0 no-query no-digest originserver name=server2 cache_peer_access server2 allow xdomain cache_peer_access server2 deny all cache allow all http_access allow all --Access.log output-- 1394187754.972 108 195.157.14.29 TCP_MISS/200 118376 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187754.992 30 62.232.36.16 TCP_MISS/200 1004 GET http://richmedia.mydomain.com/favicon.ico - FIRST_UP_PARENT/server1 image/x-icon 1394187755.163 94 62.232.36.16 TCP_MISS/200 68954 GET http://richmedia.mydomain.com/media/webinar/supplier/I-Holland-2013Webinar/slides/Slide00029.swf - FIRST_UP_PARENT/server1 application/x-shockwave-flash 1394187765.378 9794 195.157.14.29 TCP_MISS/200 1696587 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187768.885 136 195.157.14.29 TCP_MISS/200 169077 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187782.779 38 62.232.3.16 TCP_MISS/200 1611 GET http://richmedia.mydomain.com/media/webinar/supplier/I-Holland-2013Webinar/slides/Slide624911.htm - FIRST_UP_PARENT/server1 text/html 1394187783.461 35 79.171.8.14 TCP_MISS/200 8811 GET http://richmedia.mydomain.com/media/webinar/supplier/Kampffmeyer-14Nov13/index.htm - FIRST_UP_PARENT/server1 text/html 1394187788.851 19370 195.157.14.29 TCP_MISS/200 3110156 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187792.101 66784 195.157.14.29 TCP_MISS/206 3961057 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187793.415 100 195.157.14.29 TCP_MISS/200 154126 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187807.537 13461 195.157.14.29 TCP_MISS/200 2109420 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187819.670 3 95.131.10.18 TCP_MISS/200 607 GET http://richmedia.mydomain.com/crossdomain.xml - FIRST_UP_PARENT/server2 application/xml 1394187838.664 144 195.157.14.29 TCP_MISS/200 115568 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187855.303 35596 95.131.10.18 TCP_MISS/200 75550871 GET http://richmedia.mydomain.com/content/download/424921/8844388/file/Apprenticeships.mp4 - FIRST_UP_PARENT/server1 video/mp4 1394187867.488 28168 195.157.14.29 TCP_MISS/200 3961100 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg
[squid-users] Re: HTTP/1.1 pipelining
They still have to be read and processed in order. Squid reads requests out of the client connection one at a time and processes them. Could this be a bit more clarified ? I mean, when squid started to process the first request from pipeline (request forwarded to destination), will squid also start to process the next request from pipeline in parallel, or wait, until previous one completed ? Are there mayor differences in pipelining between squid2.7 and newest versions ? Actually, I am located in a remote area, ping from my client to squid is about 300-350ms. Theoretically, pipelining should be of benefit here, as I also suspect, my wireless ISP limits the amount of parallel conns. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/HTTP-1-1-pipelining-tp4658574p4665093.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] squid queue overload. request rejected
On 03/04/2014 11:39 PM, ***some text missing*** wrote: Not able to follow your reply. Rephrasing Amos' reply: You have configured Squid with an external ACL helper. That external helper cannot handle the load. In other words, that helper program is too slow for the amount of traffic your Squid is sending to it. Please suggest solution. Solutions include: * sending fewer Squid transactions to the external ACL helper (it is sometimes possible to rearrange the ACL rules so that the external ACL is not checked that often); * making the helper program work faster; * using more helpers; * using more Squid workers; * using more Squid instances; and * sending fewer requests to Squid. Some of these require Squid3. Alex. - Original Message - From: Amos Jeffries squ...@treenet.co.nz To: squid-users@squid-cache.org Cc: Sent: Wednesday, March 5, 2014 10:37 AM Subject: Re: [squid-users] squid queue overload. request rejected On 4/03/2014 8:53 p.m., ***some text missing*** wrote: Hello, From last week I am getting messages in my cache.log squid queue overload. request rejected and most of my users unable to browse the webpages. I have configured squid with external helper ACL and using squid stable version squid2.7 stable 9. Any idea about this error. You have named the ACL squid and the helper is not able to cope with the amount of req/sec your Squid proxy is needing to pass to it. Amos
Re: [squid-users] Re: cache.log errors and warnings: how fatal are they
On 03/05/2014 10:51 AM, Niki Gorchilov wrote: On Wed, Mar 5, 2014 at 6:59 PM, Niki Gorchilov n...@gorchilov.com wrote: Running a rock store experiment with SMP Squid 3.HEAD rev 13295 on 750 mbps of HTTP traffic. There're few cache.log errors and warnings that aren't clear enough for me. What triggers them, Squid bugs and/or invalid input, depending on the specific error and/or transaction. IMO, Squid should report these differently so that there are fewer questions/concerns about them, but there is currently no consensus regarding the best approach (or even regarding the existence of the systemic reporting/debugging problem itself!). Today, in most of these cases, the only way to know for sure what happened and what the exact effects are is to dive into detailed debugging logs (and even that might not be enough in corner cases). Monitoring access.log for errors may be helpful, but does not allow you to tie cache.log errors to access.log errors in most cases. and how do they affect the corresponding request that provoked them? WARNING: Ignoring malformed cache entry WARNING: swapfile header inconsistent with available data WARNING: 10 swapin MD5 mismatches Could not parse headers from on disk object clientProcessHit: Vary object loop! varyEvaluateMatch: Oops. Not a Vary object on second attempt, 'http://www.shaadi.com/' 'accept-encoding=gzip,deflate,sdch' clientIfRangeMatch: Weak ETags are not allowed in If-Range: 68390-1391166396000 ? 68390-1391166396000 fqdncacheParse: No PTR record for '77.87.212.52' One more error: idnsGrokReply: Malformed DNS response My main concern is if any one of the above actually breaks the corresponding request? In most cases, these problems are not fatal to the request, except that some DNS errors may prevent Squid from forwarding the request, of course. And yes, I know that the above does not fully answer your questions, but that is the best answer I personally can offer at this time. HTH, Alex.
Re: [squid-users] Re: HTTP/1.1 pipelining
On 03/07/2014 04:19 AM, babajaga wrote: They still have to be read and processed in order. Squid reads requests out of the client connection one at a time and processes them. Could this be a bit more clarified ? I mean, when squid started to process the first request from pipeline (request forwarded to destination), will squid also start to process the next request from pipeline in parallel, or wait, until previous one completed ? By default, Squid will not process more than one concurrent request received on the same connection. For details, please see pipeline_prefetch in squid.conf.documented or http://www.squid-cache.org/Doc/config/pipeline_prefetch/ Alex.
[squid-users] Squid not accelerating properly
I have been long searching for a solution and finally this morning I got it to work. My setup is as follows: Wan16port Dlink switchClearosmikrotiknetequalizer24 port Dlink switch I have added a squid with its input from the Wan directly and then I have put the squid directly to the mikrotik. I did the following configurations: Wan: Wan - mikrotik 172.16.10.1/24 Wan - squid 172.16.11.1/24 Mikrotik Ether1 172.16.10.2/24 Via setup CLI Ether2 (Hotspot) 10.5.50.1/24 Ether3 to squid 192.168.50.2 Via setup CLI Squid Ether1 from Wan 172.16.11.2 Ether2 from mikrotik 192.168.50.1:3128 The squid is configured transparently. The CLI commands used are as follows: #Mark All HTTP Port 80 Traffic, so that you can use these Marked Packets in Route section. /ip firewall nat add action=accept chain=srcnat disabled=no dst-port=80 protocol=tcp /ip firewall mangle add action=mark-routing chain=prerouting disabled=no dst-port=80 new-routing-mark=http passthrough=yes protocol=tcp /ip route add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=(192.168.50.1) routing-mark=http scope=30 target-scope=10 add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=(172.16.10.1) scope=30 target-scope=10 /ip firewall mangle add chain=postrouting tos=48 action=mark-packet new-packet-mark=proxy-hit passthrough=no /ip firewall mangle add chain=postrouting action=mark-packet new-packet-mark=proxy-hit passthrough=no /queue tree add name=pmark parent=global-out packet-mark=proxy-hit \ limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s /ip firewall filter add action=add-src-to-address-list address-list=Syn_Flooder address-list-timeout=30m chain=input \ comment=Add Syn Flood IP to the list connection-limit=30,32 disabled=no protocol=tcp tcp-flags=syn add action=drop chain=input comment=Drop to syn flood list disabled=no src-address-list=Syn_Flooder add action=add-src-to-address-list address-list=Port_Scanner address-list-timeout=1w chain=input comment=Port Scanner Detect\ disabled=no protocol=tcp psd=21,3s,3,1 add action=drop chain=input comment=Drop to port scan list disabled=no src-address-list=Port_Scanner add action=jump chain=input comment=Jump for icmp input flow disabled=no jump-target=ICMP protocol=icmp add action=drop chain=input\ comment=Block all access to the winbox - except to support list add action=jump chain=forward comment=Jump for icmp forward flow disabled=no jump-target=ICMP protocol=icmp add action=drop chain=forward comment=Drop to bogon list disabled=no dst-address-list=bogons add action=add-src-to-address-list address-list=spammers address-list-timeout=3h chain=forward comment=Add Spammers to the list for 3 hours\ connection-limit=30,32 disabled=no dst-port=25,587 limit=30/1m,0 protocol=tcp add action=drop chain=forward comment=Avoid spammers action disabled=no dst-port=25,587 protocol=tcp src-address-list=spammers add action=accept chain=input comment=Accept DNS - UDP disabled=no port=53 protocol=udp add action=accept chain=input comment=Accept DNS - TCP disabled=no port=53 protocol=tcp add action=accept chain=input comment=Accept to established connections connection-state=established\ disabled=no add action=accept chain=input comment=Accept to related connections connection-state=related disabled=no add action=accept chain=input comment=Full access to SUPPORT address list disabled=no src-address-list=support add action=drop chain=input comment=Drop anything else! add action=accept chain=ICMP comment=Echo request - Avoiding Ping Flood disabled=no icmp-options=8:0 limit=1,5 protocol=icmp add action=accept chain=ICMP comment=Echo reply disabled=no icmp-options=0:0 protocol=icmp add action=accept chain=ICMP comment=Time Exceeded disabled=no icmp-options=11:0 protocol=icmp add action=accept chain=ICMP comment=Destination unreachable disabled=no icmp-options=3:0-1 protocol=icmp add action=accept chain=ICMP comment=PMTUD disabled=no icmp-options=3:4 protocol=icmp add action=drop chain=ICMP comment=Drop to the other ICMPs disabled=no protocol=icmp add action=jump chain=output comment=Jump for icmp output disabled=no jump-target=ICMP protocol=icmp ip firewall nat add action=dst-nat chain=dstnat disabled=no dst-port=80 protocol=tcp to-addresses=10.5.50.5 to-ports=8080 ip firewall nat add action=dst-nat dst-port=80 protocol=tcp src-address=10.5.50.0/24 to-addresses=10.5.50.5 to-ports=8080 chain=dstnat ip firewall nat add chain=dstnat src-address=10.5.50.0/24 in-interface=ether1 dst-port=80 protocol=tcp action=dst-nat to-address=10.5.50.5 to-port=8080 ip firewall nat add chain=dstnat src-address=10.5.50.5 dst-port=80 protocol=tcp action=accept ip firewall nat add chain=dstnat src-address=10.5.50.0/24 dst-port=80 protocol=tcp action=dst-nat to-address=10.5.50.5 to-port=8080 When i run the tail command in the squid i get a lot of activity within the cache; for example 1394214401.152103
[squid-users] transparent SSL and cache_peer
I have a Squid Cache: Version 3.4.3 this --enable-ssl --enable-ssl-crtd In squid.conf: http_port 3128 transparent https_port 3129 transparent ssl-bump key=/etc/squid3/ssl/privkey.pem cert=/etc/squid3/ssl/newcert.pem ssl_bump client-first all sslproxy_flags DONT_VERIFY_PEER acl to_sniff dstdom_regex .com$ cache_peer 192.168.56.100 parent 0 no-query no-digest name=peer1 cache_peer_access peer1 allow to_sniff cache_peer_access peer1 deny all Iptables rules: iptables -t nat -A PREROUTING -s 192.168.56.42 -p tcp --dport 80 -j REDIRECT --to-port 3128 iptables -t nat -A PREROUTING -s 192.168.56.42 -p tcp --dport 443 -j REDIRECT --to-port 3129 HTTP traffic is successfuly go to the cache_peer, but HTTPS don't send any request to the cache_peer. If we disable acl to_sniff dstdom_regex .com$ ssl_bump work fine. No errors in the log file. Why HTTPS don't send any request to the cache_peer?
Re: [squid-users] RE: Squid Cache NEVER HIT's Only get TCP_MISS/200 and TCP_MISS/304
On 7/03/2014 11:53 p.m., Robin Gwynne wrote: My copy and paste was not correct in the original post. I have corrected my conf file below. Robin -Original Message- From: Robin Gwynne [mailto:robin.gwy...@wrbm.com] Sent: 07 March 2014 10:46 To: squid-users@squid-cache.org Subject: [squid-users] Squid Cache NEVER HIT's Only get TCP_MISS/200 and TCP_MISS/304 I am struggling with my Squid Reverse proxy cache. I have been all round the forums with no success in getting my Squid Proxy Cache to actually do any caching. I am running Squid 3.1 on Debian 6 Can anyone suggest what might be wrong with my Squid.conf file? I have verified that the correct permissions exist on the cache folder, cache folders are initialized, no errors are returned from running squid3 -k parse Regards, Robin --Squid.conf-- http_port 80 accel ignore-cc defaultsite=richmedia.mydomain.com cache_mem 500 MB maximum_object_size_in_memory 5 KB cache_dir ufs /var/spool/squid3 1 32 512 max-size=10485760 minimum_object_size 2 KB maximum_object_size 5000 MB So you to get thsis straight: up to 50MB objects are allowed to cache in memory up to 10MB are allowed to cache on disk nothing smaller than 2KB is allowed to be cached refresh_pattern -i \.(gif|png|jpg|jpeg|ico|bmp|xml)$ 26 90% 260009 refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv|mpg|wma|ogg|wmv|asx|asf)$ 26 90% 260009 refresh_pattern . 26 90% 260009 acl xdomain urlpath_regex ^/crossdomain.xml cache_peer 94.125.16.13 parent 80 0 no-query no-digest originserver name=server1 cache_peer_access server1 deny xdomain cache_peer 162.13.17.12 parent 8080 0 no-query no-digest originserver name=server2 cache_peer_access server2 allow xdomain cache_peer_access server2 deny all cache allow all http_access allow all Security note: The way you have designed the http_access and cache_peer_access rules means that almost all attack traffic delivered to your Squid will be relayed through the server1. One of the benefits of having a reverse-proxy is to protect the backend servers against those types of request. I highly recommend adding a dstdomain ACL defining what domain(s) you host and using that for http_access allow instead of acccepting everything. --Access.log output-- 1394187754.972108 195.157.14.29 TCP_MISS/200 118376 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187754.992 30 62.232.36.16 TCP_MISS/200 1004 GET http://richmedia.mydomain.com/favicon.ico - FIRST_UP_PARENT/server1 image/x-icon 1394187755.163 94 62.232.36.16 TCP_MISS/200 68954 GET http://richmedia.mydomain.com/media/webinar/supplier/I-Holland-2013Webinar/slides/Slide00029.swf - FIRST_UP_PARENT/server1 application/x-shockwave-flash 1394187765.378 9794 195.157.14.29 TCP_MISS/200 1696587 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187768.885136 195.157.14.29 TCP_MISS/200 169077 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187782.779 38 62.232.3.16 TCP_MISS/200 1611 GET http://richmedia.mydomain.com/media/webinar/supplier/I-Holland-2013Webinar/slides/Slide624911.htm - FIRST_UP_PARENT/server1 text/html 1394187783.461 35 79.171.8.14 TCP_MISS/200 8811 GET http://richmedia.mydomain.com/media/webinar/supplier/Kampffmeyer-14Nov13/index.htm - FIRST_UP_PARENT/server1 text/html 1394187788.851 19370 195.157.14.29 TCP_MISS/200 3110156 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187792.101 66784 195.157.14.29 TCP_MISS/206 3961057 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187793.415100 195.157.14.29 TCP_MISS/200 154126 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187807.537 13461 195.157.14.29 TCP_MISS/200 2109420 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187819.670 3 95.131.10.18 TCP_MISS/200 607 GET http://richmedia.mydomain.com/crossdomain.xml - FIRST_UP_PARENT/server2 application/xml 1394187838.664144 195.157.14.29 TCP_MISS/200 115568 GET http://richmedia.mydomain.com/content/download/383683/8226052/file/Tim%20Storer.mp3 - FIRST_UP_PARENT/server1 audio/mpeg 1394187855.303 35596 95.131.10.18 TCP_MISS/200 75550871 GET http://richmedia.mydomain.com/content/download/424921/8844388/file/Apprenticeships.mp4 - FIRST_UP_PARENT/server1 video/mp4 1394187867.488 28168
[squid-users] Re: HTTP/1.1 pipelining
Alex, then the following in http://www.squid-cache.org/Doc/config/pipeline_prefetch/ is misleading: If set to N, Squid will try to receive and process up to 1+N requests on the same connection concurrently. Note the concurrently. For older versions of squid, it is stated differently. -- View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/HTTP-1-1-pipelining-tp4658574p4665100.html Sent from the Squid - Users mailing list archive at Nabble.com.
Re: [squid-users] Squid not accelerating properly
Lets start with the title... Your Squid is being used as an interception proxy. Not an accelerator / reverse-proxy. Getting the terms right will greatly improve your ability to search for relevant information. On 8/03/2014 6:59 a.m., Oluseyi Akinboboye wrote: I have been long searching for a solution and finally this morning I got it to work. My setup is as follows: Wan16port Dlink switchClearosmikrotiknetequalizer24 port Dlink switch I have added a squid with its input from the Wan directly and then I have put the squid directly to the mikrotik. So to translate your diagram and description: WAN - Squid - Router - LAN is that correct? I am assuming from the description that Squid is running on the ClearOS machine. I did the following configurations: Wan: Wan - mikrotik 172.16.10.1/24 Wan - squid 172.16.11.1/24 Huh? if I'm reading that right you have two distinct routes that packets from the WAN - LAN may take. Only one of which goes through Squid. Be very VERY careful with the packet flows when doing this. Mikrotik Ether1 172.16.10.2/24 Via setup CLI Ether2 (Hotspot) 10.5.50.1/24 Ether3 to squid 192.168.50.2 Via setup CLI Squid Ether1 from Wan 172.16.11.2 Ether2 from mikrotik 192.168.50.1:3128 I dont understand how that relates to the actual packet flows sorry. Too many undefined details like: - how all the EtherN are plugged together - what the terminal command line interface (CLI) has to do with routing, - which part(s) of your network each of those IP ranges identifies The squid is configured transparently. How? there are 8 transparent interception configurations for Squid. And a great many more ways to mis-configure it. The CLI commands used are as follows: Are these on the Mikrotik or ClearOS? #Mark All HTTP Port 80 Traffic, so that you can use these Marked Packets in Route section. /ip firewall nat add action=accept chain=srcnat disabled=no dst-port=80 protocol=tcp /ip firewall mangle add action=mark-routing chain=prerouting disabled=no dst-port=80 new-routing-mark=http passthrough=yes protocol=tcp /ip route add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=(192.168.50.1) routing-mark=http scope=30 target-scope=10 add disabled=no distance=1 dst-address=0.0.0.0/0 gateway=(172.16.10.1) scope=30 target-scope=10 /ip firewall mangle add chain=postrouting tos=48 action=mark-packet new-packet-mark=proxy-hit passthrough=no /ip firewall mangle add chain=postrouting action=mark-packet new-packet-mark=proxy-hit passthrough=no /queue tree add name=pmark parent=global-out packet-mark=proxy-hit \ limit-at=0 queue=default priority=8 max-limit=0 burst-limit=0 burst-threshold=0 burst-time=0s /ip firewall filter add action=add-src-to-address-list address-list=Syn_Flooder address-list-timeout=30m chain=input \ comment=Add Syn Flood IP to the list connection-limit=30,32 disabled=no protocol=tcp tcp-flags=syn add action=drop chain=input comment=Drop to syn flood list disabled=no src-address-list=Syn_Flooder add action=add-src-to-address-list address-list=Port_Scanner address-list-timeout=1w chain=input comment=Port Scanner Detect\ disabled=no protocol=tcp psd=21,3s,3,1 add action=drop chain=input comment=Drop to port scan list disabled=no src-address-list=Port_Scanner You might want to ensure Squid cannot be caught and listed as a SYN-flooder. Squid will potentially open many hundreds of connections per second if lots of clients are using it. Without the proxy that would be spread over many client IPs and not hit flooding limits. add action=jump chain=input comment=Jump for icmp input flow disabled=no jump-target=ICMP protocol=icmp add action=drop chain=input\ comment=Block all access to the winbox - except to support list add action=jump chain=forward comment=Jump for icmp forward flow disabled=no jump-target=ICMP protocol=icmp add action=drop chain=forward comment=Drop to bogon list disabled=no dst-address-list=bogons add action=add-src-to-address-list address-list=spammers address-list-timeout=3h chain=forward comment=Add Spammers to the list for 3 hours\ connection-limit=30,32 disabled=no dst-port=25,587 limit=30/1m,0 protocol=tcp add action=drop chain=forward comment=Avoid spammers action disabled=no dst-port=25,587 protocol=tcp src-address-list=spammers add action=accept chain=input comment=Accept DNS - UDP disabled=no port=53 protocol=udp add action=accept chain=input comment=Accept DNS - TCP disabled=no port=53 protocol=tcp add action=accept chain=input comment=Accept to established connections connection-state=established\ disabled=no add action=accept chain=input comment=Accept to related connections connection-state=related disabled=no add action=accept chain=input comment=Full access to SUPPORT address list disabled=no src-address-list=support add action=drop chain=input comment=Drop
Re: [squid-users] transparent SSL and cache_peer
On 8/03/2014 9:03 a.m., cy...@irc.pp.ru wrote: I have a Squid Cache: Version 3.4.3 this --enable-ssl --enable-ssl-crtd In squid.conf: http_port 3128 transparent https_port 3129 transparent ssl-bump key=/etc/squid3/ssl/privkey.pem cert=/etc/squid3/ssl/newcert.pem ssl_bump client-first all sslproxy_flags DONT_VERIFY_PEER acl to_sniff dstdom_regex .com$ cache_peer 192.168.56.100 parent 0 no-query no-digest name=peer1 cache_peer_access peer1 allow to_sniff cache_peer_access peer1 deny all Iptables rules: iptables -t nat -A PREROUTING -s 192.168.56.42 -p tcp --dport 80 -j REDIRECT --to-port 3128 iptables -t nat -A PREROUTING -s 192.168.56.42 -p tcp --dport 443 -j REDIRECT --to-port 3129 HTTP traffic is successfuly go to the cache_peer, but HTTPS don't send any request to the cache_peer. If we disable acl to_sniff dstdom_regex .com$ ssl_bump work fine. No errors in the log file. Why HTTPS don't send any request to the cache_peer? Several reasons why Squid would not send HTTPS to that peer: 1) the cache_peer is insecure. Sending decrypted traffic to it invalidates the use of TLS from the client. 2) Squid does not properly support generating new CONNECT messages to re-encrypt the HTTPS traffic. This prevents sending secure traffic over insecure cache_peer like yours. What errors are you seeing? Amos
Re: [squid-users] Re: HTTP/1.1 pipelining
On 8/03/2014 4:00 p.m., babajaga wrote: Alex, then the following in http://www.squid-cache.org/Doc/config/pipeline_prefetch/ is misleading: If set to N, Squid will try to receive and process up to 1+N requests on the same connection concurrently. Note the concurrently. For older versions of squid, it is stated differently. Just a fix of the documentation. pipeline_prefetch has always been about concurrency/parallel processing even though it did not say so. All Squid will handle multiple requests on a persistent connection regardless of what pipeline_prefetch is set to. They are still received and responses delivered completely serially. HTTP/1.x requires that guarantee. pipeline_prefetch simply determines how many of the client requests can have been read in and not yet responded to. Many processing actions like parsing, validation, adaptation, cache lookup and sometimes even fetching from a fast server can be done on those requests entirely without having responded to the client. When pipeline_prefetch is enable Squid attempts to do what it can for each request while its waiting to be able to deliver the response. Amos
