[squid-users] ICAP service squid 3.3.8
Hi, i am using squid 3.3.8 on Centos 6.4, i am also using qlproxy. Lately seeing these errors every few minutes? any ideas on what can cause this? suspending ICAP service for too many failures essential ICAP service is suspended:icap://127.0.0.1:1344/mw[down,susp,fail11]
Re: [squid-users] Basic questions on transparent/intercept proxy
- Original Message - From: csn233 csn...@gmail.com To: squid-users@squid-cache.org squid-users@squid-cache.org To intercept HTTPS traffic, is SSL-bump a must? Even when I only want to record the CONNECT traffic in access.log just like a normal forward proxy without decrypting anything? No. But it will log only IPs not the host name or URL. Amm
[squid-users] Squid monitoring, access report shows upto 5 % to 7 % cache usage
Hi All For purpose of convincing about the squid cahce, I installed and started monitoring SQUID using mysar http://sourceforge.net/projects/mysar/; Is this % too low or average how much should a optimized squid setup give. Or should I dig into some more configuration to pump the cache % thanks Joseph John
Re: [squid-users] Squid monitoring, access report shows upto 5 % to 7 % cache usage
On Monday 29 July 2013 at 12:36:05, John Joseph wrote: Hi All For purpose of convincing about the squid cahce, I installed and started monitoring SQUID using mysar http://sourceforge.net/projects/mysar/; Is this % too low or average how much should a optimized squid setup give. Or should I dig into some more configuration to pump the cache % How large is your cache? How many users are going via Squid? How long have they been using it (how long has Squid been building up its cache)? How many connection requests do you have per minute/hour/day/whatever makes sense? How many cache HITs do you see in the log file compared to MISSes (ie: how often are cached objects being requested)? Any other relevant information you can supply might be helpful in providing an answer. Regards, Antony. -- Users don't know what they want until they see what they get. Please reply to the list; please don't CC me.
Re: [squid-users] ICAP service squid 3.3.8
On 07/29/2013 12:22 PM, JC Putter wrote: Hi, i am using squid 3.3.8 on Centos 6.4, i am also using qlproxy. Lately seeing these errors every few minutes? any ideas on what can cause this? suspending ICAP service for too many failures essential ICAP service is suspended:icap://127.0.0.1:1344/mw[down,susp,fail11] What ICAP Service and is it on VM or PM? also is it a 64 bit or 32 bit system? can you give the cache.log output so we can see the problem? Eliezer
[squid-users] 3.4.0.1 Works great.
the basic test that result in: 1375097944.775551 127.0.0.1 TCP_REFRESH_UNMODIFIED/304 355 GET http://www.squid-cache.org/Images/img7.gif - HIER_DIRECT/209.169.10.131 - 1375097950.082763 127.0.0.1 TCP_CLIENT_REFRESH_MISS/200 10345 GET http://www.squid-cache.org/Versions/v3/3.4/ - HIER_DIRECT/209.169.10.131 text/html Shows that the client did an active refresh. All the above is without refresh_patterns at all. Also works on youtube images and other stuff pretty nice.. Youtube videos are pretty large and for me it seems like a weird thing to cache in a cache since they have servers all over the globe doing cache helping the clients view the video nicely.. 1375098872.573 72 127.0.0.1 TCP_REFRESH_UNMODIFIED/304 212 GET http://i1.ytimg.com/vi/HbjUK7i84YQ/default.jpg - HIER_DIRECT/212.199.219.236 - 1375098872.944 0 127.0.0.1 TCP_MEM_HIT/200 628 GET http://r5---sn-nhpax-ua8e.c.youtube.com/crossdomain.xml - HIER_NONE/- text/x-cross-domain-policy 1375098873.084141 127.0.0.1 TCP_MISS/204 424 GET http://s.youtube.com/stream_204? - HIER_DIRECT/212.199.205.241 text/html Eliezer
[squid-users] MX issues? (was Re: Basic questions on transparent/intercept proxy)
Is there some issue with mailing list? (I am assuming Yahoo! mail would not have issue) My past two-three e-mails were delivered much late to list. One even bounced back. (which I resent) The one below was delivered after more than 24hrs or so. It appears only one MX is working. squid-cache.org mail exchanger = 10 squid-cache.org. squid-cache.org mail exchanger = 90 mx2.squid-cache.org. mx2 does not seem to be working. Regards, Amm. - Original Message - From: Amm ammdispose-sq...@yahoo.com To: squid-users@squid-cache.org squid-users@squid-cache.org Cc: Sent: Sunday, 28 July 2013 6:41 PM Subject: Re: [squid-users] Basic questions on transparent/intercept proxy - Original Message - From: csn233 csn...@gmail.com To: squid-users@squid-cache.org squid-users@squid-cache.org To intercept HTTPS traffic, is SSL-bump a must? Even when I only want to record the CONNECT traffic in access.log just like a normal forward proxy without decrypting anything? No. But it will log only IPs not the host name or URL. Amm
[squid-users] squid cache-server misses all
hi guys, I have squid-3.1.20 installed on ubuntu-12.10. I didn't change much in squid's default configurations. I just defined cache_dir and nameservers and enabled localnet. now when I open any webpages from client, in my access.log I see lots and lots of this message (for almost all contents): ... 1375088677.969 3341 192.168.0.81 TCP_MISS/200 111920 GET http://www.imdb.com/ - DIRECT/72.21.215.52 text/html ... 1375088683.295 479 192.168.0.81 TCP_MISS/200 648 GET http://s.media-imdb.com/twilight/? - DIRECT/72.21.215.52 image/gif I look at squid's cache folder and I see size of cache folder grows (so content is being cached?) but apparently no cached content is being used. I googled but didn't find anything that help me. is there anything I'm missing? Does anyone have a thought or suggestion? thank you.
Re: [squid-users] 3.4.0.1 Works great.
I just realized that my squid.conf for 3.3.8 is unusable with 3.4.0.1
just because of the refresh patterns. Funnily, even the default
refresh patterns are seen as bungled. Same config is usable with
3.HEAD-20130717-r12946
Here are mine, which are ALL rejected as bungled - all commented out,
but still even the default ones are 'bungled'.
# various windows versions
#!refresh_pattern http://.*\.windowsupdate\.microsoft\.com/ 0 80% 20160
#!refresh_pattern http://.*\.update\.microsoft\.com/ 0 80% 20160
#!refresh_pattern http://download\.microsoft\.com/ 0 80% 20160
#!refresh_pattern http://windowsupdate\.microsoft\.com/ 0 80% 20160
#!refresh_pattern http://office\.microsoft\.com/ 0 80% 20160
#!refresh_pattern http://w?xpsp[0-9]\.microsoft\.com/ 0 80% 20160
#!refresh_pattern http://w2ksp[0-9]\.microsoft\.com/ 0 80% 20160
#!refresh_pattern -i
microsoft.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 90%
43200
#!refresh_pattern -i
windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|asf|wm[v|a]|dat|zip) 4320 90%
43200
#refresh_pattern windowsupdate.com/.*\.(cab|exe|dll|msi) 10080 100% 43200
#refresh_pattern download.microsoft.com/.*\.(cab|exe|dll|msi) 10080 100% 43200
#refresh_pattern www.microsoft.com/.*\.(cab|exe|dll|msi) 10080 100% 43200
#refresh_pattern au.download.windowsupdate.com/.*\.(cab|exe|dll|msi)
4320 100% 43200
# and some other windows updaters
#!refresh_pattern http://download\.macromedia\.com/ 0 80% 20160
#!refresh_pattern ftp://ftp\.nai\.com/ 0 80% 20160
#!refresh_pattern http://ftp\.software\.ibm\.com/ 0 80% 20160
# Others
#!refresh_pattern -i \.mp4 1440 90% 43200
#!refresh_pattern -i \.(gif|png|jpg|jpeg|ico)$ 40320 75% 86400
#!refresh_pattern -i \.(iso|avi|wav|mp3|mpeg|swf|flv|x-flv)$ 1440 40% 40320
#!refresh_pattern -i \.(css|js)$ 300 40% 7200
#!refresh_pattern -i \.(html|htm)$ 300 40% 7200
And the default ones:
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 144020% 10080
refresh_pattern ^gopher:14400% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
[root@jaribu] /opt/squid34/etc# ../sbin/squid -v
Squid Cache: Version 3.4.0.1
configure options: '--prefix=/opt/squid34'
'--enable-removal-policies=lru heap' '--disable-epoll' '--enable-auth'
'--enable-auth-basic=DB NCSA PAM MSNT PAM POP3 SMB SSPI MSNT'
'--enable-external-acl-helpers=session unix_group wbinfo_group
file_userip' '--enable-auth-ntlm=smb_lm SSPI'
'--enable-auth-negotiate=SSPI kerberos' '--with-pthreads'
'--enable-storeio=ufs diskd aufs' '--enable-delay-pools'
'--enable-snmp' '--with-openssl=/usr' '--enable-forw-via-db'
'--enable-cache-digests' '--enable-wccpv2'
'--enable-follow-x-forwarded-for' '--with-large-files'
'--enable-large-cache-files' '--enable-error_default_language=English'
'--enable-esi' '--enable-kqueue' '--enable-icap-client'
'--enable-kill-parent-hack' '--enable-ssl' '--enable-leakfinder'
'--enable-ssl-crtd' '--enable-url-rewrite-helpers'
'--enable-xmalloc-statistics' '--enable-stacktraces'
'--enable-zph-qos' '--enable-eui' '--enable-pf-transparent'
'--enable-ipf-transparent' --enable-ltdl-convenience
[root@jaribu] /opt/squid34/etc# ../sbin/squid -k parse
2013/07/29 18:35:04| Startup: Initializing Authentication Schemes ...
2013/07/29 18:35:04| Startup: Initialized Authentication Scheme 'basic'
2013/07/29 18:35:04| Startup: Initialized Authentication Scheme 'digest'
2013/07/29 18:35:04| Startup: Initialized Authentication Scheme 'negotiate'
2013/07/29 18:35:04| Startup: Initialized Authentication Scheme 'ntlm'
2013/07/29 18:35:04| Startup: Initialized Authentication.
2013/07/29 18:35:04| Processing Configuration File:
/opt/squid34/etc/squid.conf (depth 0)
2013/07/29 18:35:04| Processing: http_port 13128
2013/07/29 18:35:04| Processing: icp_port 3130
2013/07/29 18:35:04| Processing: icp_query_timeout 0
2013/07/29 18:35:04| Processing: mcast_icp_query_timeout 2000
2013/07/29 18:35:04| Processing: dead_peer_timeout 10 seconds
2013/07/29 18:35:04| Processing: cache_mem 128 MB
2013/07/29 18:35:04| Processing: maximum_object_size 4194240 KB
2013/07/29 18:35:04| Processing: maximum_object_size_in_memory 1024 KB
2013/07/29 18:35:04| Processing: balance_on_multiple_ip off
2013/07/29 18:35:04| Processing: logformat combined %a %ui %un [%tl]
%rm %ru HTTP/%rv %Hs %st%{Referer}h %{User-Agent}h %Ss:%Sh
2013/07/29 18:35:04| Processing: cache_dir aufs /usr/local/squid/cache
5120 16 256
2013/07/29 18:35:04| Processing: access_log
stdio:/usr/local/squid/logs/access.log squid
2013/07/29 18:35:04| Processing: cache_log
/usr/local/squid/logs/cache.log squid
2013/07/29 18:35:04| Processing: cache_store_log none
2013/07/29 18:35:04| Processing: coredump_dir /usr/local/squid/logs
2013/07/29 18:35:04| Processing: pid_filename /usr/local/squid/logs/squid.pid
2013/07/29 18:35:04| Processing: refresh_pattern^ftp:
144020% 10080
2013/07/29 18:35:04| ERROR: A percentage value is missing.
Re: [squid-users] Basic questions on transparent/intercept proxy
On Sun, Jul 28, 2013 at 9:11 PM, Amm ammdispose-sq...@yahoo.com wrote: - Original Message - From: csn233 csn...@gmail.com To: squid-users@squid-cache.org squid-users@squid-cache.org To intercept HTTPS traffic, is SSL-bump a must? Even when I only want to record the CONNECT traffic in access.log just like a normal forward proxy without decrypting anything? No. But it will log only IPs not the host name or URL. Amm No, as in ssl-bump is not a requirement for HTTPS traffic to be logged? Your answer seems to be different from other replies. Can you provide examples of how?
Re: [squid-users] squid cache-server misses all
On 30/07/2013 1:49 a.m., ana any wrote: hi guys, I have squid-3.1.20 installed on ubuntu-12.10. I didn't change much in squid's default configurations. I just defined cache_dir and nameservers and enabled localnet. now when I open any webpages from client, in my access.log I see lots and lots of this message (for almost all contents): ... 1375088677.969 3341 192.168.0.81 TCP_MISS/200 111920 GET http://www.imdb.com/ - DIRECT/72.21.215.52 text/html ... 1375088683.295479 192.168.0.81 TCP_MISS/200 648 GET http://s.media-imdb.com/twilight/? - DIRECT/72.21.215.52 image/gif I look at squid's cache folder and I see size of cache folder grows (so content is being cached?) but apparently no cached content is being used. In direct accordance with IMDB cache control rules about what may be done with their website. The first of those URLs presents an index page containing proprietary and copyright infromation under a limited license. It also contains embeded user-specific content. It is marked as *private* information for the individual client and may not be stored by shared caches such as Squid. The second URL is unclear exactly since you have cropped off all the query-string portion which would aid us in identifying the exact problem. Other similar URLs are marked with no-cache, must-revalidate and an Expiry timestamp long in the past. To cause HTTP/1.0 software (such as squid-3.1) not to cache or to revalidate on every use, and HTTP/1.1 compliant softwrae (such as squid-3.2 and later) to cache it but revalidate before any use. I googled but didn't find anything that help me. is there anything I'm missing? Does anyone have a thought or suggestion? Best thing to do is upgrade to 3.2 or 3.3. There are refresh_pattern directive options which can fine tune behavioru for unfriendly site. However I don't advise doing anything with those particular URLs due to the embeded user-specific content IMDB adds to their pages (you do not want to be responsible for your users getting each others IMDB account details or notices). If the problem is more widespread in other sites though you may want to share your whole config file (without the # comment lines) and we may be able to find something to help. Amos
Re: [squid-users] Basic questions on transparent/intercept proxy
From: csn233 csn...@gmail.com Sent: Monday, 29 July 2013 10:40 PM Subject: Re: [squid-users] Basic questions on transparent/intercept proxy On Sun, Jul 28, 2013 at 9:11 PM, Amm ammdispose-sq...@yahoo.com wrote: - Original Message - From: csn233 csn...@gmail.com To: squid-users@squid-cache.org squid-users@squid-cache.org To intercept HTTPS traffic, is SSL-bump a must? Even when I only want to record the CONNECT traffic in access.log just like a normal forward proxy without decrypting anything? No. But it will log only IPs not the host name or URL. Amm No, as in ssl-bump is not a requirement for HTTPS traffic to be logged? Your answer seems to be different from other replies. Can you provide examples of how? I am not sure if I understood your previous question right. I think what others said is right. Here is what I have done. (simplified version) https_port 8081 intercept ssl-bump generate-host-certificates=on cert=/etc/squid/ssl_cert/squid.pem #ssl_bump none all #--- this line is not required So ssl-bump as a keyword is required on https_port but you dont need ssl_bump ACL line (by default it bumps nothing). Traffic will be logged just as IP. (Not actual hostname) Regards, Amm.
