Re: [squid-users] HTTP/1.1 support and Chunked
Regardt van de Vyver wrote: Adam Squids wrote: Hello, I am using Squid 2.5Build10, from what I understand squid does not fully support HTTP/1.1, not in this version, not in 2.6 and not in 3.0. Is this correct? I need to enable my requests to reach my backend servers in HTTP/1.1 so I'll be able to multiplex them. Another issue, I've tried to disable chunked in the http header via this workaround: acl broken dstdomain domain request_header_access Accept-Encoding deny broken but it failed. I am still getting encoding: Chucked. Thanks a million, Adam Hi Adam, The HTTP1.1 efforts are still a work in progress AFAIK - but those with more knowledge may correct me ;-) As to the chunked encoding ... squid 2.5 Build 10 almost certainly does not have support for the chunked bits. I'm sure Adrian will mention this ... but it's really recommended you run either 2.7stable5 or 3stable11 for testing/debugging support. Regardt Right on all counts. Though 2.7 has partial HTTP/1.1 enabled. What really matters though is which parts of HTTP/1.1 you need to 'multiplex'. Most HTTP/1.1 stuff is simply passed through current Squid or handled in newest Squid despite advertising 1.0. The lack in Squid is self-handling and modification of some things like 100-expect replies. But, If by multiplex you mean what I think you mean, then some cache_peer config and the accelerator improvements in current Squid should do that for you. What sort of topology and service are you trying to provide? Amos -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] transparent proxy not working!! any advice?
Hello, actually I have both of set on the lan interface ( am I mistaken to set the redirect out on the lan interface? should I be setting it on the interface facing the internet?) ip wccp 80 redirect in ip wccp 90 redirect out as for the wiki provided, I fail to see what's missing! obviously there is something, but I'm not detecting it! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Monday, January 05, 2009 12:46 AM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: Hello, the output of the debugging is as such: *Jan 4 23:16:43.205: WCCP-EVNT:D90: Here_I_Am packet from 192.168.0.183: service not active *Jan 4 23:16:43.205: WCCP-EVNT:D80: Here_I_Am packet from 192.168.0.183: service not active what service is that?! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Sunday, January 04, 2009 9:33 PM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: i've just created a new box with the following options: but wccp with router is still not working! any advice? using centos 5.2 and squid 2.6 firewall enabled SElinux permissive --- done the following: yum update yum yum install squid squid -z --- gedit /etc/rc.d/init.d/rc.local #added: modprobe ip_gre ifconfig gre0 192.168.0.183 netmask 255.255.255.0 up #this is the same ip as my eth0 gedit /etc/sysconfig/iptables #added: -A INPUT -i gre0 -j ACCEPT -A INPUT -i gre0 -j ACCEPT -A INPUT -p gre -j ACCEPT #my routers lan interface 192.168.0.1 -A RH-Firewall-1-INPUT -s 192.168.0.1/24 -p udp -m udp --dport 2048 -j ACCEPT --- service iptables condrestart gedit /etc/squid/squid.conf #edited/added the follwoing: http_port 80 transparent http_access allow all wccp2_router 192.168.0.1 wccp_version 4 wccp2_rebuild_wait on wccp2_forwarding_method 1 wccp2_return_method 1 wccp2_assignment_method 1 wccp2_service dynamic 80 wccp2_service dynamic 90 wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80 wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source priority=240 ports=80 -- Cisco router 2811 side: conf t ip wccp version 2 ip wccp web-cache int f0/1 (Lan interface) ip wccp 80 redirect in ip wccp 90 redirect out -- service squid restart then sh ip wccp on router gave me all hits as 0 no hits from squid to router!! -- service iptables status [r...@localhost ~]# service iptables status Table: filter Chain INPUT (policy ACCEPT) num target prot opt source destination 1RH-Firewall-1-INPUT all -- 0.0.0.0/00.0.0.0/0 2ACCEPT all -- 0.0.0.0/00.0.0.0/0 3ACCEPT all -- 0.0.0.0/00.0.0.0/0 4ACCEPT 47 -- 0.0.0.0/00.0.0.0/0 Chain FORWARD (policy ACCEPT) num target prot opt source destination 1RH-Firewall-1-INPUT all -- 0.0.0.0/00.0.0.0/0 Chain OUTPUT (policy ACCEPT) num target prot opt source destination Chain RH-Firewall-1-INPUT (2 references) num target prot opt source destination 1ACCEPT all -- 0.0.0.0/00.0.0.0/0 2ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 icmp type 255 3ACCEPT esp -- 0.0.0.0/00.0.0.0/0 4ACCEPT ah -- 0.0.0.0/00.0.0.0/0 5ACCEPT udp -- 0.0.0.0/0224.0.0.251 udp dpt:5353 6ACCEPT udp -- 0.0.0.0/00.0.0.0/0 udp dpt:631 7ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 tcp dpt:631 8ACCEPT all -- 0.0.0.0/00.0.0.0/0 state RELATED,ESTABLISHED 9ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 state NEW tcp dpt:22 10 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 state NEW tcp dpt:80 11 ACCEPT tcp -- 0.0.0.0/00.0.0.0/0 state NEW tcp dpt:5900 12 ACCEPT udp -- 192.168.0.0/24 0.0.0.0/0 udp dpt:2048 13 REJECT all -- 0.0.0.0/00.0.0.0/0 reject-with icmp-host-prohibited --- lsmod: Module Size Used by ip_conntrack_netbios_ns 6977 0 xt_state6209 4 ip_conntrack 53025 2 ip_conntrack_netbios_ns,xt_state nfnetlink 10713 1
Re: [squid-users] HTTP/1.1 support and Chunked
Adam Squids wrote: Hello, I am using Squid 2.5Build10, from what I understand squid does not fully support HTTP/1.1, not in this version, not in 2.6 and not in 3.0. Is this correct? I need to enable my requests to reach my backend servers in HTTP/1.1 so I'll be able to multiplex them. Another issue, I've tried to disable chunked in the http header via this workaround: acl broken dstdomain domain request_header_access Accept-Encoding deny broken but it failed. I am still getting encoding: Chucked. Thanks a million, Adam Hi Adam, The HTTP1.1 efforts are still a work in progress AFAIK - but those with more knowledge may correct me ;-) As to the chunked encoding ... squid 2.5 Build 10 almost certainly does not have support for the chunked bits. I'm sure Adrian will mention this ... but it's really recommended you run either 2.7stable5 or 3stable11 for testing/debugging support. Regardt
[squid-users] Trying to improve the Byte Hit Ratio, any tips ?
Hello, I'm trying to improve the Byte Hit Ratio of SquidCache on my network. There is 220 computers in the LAN, using internet on a general usage basis. The maximum bandwidth is 4Mbps in/out, the total amount of data is estimated to be 30 to 60 Gbytes daily. This is the report from cachemgr: = Average HTTP requests per minute since start: 1023.9 Average ICP messages per minute since start:0.0 Select loop called: 1208577 times, 5.619 ms avg Cache information for squid: Request Hit Ratios: 5min: 37.9%, 60min: 41.1% Byte Hit Ratios: 5min: 13.2%, 60min: 13.8% (It's quite low, these values are usual) Request Memory Hit Ratios: 5min: 2.0%, 60min: 2.6% (I rebooted the server 3 hours ago, this can explain these low values) Request Disk Hit Ratios:5min: 41.3%, 60min: 36.3% Storage Swap size: 27654312 KB Storage Mem size: 190364 KB Mean Object Size: 29.65 KB Requests given to unlinkd: 33035 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.23230 0.46965 Cache Misses: 0.35832 0.72387 Cache Hits:0.19742 0.35832 Near Hits: 0.20843 0.55240 Not-Modified Replies: 0.03829 0.05331 DNS Lookups: 0.00094 0.00779 ICP Queries: 0.0 0.0 = This is my squid.conf file: = http_port 3128 transparent hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? cache deny QUERY acl apache rep_header Server ^Apache broken_vary_encoding allow apache cache_mem 250 MB maximum_object_size 128 MB maximum_object_size_in_memory 50 KB cache_replacement_policy heap LFUDA cache_dir ufs /data/spool/squid 3 16 256 access_log none cache_log none cache_store_log none log_ip_on_direct off hosts_file /etc/hosts refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern . 0 20% 4320 quick_abort_min 0 KB quick_abort_max 0 KB range_offset_limit 0 KB half_closed_clients off shutdown_lifetime 0 seconds acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443 # https acl SSL_ports port 563 # snews acl SSL_ports port 873 # rsync acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl Safe_ports port 631 # cups acl Safe_ports port 873 # rsync acl Safe_ports port 901 # SWAT acl purge method PURGE acl CONNECT method CONNECT acl ReseauLocal src 10.0.0.0/16 http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access allow localhost http_access allow ReseauLocal http_access deny all http_reply_access allow all icp_access deny all cache_effective_group proxy httpd_suppress_version_string on via off forwarded_for off log_icp_queries off client_db off coredump_dir /var/spool/squid pipeline_prefetch off = Do you see something that need to be improved ? Did I miss something? Regards, Mr Lejeune
[squid-users] HTTP/1.1 support and Chunked
Hello, I am using Squid 2.5Build10, from what I understand squid does not fully support HTTP/1.1, not in this version, not in 2.6 and not in 3.0. Is this correct? I need to enable my requests to reach my backend servers in HTTP/1.1 so I'll be able to multiplex them. Another issue, I've tried to disable chunked in the http header via this workaround: acl broken dstdomain domain request_header_access Accept-Encoding deny broken but it failed. I am still getting encoding: Chucked. Thanks a million, Adam
Re: [squid-users] Squid multiple instances log problem
- Original Message From: John S uhimat...@hotmail.com To: squid-users@squid-cache.org Sent: Thursday, January 1, 2009 5:22:55 PM Subject: [squid-users] Squid multiple instances log problem I'm trying to run multiple instances of squid and I've followed the available directions. The second instance of squid throws an error when it tries to open it's access log file. Jan 1 10:32:12 desktop squid[15527]: Squid Parent: child process 15529 started Jan 1 10:32:12 desktop (squid): Cannot open '/var/log/squid3/accessSquid3HTTPProxy.log' for writing. ^IThe parent directory must be writeable by the ^Iuser 'proxy', which is the cache_effective_user ^Iset in squid.conf. Jan 1 10:32:12 desktop squid[15527]: Squid Parent: child process 15529 exited with status 1 Jan 1 10:32:42 desktop squid[15540]: Exiting due to repeated, frequent failures An ls -l /var/log/squid3/ shows, -rw-r- 1 proxy proxy 0 2009-01-01 07:56 access.log -rw-r- 1 proxy proxy 118302 2008-12-31 20:46 access.log.1 -rw-r- 1 proxy proxy 0 2009-01-01 10:29 accessSquid3HTTPProxy.log -rw-r--r-- 1 proxy proxy 6773 2009-01-01 10:11 cache.log -rw-r--r-- 1 proxy proxy 112239 2008-12-31 19:58 cache.log.1 -rw-r--r-- 1 proxy proxy 7005 2009-01-01 10:32 cacheSquid3HTTPProxy.log -rw-r- 1 proxy proxy603 2009-01-01 09:58 store.log -rw-r- 1 proxy proxy 154882 2009-01-01 06:58 store.log.1 Why can't it write to it's log file when the first instance can write to it's log file and starts up properly? Is the second squid also running as proxy/proxy? (cache_effective_user/cache_effective_group) Did you check the /var/log/squid3 permissions? If you stop/start the working squid, can it still open the log file? JD
RE: [squid-users] transparent proxy not working!! any advice?
The error on the Cisco router is stating that the squid box is trying to tell the router that it is able to service the wccp group 80 and 90, but for some reason the router does not see those groups as ones it is servicing. This is odd. Try doing the following in the router: ip access-list 180 permit any any ip wccp web-cache redirect-list 180 ip wccp 80 redirect-list 180 ip wccp 90 redirect-list 180 Is the squid box on the same router interface as the rest of the clients? If it is, you may need to add lines to the access-list 180, or put the squid box on the secondary interface of the router and do a ip wccp redirect exclude in statement on that interface. Which IOS feature set and version is this? WCCP is buggy in some IOS releases. From: r_o_l_a_...@hotmail.com [mailto:r_o_l_a_...@hotmail.com] Sent: Mon 1/5/2009 8:43 AM To: sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, actually I have both of set on the lan interface ( am I mistaken to set the redirect out on the lan interface? should I be setting it on the interface facing the internet?) ip wccp 80 redirect in ip wccp 90 redirect out as for the wiki provided, I fail to see what's missing! obviously there is something, but I'm not detecting it! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Monday, January 05, 2009 12:46 AM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: Hello, the output of the debugging is as such: *Jan 4 23:16:43.205: WCCP-EVNT:D90: Here_I_Am packet from 192.168.0.183: service not active *Jan 4 23:16:43.205: WCCP-EVNT:D80: Here_I_Am packet from 192.168.0.183: service not active what service is that?! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Sunday, January 04, 2009 9:33 PM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: i've just created a new box with the following options: but wccp with router is still not working! any advice? using centos 5.2 and squid 2.6 firewall enabled SElinux permissive --- done the following: yum update yum yum install squid squid -z --- gedit /etc/rc.d/init.d/rc.local #added: modprobe ip_gre ifconfig gre0 192.168.0.183 netmask 255.255.255.0 up #this is the same ip as my eth0 gedit /etc/sysconfig/iptables #added: -A INPUT -i gre0 -j ACCEPT -A INPUT -i gre0 -j ACCEPT -A INPUT -p gre -j ACCEPT #my routers lan interface 192.168.0.1 -A RH-Firewall-1-INPUT -s 192.168.0.1/24 -p udp -m udp --dport 2048 -j ACCEPT --- service iptables condrestart gedit /etc/squid/squid.conf #edited/added the follwoing: http_port 80 transparent http_access allow all wccp2_router 192.168.0.1 wccp_version 4 wccp2_rebuild_wait on wccp2_forwarding_method 1 wccp2_return_method 1 wccp2_assignment_method 1 wccp2_service dynamic 80 wccp2_service dynamic 90 wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80 wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source priority=240 ports=80 -- Cisco router 2811 side: conf t ip wccp version 2 ip wccp web-cache int f0/1 (Lan interface) ip wccp 80 redirect in ip wccp 90 redirect out -- service squid restart then sh ip wccp on router gave me all hits as 0 no hits from squid to router!! -- service iptables status [r...@localhost ~]# service iptables status Table: filter Chain INPUT (policy ACCEPT) num target prot opt source destination 1RH-Firewall-1-INPUT all -- 0.0.0.0/00.0.0.0/0 2ACCEPT all -- 0.0.0.0/00.0.0.0/0 3ACCEPT all -- 0.0.0.0/00.0.0.0/0 4ACCEPT 47 -- 0.0.0.0/00.0.0.0/0 Chain FORWARD (policy ACCEPT) num target prot opt source destination 1RH-Firewall-1-INPUT all -- 0.0.0.0/00.0.0.0/0 Chain OUTPUT (policy ACCEPT) num target prot opt source destination Chain RH-Firewall-1-INPUT (2 references) num target prot opt source destination 1ACCEPT all -- 0.0.0.0/00.0.0.0/0 2ACCEPT icmp -- 0.0.0.0/00.0.0.0/0 icmp type 255 3ACCEPT esp -- 0.0.0.0/00.0.0.0/0 4ACCEPT
RE: [squid-users] NTLM and transparent/interception confusion
Keep in mind, group policies cannot always be used as in our environment. We are a K-12 education and are mandated by federal law to monitor and protect student access to the internet. We are now allowing students to bring their own notebooks in on a trial basis (to be permanent after this summer when we work out the bugs) to do research on their own computers. We have to monitor their access to the internet and deny bad sites, again mandated by federal law. So their authentication mechanism is AD/LDAP to their user ID set up for them to access network resources on the network. Since their computers are not on our domain (nor do we want them to be), we cannot push group policies down to their computer. The solution Bluecoat had was very secure, but again their devices are about $50,000usd / device. As an education provider, that money is hard to come by especially when we would need 3 devices for the load. Their authentication mechanism is SOX (sarbane oxley) tested and compliant. It also works with any computer outbound to the internet. There's no proxy configuration to worry about; it's all done at the proxy. Granted, I used WCCP to configure this on Bluecoat which allowed me a lot of flexibility to add in multiple proxies with ease (and the users would never know the difference). sj -Original Message- From: Kinkie [mailto:gkin...@gmail.com] Sent: Saturday, January 03, 2009 12:51 PM To: Guido Serassio Cc: Johnson, S; squid-users@squid-cache.org Subject: Re: [squid-users] NTLM and transparent/interception confusion On Sat, Jan 3, 2009 at 11:14 AM, Guido Serassio guido.seras...@acmeconsulting.it wrote: Hi Kinkie, At 18.45 02/01/2009, Kinkie wrote: Could you try to get a network trace of a successfully authenticated http transaction? I would love to see how they do it... Websense too is using something similar for filtering: They maintain an IP Address/Username table on the policy server. The table can be populated using different ways: - A logon agent, a little executable running on every client at logon time - Direct query to the user workstation - A DC agent that query DCs for user sessions There isn't any kind of web browser authentication, and this solution cannot work with non Windows clients or machine non domain member. Multiuser terminal server environments cannot be supported and the WS policy server should be Windows based and domain member for full functionality. Yuck... IIRC Squid's session helper can do that too then. This is NOT authentication and it's absolutely insecure: even windows nowadays supports remote desktops (3 users can share one IP) and SNAT (connection sharing), and it's pretty easy to hijack an user's credentials (simply log on to his workstation as soon as possible after he's logged out). an nmblookup-based external authentication helper could be set up to do one of these, but after all what's the point? If the user has a proper Windows infrasctructure, it's much easier to use group policies to configure the browsers.. Thanks for the clarification Guido! -- /kinkie -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Re: [squid-users] transparent proxy not working!! any advice?
Hello, thanks for the advice ill proceed and add the new ACL. in the meantime, to answer your question yes Squid is on the same interface as all the other clients. what sort of entries should I add to tht access list? PS: my IOS is Version 12.4(17b), RELEASE SOFTWARE (fc2) Cisco 2811 (revision 53.51) -- From: Ritter, Nicholas nicholas.rit...@americantv.com Sent: Monday, January 05, 2009 9:23 PM To: r_o_l_a_...@hotmail.com; sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: RE: [squid-users] transparent proxy not working!! any advice? The error on the Cisco router is stating that the squid box is trying to tell the router that it is able to service the wccp group 80 and 90, but for some reason the router does not see those groups as ones it is servicing. This is odd. Try doing the following in the router: ip access-list 180 permit any any ip wccp web-cache redirect-list 180 ip wccp 80 redirect-list 180 ip wccp 90 redirect-list 180 Is the squid box on the same router interface as the rest of the clients? If it is, you may need to add lines to the access-list 180, or put the squid box on the secondary interface of the router and do a ip wccp redirect exclude in statement on that interface. Which IOS feature set and version is this? WCCP is buggy in some IOS releases. From: r_o_l_a_...@hotmail.com [mailto:r_o_l_a_...@hotmail.com] Sent: Mon 1/5/2009 8:43 AM To: sq...@vdvyver.net Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Hello, actually I have both of set on the lan interface ( am I mistaken to set the redirect out on the lan interface? should I be setting it on the interface facing the internet?) ip wccp 80 redirect in ip wccp 90 redirect out as for the wiki provided, I fail to see what's missing! obviously there is something, but I'm not detecting it! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Monday, January 05, 2009 12:46 AM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: Hello, the output of the debugging is as such: *Jan 4 23:16:43.205: WCCP-EVNT:D90: Here_I_Am packet from 192.168.0.183: service not active *Jan 4 23:16:43.205: WCCP-EVNT:D80: Here_I_Am packet from 192.168.0.183: service not active what service is that?! -- From: Regardt van de Vyver sq...@vdvyver.net Sent: Sunday, January 04, 2009 9:33 PM Cc: squid-users@squid-cache.org Subject: Re: [squid-users] transparent proxy not working!! any advice? Roland Roland wrote: i've just created a new box with the following options: but wccp with router is still not working! any advice? using centos 5.2 and squid 2.6 firewall enabled SElinux permissive --- done the following: yum update yum yum install squid squid -z --- gedit /etc/rc.d/init.d/rc.local #added: modprobe ip_gre ifconfig gre0 192.168.0.183 netmask 255.255.255.0 up #this is the same ip as my eth0 gedit /etc/sysconfig/iptables #added: -A INPUT -i gre0 -j ACCEPT -A INPUT -i gre0 -j ACCEPT -A INPUT -p gre -j ACCEPT #my routers lan interface 192.168.0.1 -A RH-Firewall-1-INPUT -s 192.168.0.1/24 -p udp -m udp --dport 2048 -j ACCEPT --- service iptables condrestart gedit /etc/squid/squid.conf #edited/added the follwoing: http_port 80 transparent http_access allow all wccp2_router 192.168.0.1 wccp_version 4 wccp2_rebuild_wait on wccp2_forwarding_method 1 wccp2_return_method 1 wccp2_assignment_method 1 wccp2_service dynamic 80 wccp2_service dynamic 90 wccp2_service_info 80 protocol=tcp flags=src_ip_hash priority=240 ports=80 wccp2_service_info 90 protocol=tcp flags=dst_ip_hash,ports_source priority=240 ports=80 -- Cisco router 2811 side: conf t ip wccp version 2 ip wccp web-cache int f0/1 (Lan interface) ip wccp 80 redirect in ip wccp 90 redirect out -- service squid restart then sh ip wccp on router gave me all hits as 0 no hits from squid to router!! -- service iptables status [r...@localhost ~]# service iptables status Table: filter Chain INPUT (policy ACCEPT) num target prot opt source destination 1RH-Firewall-1-INPUT all -- 0.0.0.0/00.0.0.0/0 2ACCEPT all -- 0.0.0.0/00.0.0.0/0 3ACCEPT all -- 0.0.0.0/00.0.0.0/0 4ACCEPT 47 -- 0.0.0.0/00.0.0.0/0 Chain FORWARD (policy ACCEPT)
Re: [squid-users] Squid consumes a lot dsk space
Where is the disk space going - the cache dir(s) or the logs? You can set the size of cache you wish to use in squid.conf; but if you are not mindful of your logs they can grow out of control as well. squid -k rotate will rotate your logs and allow you to clean up the old ones if you wish. Wilson Hernandez wrote: Hello again. I would like to fine tune squid so, that it won't cache so many things. I noticed that in less than a month a network with about 30 users consumed 65GB of harddrive. I don't think that's normal if it is please correct me.
RE: [squid-users] Squid consumes a lot dsk space
I'm surprised 30 users haven't consumed more than 65GB worth of internet in that amount of time :) Keep in mind that Squid will keep stale items in cache (not serving them of course) until it hits its threshold (default 90-something percent cache usage), because Squid doesn't want to waste time purging stale objects until necessary. See cache_swap_high/low parameters for more information on these thresholds. Also see http://wiki.squid-cache.org/SquidFaq/InnerWorkings#head-3ccaef79f36bf2d7 4c7cdde76eeb163b8c8e691e to learn about Squid's cache replacement algorithm. If you still want to fine-tune, I would recommend putting some profiling in place (see cachemgr or snmp) so you have a 'before' to compare against when making changes. http://wiki.squid-cache.org/SquidFaq/SquidProfiling -Original Message- From: Wilson Hernandez [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 12:52 PM To: squid-users@squid-cache.org Subject: [squid-users] Squid consumes a lot dsk space Hello again. I would like to fine tune squid so, that it won't cache so many things. I noticed that in less than a month a network with about 30 users consumed 65GB of harddrive. I don't think that's normal if it is please correct me.
[squid-users] Problem configure squid 3.1
Hello. Me again. It seems that everyhting I try to do can't go smoothly. Now, I'm trying to get squid-3.1.0.3 installed in my system trying to upgrade from an older version but now come accross a problem when I run ./configure I get the following error (I searched the internet but, can't get a solutions) : checking for C++ compiler default output file name... configure: error: C++ compiler cannot create executables See `config.log' for more details. configure: error: ./configure failed for lib/libTrie I removed the previous squid version which was installed as a package. Please help. Thanks.
RE: [squid-users] Problem configure squid 3.1
Sounds like you need a c++ compiler, do a 'apt-get gcc' (you're running debian IIRC) -Original Message- From: Wilson Hernandez [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 1:50 PM To: squid-users@squid-cache.org Subject: [squid-users] Problem configure squid 3.1 Hello. Me again. It seems that everyhting I try to do can't go smoothly. Now, I'm trying to get squid-3.1.0.3 installed in my system trying to upgrade from an older version but now come accross a problem when I run ./configure I get the following error (I searched the internet but, can't get a solutions) : checking for C++ compiler default output file name... configure: error: C++ compiler cannot create executables See `config.log' for more details. configure: error: ./configure failed for lib/libTrie I removed the previous squid version which was installed as a package. Please help. Thanks.
Re: [squid-users] Problem configure squid 3.1
I've already have it installed and still not working. Gregori Parker wrote: Sounds like you need a c++ compiler, do a 'apt-get gcc' (you're running debian IIRC) -Original Message- From: Wilson Hernandez [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 1:50 PM To: squid-users@squid-cache.org Subject: [squid-users] Problem configure squid 3.1 Hello. Me again. It seems that everyhting I try to do can't go smoothly. Now, I'm trying to get squid-3.1.0.3 installed in my system trying to upgrade from an older version but now come accross a problem when I run ./configure I get the following error (I searched the internet but, can't get a solutions) : checking for C++ compiler default output file name... configure: error: C++ compiler cannot create executables See `config.log' for more details. configure: error: ./configure failed for lib/libTrie I removed the previous squid version which was installed as a package. Please help. Thanks. -- *Wilson Hernandez* Presidente 829.848.9595 809.766.0441 www.msdrd.com http://www.msdrd.com Conservando el medio ambiente
RE: [squid-users] Problem configure squid 3.1
Try 'apt-get libc-dev' and report back -Original Message- From: Wilson Hernandez - MSD, S. A. [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 6:01 PM To: Gregori Parker Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Problem configure squid 3.1 I've already have it installed and still not working. Gregori Parker wrote: Sounds like you need a c++ compiler, do a 'apt-get gcc' (you're running debian IIRC) -Original Message- From: Wilson Hernandez [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 1:50 PM To: squid-users@squid-cache.org Subject: [squid-users] Problem configure squid 3.1 Hello. Me again. It seems that everyhting I try to do can't go smoothly. Now, I'm trying to get squid-3.1.0.3 installed in my system trying to upgrade from an older version but now come accross a problem when I run ./configure I get the following error (I searched the internet but, can't get a solutions) : checking for C++ compiler default output file name... configure: error: C++ compiler cannot create executables See `config.log' for more details. configure: error: ./configure failed for lib/libTrie I removed the previous squid version which was installed as a package. Please help. Thanks. -- *Wilson Hernandez* Presidente 829.848.9595 809.766.0441 www.msdrd.com http://www.msdrd.com Conservando el medio ambiente
RE: [squid-users] Problem configure squid 3.1
I'm sorry, I meant apt-get install libc-dev (I'm obviously not a Debian user) I've also read that you may need the 'build-essential' package as well, so you might want to try that -Original Message- From: Gregori Parker [mailto:gregori.par...@theplatform.com] Sent: Monday, January 05, 2009 4:33 PM To: w...@msdrd.com Cc: squid-users@squid-cache.org Subject: RE: [squid-users] Problem configure squid 3.1 Try 'apt-get libc-dev' and report back -Original Message- From: Wilson Hernandez - MSD, S. A. [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 6:01 PM To: Gregori Parker Cc: squid-users@squid-cache.org Subject: Re: [squid-users] Problem configure squid 3.1 I've already have it installed and still not working. Gregori Parker wrote: Sounds like you need a c++ compiler, do a 'apt-get gcc' (you're running debian IIRC) -Original Message- From: Wilson Hernandez [mailto:w...@msdrd.com] Sent: Monday, January 05, 2009 1:50 PM To: squid-users@squid-cache.org Subject: [squid-users] Problem configure squid 3.1 Hello. Me again. It seems that everyhting I try to do can't go smoothly. Now, I'm trying to get squid-3.1.0.3 installed in my system trying to upgrade from an older version but now come accross a problem when I run ./configure I get the following error (I searched the internet but, can't get a solutions) : checking for C++ compiler default output file name... configure: error: C++ compiler cannot create executables See `config.log' for more details. configure: error: ./configure failed for lib/libTrie I removed the previous squid version which was installed as a package. Please help. Thanks. -- *Wilson Hernandez* Presidente 829.848.9595 809.766.0441 www.msdrd.com http://www.msdrd.com Conservando el medio ambiente
RE: [squid-users] Handling websites that switch between http https
You've just reminded me of the hotmail problems... Joseph: see if it disappears when you turn balance_on_multiple_ip off. It still defaults to on in most Squid installs. Amos, Still no luck, if it matters I am on the upstream packaged 2.6 stable 5 from RH. If I moved that up to a more recent version do you think this issue might be handled better? Thanks for all the help guys! jlc
RE: [squid-users] Handling websites that switch between http https
Amos, Still no luck, if it matters I am on the upstream packaged 2.6 stable 5 from RH. If I moved that up to a more recent version do you think this issue might be handled better? Before I even started this thread, I had removed the url_rewrite_program reference to squidguard as I assumed that was the issue and it never made a difference but I must have done something wrong as I just double checked that testing squid-3.0.STABLE9-1.el5 versus squid-2.6.STABLE5-1.el5 and it is absolutely working when squidguard is disabled. So sorry for the noise guys... jlc
[squid-users] Configuration Change
When editing squid.conf is not sufficient to restart the squid service to enact changes, or does one need to execute squid -k reconfigure always as well? Thanks! jlc
Re: [squid-users] NTLM and transparent/interception confusion
Johnson, S wrote: Keep in mind, group policies cannot always be used as in our environment. We are a K-12 education and are mandated by federal law to monitor and protect student access to the internet. We are now allowing students to bring their own notebooks in on a trial basis (to be permanent after this summer when we work out the bugs) to do research on their own computers. We have to monitor their access to the internet and deny bad sites, again mandated by federal law. So their authentication mechanism is AD/LDAP to their user ID set up for them to access network resources on the network. Since their computers are not on our domain (nor do we want them to be), we cannot push group policies down to their computer. In that case your best bet would be to lock down general port-80 access to them entirely. Using WPAD 'auto-detect' or with students setting browsers set manually. That will go a long way toward blocking risky behavior by malware on mobile devices. Second best after that would be to setup some helper where they can authenticate against some other system and the helper permits their requests past Squid for a time. This provides almost no protection from malware once the student is browsing a legit session. Amos The solution Bluecoat had was very secure, but again their devices are about $50,000usd / device. As an education provider, that money is hard to come by especially when we would need 3 devices for the load. Their authentication mechanism is SOX (sarbane oxley) tested and compliant. It also works with any computer outbound to the internet. There's no proxy configuration to worry about; it's all done at the proxy. Granted, I used WCCP to configure this on Bluecoat which allowed me a lot of flexibility to add in multiple proxies with ease (and the users would never know the difference). sj -Original Message- From: Kinkie [mailto:gkin...@gmail.com] Sent: Saturday, January 03, 2009 12:51 PM To: Guido Serassio Cc: Johnson, S; squid-users@squid-cache.org Subject: Re: [squid-users] NTLM and transparent/interception confusion On Sat, Jan 3, 2009 at 11:14 AM, Guido Serassio guido.seras...@acmeconsulting.it wrote: Hi Kinkie, At 18.45 02/01/2009, Kinkie wrote: Could you try to get a network trace of a successfully authenticated http transaction? I would love to see how they do it... Websense too is using something similar for filtering: They maintain an IP Address/Username table on the policy server. The table can be populated using different ways: - A logon agent, a little executable running on every client at logon time - Direct query to the user workstation - A DC agent that query DCs for user sessions There isn't any kind of web browser authentication, and this solution cannot work with non Windows clients or machine non domain member. Multiuser terminal server environments cannot be supported and the WS policy server should be Windows based and domain member for full functionality. Yuck... IIRC Squid's session helper can do that too then. This is NOT authentication and it's absolutely insecure: even windows nowadays supports remote desktops (3 users can share one IP) and SNAT (connection sharing), and it's pretty easy to hijack an user's credentials (simply log on to his workstation as soon as possible after he's logged out). an nmblookup-based external authentication helper could be set up to do one of these, but after all what's the point? If the user has a proper Windows infrasctructure, it's much easier to use group policies to configure the browsers.. Thanks for the clarification Guido! -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] Configuration Change
Joseph L. Casale wrote: When editing squid.conf is not sufficient to restart the squid service to enact changes, or does one need to execute squid -k reconfigure always as well? Thanks! jlc Most things alter with only reconfigure. Some like Auth though need a full restart (stop + start) Amos -- Please be using Current Stable Squid 2.7.STABLE5 or 3.0.STABLE11 Current Beta Squid 3.1.0.3
Re: [squid-users] Trying to improve the Byte Hit Ratio, any tips ?
Vianney Lejeune wrote: Hello, I'm trying to improve the Byte Hit Ratio of SquidCache on my network. There is 220 computers in the LAN, using internet on a general usage basis. The maximum bandwidth is 4Mbps in/out, the total amount of data is estimated to be 30 to 60 Gbytes daily. This is the report from cachemgr: = Average HTTP requests per minute since start:1023.9 Average ICP messages per minute since start:0.0 Select loop called: 1208577 times, 5.619 ms avg Cache information for squid: Request Hit Ratios:5min: 37.9%, 60min: 41.1% Byte Hit Ratios:5min: 13.2%, 60min: 13.8% (It's quite low, these values are usual) Request Memory Hit Ratios:5min: 2.0%, 60min: 2.6% (I rebooted the server 3 hours ago, this can explain these low values) Request Disk Hit Ratios:5min: 41.3%, 60min: 36.3% Storage Swap size:27654312 KB Storage Mem size:190364 KB Mean Object Size:29.65 KB Requests given to unlinkd:33035 Median Service Times (seconds) 5 min60 min: HTTP Requests (All): 0.23230 0.46965 Cache Misses: 0.35832 0.72387 Cache Hits:0.19742 0.35832 Near Hits: 0.20843 0.55240 Not-Modified Replies: 0.03829 0.05331 DNS Lookups: 0.00094 0.00779 ICP Queries: 0.0 0.0 = This is my squid.conf file: = http_port 3128 transparent hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? cache deny QUERY Without cache peers you can drop the above QEURY acl. That will raise both hit ratios on semi-dynamic objects. BUT, see addition to refresh_pattern below... acl apache rep_header Server ^Apache broken_vary_encoding allow apache maximum_object_size 128 MB Re: the above maximum. There may be huge objects going through that can be cached. cache_mem 250 MB maximum_object_size_in_memory 50 KB memory, memory, memory. The more you can throw at the problem the more objects can be kept and served while hot. Squid with 64-bit can easily handle many GBs of memory cache. (at cost of slow shutdown when it saves the hottest to disk for the next round.) cache_replacement_policy heap LFUDA Been a while since I looked at these, to maximize bytes you want the policy that looks at object size as well as 'coldness'. To remove the smaller cool objects before the larger equally cool ones. cache_dir ufs /data/spool/squid 3 16 256 Your cache dir is only 30GB. Thats one days traffic or less by your above statements. For good hit ratios you may need at least 7 days, preferrably as close to 30 as possible. Depending on your OS, AUFS(Linux) or diskd(*BSD) may prove much faster access than UFS. access_log none cache_log none The above is generating log file named none. Would be more useful to set debug_options ALL,0. If you really don't want to know about the critical problems that do happen then set filename to /dev/null as well. cache_store_log none log_ip_on_direct off hosts_file /etc/hosts refresh_pattern ^ftp:144020%10080 refresh_pattern ^gopher:14400%1440 without QUERY acl above, you wil need this right here in the pattern order: refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 refresh_pattern .020%4320 quick_abort_min 0 KB quick_abort_max 0 KB range_offset_limit 0 KB Be careful, but you may want to play at setting these to continue downloads. (quick_abort -1 KB) That will cause all partial and restarted downloads to become HIT later. At risk of some wastage. half_closed_clients off shutdown_lifetime 0 seconds acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl to_localhost dst 127.0.0.0/8 acl SSL_ports port 443# https acl SSL_ports port 563# snews acl SSL_ports port 873# rsync acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443# https acl Safe_ports port 70# gopher acl Safe_ports port 210# wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280# http-mgmt acl Safe_ports port 488# gss-http acl Safe_ports port 591# filemaker acl Safe_ports port 777# multiling http acl Safe_ports port 631# cups acl Safe_ports port 873# rsync acl Safe_ports port 901# SWAT acl purge method PURGE acl CONNECT method CONNECT acl ReseauLocal src 10.0.0.0/16 http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access allow localhost http_access allow ReseauLocal http_access deny all http_reply_access allow all icp_access deny all cache_effective_group proxy httpd_suppress_version_string on via off forwarded_for off log_icp_queries off client_db off coredump_dir /var/spool/squid pipeline_prefetch off = Do you see something that need to be improved ? Did I miss something? Theres
