RE: [squid-users] 100% CPU Load problem with squid 3.3.8
All workarounds failed except adding ulimit -n 65000 to squid init file Adding session required pam_limits.so to /etc/pam.d/common-session also failed for me. The box never read '/etc/security/limits.conf' at boot time OK so now there is another thing That I have tested: /etc/pam.d/common-session dosn't have the limit module as a default so the admin will set it as he wants and to prevent a problem.. adding this line: session required pam_limits.so to the common-session file forces the ulimits on a PAM session startup and end.. this forces the bash(which is a pam) session to use the limits that are set by the admin in the limits.conf... It's not such a good idea to allow a users such a thing but this is the admin choice. Eliezer
[squid-users] WCCP issues with Centos 6.3 and Cisco 2901
Hi Squid community, I have an issue whereby I am just struggling to find out why it wont work. I have trawled through multiple forums, howto's, faq's etc but no matter what I do, I cannot get it to work properly. Here is what I have done so far: Router IP: 10.114.3.34 Squid IP: 10.112.4.4 WAN Subnet: 10.112.0.0 / 255.252.0.0 Squid Config: http_port 3127 intercept wccp2_router 10.114.3.34 wccp2_forwarding_method gre wccp2_return_method gre wccp2_service standard 0 Confirm I can access and use port 3127 directly without issue from any location in the WAN. Router Config: ip wccp web-cache interface G0/1 !Inside interface ip wccp web-cache redirect in Added to sysctl.conf: # Controls IP packet forwarding net.ipv4.ip_forward = 1 # Controls source route verification net.ipv4.conf.default.rp_filter = 0 net.ipv4.conf.eth0.rp_filter = 0 net.ipv4.conf.eth0.ip_filter = 0 net.ipv4.conf.gre0.rp_filter = 0 net.ipv4.conf.gre0.ip_filter = 0 Added to /etc/sysconfig/network-scripts/ifcfg-gre0 DEVICE=gre0 BOOTPROTO=static IPADDR=127.0.0.2 NETMASK=255.255.255.0 ONBOOT=YES IPV6INIT=NO Linux Configuration: modprobe ip_gre ifup gre0 iptables -t nat -F iptables -t nat -A PREROUTING -i gre0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 10.112.4.4:3127 If, I then do a tcpdump -i gre0 I can see packets flowing through this interface with destination port 80. Unfortunately it seems as if they are somehow not being natted to the squid server. I've tried different varying methods of doing this, but none of them seem to work. Does anyone have any ideas? Regards, Jordan.
Re: [squid-users] Disable -Werror when building Squid
On Sat, Sep 14, 2013 at 11:59 PM, Amos Jeffries squ...@treenet.co.nz wrote: On 14/09/2013 6:28 a.m., Jeffrey Walton wrote: I'm trying to get analysis tools on squid, but I'm having trouble due to -Werror. -Werror is causing a continuous stream of compile and link failures. How does one disable -Werror? INSTALL does not offer any directions or mention any options. Please start with reporting the warning that is coming out of the compiler. Things will only get worse unless the developers know it needs fixing. You can use possibly use --disable-error-checking if it is a warning being escalated. Thanks Amos. configuring with `--disable-error-checking` has the undesirable effect of suppressing warnings (i.e., removing -Wall). I want want the opposite: keep -Wall, but remove -Werror because it stops compilation on the first warning treated as an error.
Re: [squid-users] 100% CPU Load problem with squid 3.3.8
What?? what OS are you using? Eliezer On 09/15/2013 09:07 AM, Mohsen Dehghani wrote: All workarounds failed except adding ulimit -n 65000 to squid init file Adding session required pam_limits.so to /etc/pam.d/common-session also failed for me. The box never read '/etc/security/limits.conf' at boot time OK so now there is another thing That I have tested: /etc/pam.d/common-session dosn't have the limit module as a default so the admin will set it as he wants and to prevent a problem.. adding this line: session required pam_limits.so to the common-session file forces the ulimits on a PAM session startup and end.. this forces the bash(which is a pam) session to use the limits that are set by the admin in the limits.conf... It's not such a good idea to allow a users such a thing but this is the admin choice. Eliezer
Re: [squid-users] Re: squid 3.2.0.14 with TPROXY = commBind: Cannot bind socket FD 773 to xxx.xxx.xxx.xx: (98) Address
On Sat, Sep 14, 2013 at 11:59 PM, Eliezer Croitoru elie...@ngtech.co.il wrote: OK so let's make this experience that you already have as a public resource.. here it is: a simple php script that demonstrates the issue: https://gist.github.com/ngorchilov/6570413#file-s-php This way more then just you will have the option to look at it and understand the real and the main issue. on what OS again this test was done? Different versions of Ubuntu between 10.04 - 12.04. I will compare couple of them to make sure what kernel are we talking about... I believe this problem exists since long, long time.
Re: [squid-users] Re: squid 3.2.0.14 with TPROXY = commBind: Cannot bind socket FD 773 to xxx.xxx.xxx.xx: (98) Address
On Sun, Sep 15, 2013 at 12:52 AM, Eliezer Croitoru elie...@ngtech.co.il wrote: I have found the problem and I will rephrase the problem description: While using tproxy the main issue is that the ports of the source IP is NOPE. As I said before, it's NOT related to TPROXY code at all. Same problem exists, even when you try to bind with 2+ local IPs. Check both scenarios with my test script provided above. beeing decreased to half for the same pair of ip:Xport to ip:Xport. Which means that 192.168.1.1 cannot connect like regular proxy to 65k ports but to 32k ports which makes IP cheaper. it's the same for server and client both.. While using the port range of: # cat /proc/sys/net/ipv4/ip_local_port_range 32768 32867 #end the main issue is that the OS tries to bind using a 0 value maximum ports per IP by the above mentioned value. Let me rephrase the issue. With the above config (100 ports allowed for auto-selection) the maximum number of ports you can assign is exactly 100. But it has to be n*100, where n is the number of IPs you use (either local or remote with TPROXY) the kernel itself wont even try to bind an already binded ip+port so there is no need for the upper layers of the user-land to recover from such a state. leaving these matters to the kernel level is much more appropriate from any aspect you look at the OS. That's for sure. The problem is that I don't believe the kernel guys will fix this issue soon. So we have to adapt on application layer. Niki
Re: [squid-users] 100% CPU Load problem with squid 3.3.8
I got the same result as Mohsen. The only thing that worked was adding ulimit -n mynumber to the init script. It was weird for me, because the script is run by root, not the squid user, and i thought ulimit -n applied only to the current logged in user. But I think it applies to any session that will start later. But at boot time, seems like PAM has no effect. I'm using RHEL with SELinux. Maybe it is a SELinux behavior... On Sun, Sep 15, 2013 at 8:14 AM, Eliezer Croitoru elie...@ngtech.co.il wrote: What?? what OS are you using? Eliezer On 09/15/2013 09:07 AM, Mohsen Dehghani wrote: All workarounds failed except adding ulimit -n 65000 to squid init file Adding session required pam_limits.so to /etc/pam.d/common-session also failed for me. The box never read '/etc/security/limits.conf' at boot time OK so now there is another thing That I have tested: /etc/pam.d/common-session dosn't have the limit module as a default so the admin will set it as he wants and to prevent a problem.. adding this line: session required pam_limits.so to the common-session file forces the ulimits on a PAM session startup and end.. this forces the bash(which is a pam) session to use the limits that are set by the admin in the limits.conf... It's not such a good idea to allow a users such a thing but this is the admin choice. Eliezer
Re: [squid-users] 100% CPU Load problem with squid 3.3.8
On 09/15/2013 03:51 PM, Carlos Defoe wrote: I got the same result as Mohsen. The only thing that worked was adding ulimit -n mynumber to the init script. It was weird for me, because the script is run by root, not the squid user, and i thought ulimit -n applied only to the current logged in user. But I think it applies to any session that will start later. But at boot time, seems like PAM has no effect. I'm using RHEL with SELinux. Maybe it is a SELinux behavior... Or this is how it was designed.. Eliezer On Sun, Sep 15, 2013 at 8:14 AM, Eliezer Croitoru elie...@ngtech.co.il wrote: What?? what OS are you using? Eliezer On 09/15/2013 09:07 AM, Mohsen Dehghani wrote: All workarounds failed except adding ulimit -n 65000 to squid init file Adding session required pam_limits.so to /etc/pam.d/common-session also failed for me. The box never read '/etc/security/limits.conf' at boot time OK so now there is another thing That I have tested: /etc/pam.d/common-session dosn't have the limit module as a default so the admin will set it as he wants and to prevent a problem.. adding this line: session required pam_limits.so to the common-session file forces the ulimits on a PAM session startup and end.. this forces the bash(which is a pam) session to use the limits that are set by the admin in the limits.conf... It's not such a good idea to allow a users such a thing but this is the admin choice. Eliezer
[squid-users] Can't cache static content (exe file)
i'm can't cache this file using squid-3.HEAD-20130910-r13002 http://download.nullsoft.com/winamp/client/winamp565_full_emusic-7plus_en-us.exe i think that is static file but can't be cache. here my squid.conf # ACL Local Network acl localnet src 192.168.1.0/24 # localnet # ACL ports group acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT # ACL url rewrite acl trick Warning!!! dont change this position acl blocksite url_regex -i siteblock-forbiden http_access deny blocksite # ACL store id acl QUERY urlpath_regex -i (begin|start)\= acl QUERY urlpath_regex -i cgi-bin \? .php$ .asp$ .shtml$ .cfm$ .cfml$ .phtml$ .php3$ localhost acl dontrewrite url_regex -i c\.youtube\.com\/.*(begin|start)\=.* acl dontrewrite url_regex redbot\.org acl getmethod method GET acl redir urlpath_regex -i redirect_counter=1cms_redirect=yes acl redir urlpath_regex -i ir=1rr=12 acl rewrite_prog url_regex -i siteblock-forbiden # ACL url rewrite trick deny for store id acl yutub url_regex -i youtube\.com\/(generate_204|ptracking|stream_204|player_204|s|(.*(playback|watchtime|delayplay)))\?.*$ acl yutub url_regex -i gstatic\.com\/csi\?.*$ acl rewritedoms url_regex -i dl\.sourceforge\.net.* acl rewritedoms url_regex -i i[0-9]*\.ytimg\.com.* acl rewritedoms url_regex -i ak\.fbcdn\.net.* acl rewritedoms url_regex -i (youtube|google).*\/videoplayback\?.* acl rewritedoms url_regex -i [a-z][a-z][0-9][0-9]\.filehippo\.com\/.*(exe|zip|rar) #banking deny ssl-bump acl bank dstdomain ib.bri.co.id # HTTP access rules http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all # Store-ID cache rules cache allow rewritedoms cache deny QUERY cache deny redir cache allow all # squidGuard url_rewrite url_rewrite_program /usr/bin/squidGuard -c /etc/squid3/squidGuard/squidGuard.conf # Store-ID running and rules store_id_program /usr/lib/squid3/mynet-storeid.pl store_id_children 20 startup=10 idle=5 concurrency=30 store_id_access deny !getmethod store_id_access deny redir store_id_access deny rewrite_prog # deny url rewrite trick access by store id store_id_access deny dontrewrite store_id_access allow rewritedoms store_id_access deny all # Squid listens to port 3128 http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB key=/etc/squid3/cert/private.pem cert=/etc/squid3/cert/public.pem always_direct allow all ssl_bump none bank ssl_bump client-first all sslproxy_cert_error allow all # Or may be deny all according to your company policy # sslproxy_cert_error deny all sslproxy_flags DONT_VERIFY_PEER sslcrtd_program /usr/lib/squid3/ssl_crtd -s /etc/squid3/ssl_db/ -M 4MB sslcrtd_children 20 ssl_unclean_shutdown on # cache directory cache_dir aufs /cache01 5320 12 256 max-size=128000 cache_dir aufs /cache02 5320 12 256 max-size=128000 cache_dir aufs /cache03 87115 10 256 min-size=128000 cache_dir aufs /cache04 87115 10 256 min-size=128000 cache_dir aufs /cache05 87115 10 256 min-size=128000 # Leave coredumps in the first cache dir coredump_dir /var/spool/squid3 # Cache options memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA cache_mem 128 MB maximum_object_size_in_memory 32 KB minimum_object_size 0 KB maximum_object_size 1024 MB cache_swap_low 95 cache_swap_high 99 # Refresh pattern custom # Max stale max_stale 1 week #PATTERN REFRESH refresh_pattern .*(begin|start)\=[1-9][0-9].* 0 0% 0 refresh_pattern -i (cgi-bin|mrtg|graph) 0 0% 0 refresh_pattern -i \.(php|lst|ui|ini|list)$ 0 0% 0 refresh_pattern (update.ini|Update.ini|version.list|Version.list|update.1st|update.exe|autoup.exe) 0 0% 0 refresh_pattern (hackshield|nprotect) 240 100% 420 override-expire override-lastmod reload-into-ims refresh_pattern \.gemscool.com.*\.(exe|dll|cab|zip|iop|npz|swf)$ 1440 100% 4320 override-expire override-lastmod reload-into-ims ignore-auth store-stale refresh_pattern \.crossfire.web.id.*\.(cab|zip|exe|rar|dat|swf)$ 1440 100% 4320 override-expire override-lastmod reload-into-ims ignore-auth store-stale refresh_pattern \.cabalonline.co.id.*\.(cab|zip|exe|rar|dat|swf) 1440 100% 4320 override-expire override-lastmod reload-into-ims ignore-auth store-stale refresh_pattern \.megaxus.com.*\.(cab|zip|exe|rar|dat|swf) 1440 100% 4320 override-expire override-lastmod reload-into-ims
Re: [squid-users] Can't cache static content (exe file)
On Sunday 15 September 2013 at 18:22:31, r...@linuxcoding.org wrote: i'm can't cache this file using squid-3.HEAD-20130910-r13002 http://download.nullsoft.com/winamp/client/winamp565_full_emusic-7plus_en-u s.exe i think that is static file but can't be cache. What does access.log show you for the first, and the second, times you try to download this? Please ensure the first and second downloads are from different clients. Regards, Antony. -- Once you have a panic, things tend to become rather undefined. - murble Please reply to the list; please don't CC me.
Re: [squid-users] 100% CPU Load problem with squid 3.3.8
On Sun, Sep 15, 2013 at 2:51 PM, Carlos Defoe carlosde...@gmail.com wrote: I got the same result as Mohsen. The only thing that worked was adding ulimit -n mynumber to the init script. It was weird for me, because the script is run by root, not the squid user, and i thought ulimit -n applied only to the current logged in user. But I think it applies to any session that will start later. Ulimits are inherited by all child processes; lowering them is always possible, raising them may be an administrator-only action. bash's manual (man 1 bash) has an informative chapter on ulimit. Otherwise you may want to check setrlimit(2). System-wide settings may be set in /etc/security/limits.conf (or /etc/limits.conf, depending on your distro). Man 5 limits.conf has the details (at least on my Ubuntu Raring system). Kinkie
Re: [squid-users] 100% CPU Load problem with squid 3.3.8
Seems right, Kinkie. ulimit Provides control over the resources available to the shell and to processes started by it. So that's why squid process inherits the configuration made on the initialization script. I assume that doesn't matter which user runs the subprocess. But limits.conf is a PAM feature. As you said, it should configure limits on a system basis, for all users configured. What we are saying is that it does not work at boot time, when squid runs. The configuration on the init script is the only way to make it work. On Sun, Sep 15, 2013 at 4:59 PM, Kinkie gkin...@gmail.com wrote: On Sun, Sep 15, 2013 at 2:51 PM, Carlos Defoe carlosde...@gmail.com wrote: I got the same result as Mohsen. The only thing that worked was adding ulimit -n mynumber to the init script. It was weird for me, because the script is run by root, not the squid user, and i thought ulimit -n applied only to the current logged in user. But I think it applies to any session that will start later. Ulimits are inherited by all child processes; lowering them is always possible, raising them may be an administrator-only action. bash's manual (man 1 bash) has an informative chapter on ulimit. Otherwise you may want to check setrlimit(2). System-wide settings may be set in /etc/security/limits.conf (or /etc/limits.conf, depending on your distro). Man 5 limits.conf has the details (at least on my Ubuntu Raring system). Kinkie
Re: [squid-users] 100% CPU Load problem with squid 3.3.8
Well for me it works when I start a shell using su - user. But there is a need to know and map the linux boot process and them findout why bash is limited to 1024 FD instead of 4k or 4m. The basic issue is a security issue.. which I support enforcing as it is now. What squid can do as a process to force FD limit? squid as a limited process can only reach the LIMIT as at is now. since squid 3.2 does a forking style under-the-ground it is indeed the best practice to limit the init.d script access and execution.. then upper the ULIMIT in the init.d script to make sure that the limit makes sense.. if there is a start-stop-daemon feature then this is why it was invented anyway. So we can choose to either work with an execution proxy the will force all users options or force the ULIMIT in the init.d(bash) script. I would not try to run under bash a for loop that opens more then 512 FD in a case I want to stay sane. Eliezer On 09/16/2013 12:58 AM, Carlos Defoe wrote: Seems right, Kinkie. ulimit Provides control over the resources available to the shell and to processes started by it. So that's why squid process inherits the configuration made on the initialization script. I assume that doesn't matter which user runs the subprocess. But limits.conf is a PAM feature. As you said, it should configure limits on a system basis, for all users configured. What we are saying is that it does not work at boot time, when squid runs. The configuration on the init script is the only way to make it work. On Sun, Sep 15, 2013 at 4:59 PM, Kinkie gkin...@gmail.com wrote: On Sun, Sep 15, 2013 at 2:51 PM, Carlos Defoe carlosde...@gmail.com wrote: I got the same result as Mohsen. The only thing that worked was adding ulimit -n mynumber to the init script. It was weird for me, because the script is run by root, not the squid user, and i thought ulimit -n applied only to the current logged in user. But I think it applies to any session that will start later. Ulimits are inherited by all child processes; lowering them is always possible, raising them may be an administrator-only action. bash's manual (man 1 bash) has an informative chapter on ulimit. Otherwise you may want to check setrlimit(2). System-wide settings may be set in /etc/security/limits.conf (or /etc/limits.conf, depending on your distro). Man 5 limits.conf has the details (at least on my Ubuntu Raring system). Kinkie
Re: [squid-users] Can't cache static content (exe file)
The file is valid for about a DAY.. I would say that the main thing will be to look at the access.log and then into the squid.conf and then into the store.log. If you can share squid.conf and access.log We will be able to help you. Notice that there is kind of a bug that forces squid 3.4 or couple other versions into a state which the maximum file size for a cache_dir must be declared in the cache_dir line. I do think and hope the above hint will help you. Eliezer On 09/15/2013 07:22 PM, r...@linuxcoding.org wrote: i'm can't cache this file using squid-3.HEAD-20130910-r13002 http://download.nullsoft.com/winamp/client/winamp565_full_emusic-7plus_en-us.exe i think that is static file but can't be cache. here my squid.conf # ACL Local Network acl localnet src 192.168.1.0/24 # localnet # ACL ports group acl SSL_ports port 443 acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443# https acl Safe_ports port 70# gopher acl Safe_ports port 210# wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280# http-mgmt acl Safe_ports port 488# gss-http acl Safe_ports port 591# filemaker acl Safe_ports port 777# multiling http acl CONNECT method CONNECT # ACL url rewrite acl trick Warning!!! dont change this position acl blocksite url_regex -i siteblock-forbiden http_access deny blocksite # ACL store id acl QUERY urlpath_regex -i (begin|start)\= acl QUERY urlpath_regex -i cgi-bin \? .php$ .asp$ .shtml$ .cfm$ .cfml$ .phtml$ .php3$ localhost acl dontrewrite url_regex -i c\.youtube\.com\/.*(begin|start)\=.* acl dontrewrite url_regex redbot\.org acl getmethod method GET acl redir urlpath_regex -i redirect_counter=1cms_redirect=yes acl redir urlpath_regex -i ir=1rr=12 acl rewrite_prog url_regex -i siteblock-forbiden # ACL url rewrite trick deny for store id acl yutub url_regex -i youtube\.com\/(generate_204|ptracking|stream_204|player_204|s|(.*(playback|watchtime|delayplay)))\?.*$ acl yutub url_regex -i gstatic\.com\/csi\?.*$ acl rewritedoms url_regex -i dl\.sourceforge\.net.* acl rewritedoms url_regex -i i[0-9]*\.ytimg\.com.* acl rewritedoms url_regex -i ak\.fbcdn\.net.* acl rewritedoms url_regex -i (youtube|google).*\/videoplayback\?.* acl rewritedoms url_regex -i [a-z][a-z][0-9][0-9]\.filehippo\.com\/.*(exe|zip|rar) #banking deny ssl-bump acl bank dstdomain ib.bri.co.id # HTTP access rules http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all # Store-ID cache rules cache allow rewritedoms cache deny QUERY cache deny redir cache allow all # squidGuard url_rewrite url_rewrite_program /usr/bin/squidGuard -c /etc/squid3/squidGuard/squidGuard.conf # Store-ID running and rules store_id_program /usr/lib/squid3/mynet-storeid.pl store_id_children 20 startup=10 idle=5 concurrency=30 store_id_access deny !getmethod store_id_access deny redir store_id_access deny rewrite_prog # deny url rewrite trick access by store id store_id_access deny dontrewrite store_id_access allow rewritedoms store_id_access deny all # Squid listens to port 3128 http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB key=/etc/squid3/cert/private.pem cert=/etc/squid3/cert/public.pem always_direct allow all ssl_bump none bank ssl_bump client-first all sslproxy_cert_error allow all # Or may be deny all according to your company policy # sslproxy_cert_error deny all sslproxy_flags DONT_VERIFY_PEER sslcrtd_program /usr/lib/squid3/ssl_crtd -s /etc/squid3/ssl_db/ -M 4MB sslcrtd_children 20 ssl_unclean_shutdown on # cache directory cache_dir aufs /cache01 5320 12 256 max-size=128000 cache_dir aufs /cache02 5320 12 256 max-size=128000 cache_dir aufs /cache03 87115 10 256 min-size=128000 cache_dir aufs /cache04 87115 10 256 min-size=128000 cache_dir aufs /cache05 87115 10 256 min-size=128000 # Leave coredumps in the first cache dir coredump_dir /var/spool/squid3 # Cache options memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA cache_mem 128 MB maximum_object_size_in_memory 32 KB minimum_object_size 0 KB maximum_object_size 1024 MB cache_swap_low 95 cache_swap_high 99 # Refresh pattern custom # Max stale max_stale 1 week #PATTERN REFRESH refresh_pattern .*(begin|start)\=[1-9][0-9].* 0 0% 0 refresh_pattern -i (cgi-bin|mrtg|graph) 0 0% 0 refresh_pattern -i \.(php|lst|ui|ini|list)$ 0 0% 0 refresh_pattern (update.ini|Update.ini|version.list|Version.list|update.1st|update.exe|autoup.exe) 0 0% 0 refresh_pattern (hackshield|nprotect) 240 100% 420
[squid-users] Caching linux distro VS Mirroring them...
I have seen couple questions about trying to force of linux distributions mirrors into squid cache. This is nice but it missed the whole point of squid.. let say I do have a log of the proxy traffic domains and I see a server or a domain that takes a nice amount of traffic like debian deb files etc.. for a sysadmin to turn on a small VM with 10-20GB HD size can take about couple minutes or a little more. configuring it as a mirror also can take minutes... posting it as a mirror can take more then a day but then you got it right. instead of just force your clients a cache and harm the CIA of data just turn on a MIRROR if needed. it adds a small maintenance TASK but compared to caching all of the site as part of the cache and as a store?? also a mirror can be even a reverse proxy to the main site or another local\closer mirror. This way you do not harm the integrity of the data and also make sure that the mirror is a cache only mirror with a longer cache period which can help a lot. Eliezer
Re: [squid-users] Can't cache static content (exe file)
this is my squid.conf and access.log i hope someone can solved this problem On 2013-09-16 08:34, Eliezer Croitoru wrote: The file is valid for about a DAY.. I would say that the main thing will be to look at the access.log and then into the squid.conf and then into the store.log. If you can share squid.conf and access.log We will be able to help you. Notice that there is kind of a bug that forces squid 3.4 or couple other versions into a state which the maximum file size for a cache_dir must be declared in the cache_dir line. I do think and hope the above hint will help you. Eliezer On 09/15/2013 07:22 PM, r...@linuxcoding.org wrote: i'm can't cache this file using squid-3.HEAD-20130910-r13002 http://download.nullsoft.com/winamp/client/winamp565_full_emusic-7plus_en-us.exe i think that is static file but can't be cache. here my squid.conf # ACL Local Network acl localnet src 192.168.1.0/24 # localnet # ACL ports group acl SSL_ports port 443 acl Safe_ports port 80# http acl Safe_ports port 21# ftp acl Safe_ports port 443# https acl Safe_ports port 70# gopher acl Safe_ports port 210# wais acl Safe_ports port 1025-65535# unregistered ports acl Safe_ports port 280# http-mgmt acl Safe_ports port 488# gss-http acl Safe_ports port 591# filemaker acl Safe_ports port 777# multiling http acl CONNECT method CONNECT # ACL url rewrite acl trick Warning!!! dont change this position acl blocksite url_regex -i siteblock-forbiden http_access deny blocksite # ACL store id acl QUERY urlpath_regex -i (begin|start)\= acl QUERY urlpath_regex -i cgi-bin \? .php$ .asp$ .shtml$ .cfm$ .cfml$ .phtml$ .php3$ localhost acl dontrewrite url_regex -i c\.youtube\.com\/.*(begin|start)\=.* acl dontrewrite url_regex redbot\.org acl getmethod method GET acl redir urlpath_regex -i redirect_counter=1cms_redirect=yes acl redir urlpath_regex -i ir=1rr=12 acl rewrite_prog url_regex -i siteblock-forbiden # ACL url rewrite trick deny for store id acl yutub url_regex -i youtube\.com\/(generate_204|ptracking|stream_204|player_204|s|(.*(playback|watchtime|delayplay)))\?.*$ acl yutub url_regex -i gstatic\.com\/csi\?.*$ acl rewritedoms url_regex -i dl\.sourceforge\.net.* acl rewritedoms url_regex -i i[0-9]*\.ytimg\.com.* acl rewritedoms url_regex -i ak\.fbcdn\.net.* acl rewritedoms url_regex -i (youtube|google).*\/videoplayback\?.* acl rewritedoms url_regex -i [a-z][a-z][0-9][0-9]\.filehippo\.com\/.*(exe|zip|rar) #banking deny ssl-bump acl bank dstdomain ib.bri.co.id # HTTP access rules http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access allow localhost manager http_access deny manager http_access allow localnet http_access allow localhost http_access deny all # Store-ID cache rules cache allow rewritedoms cache deny QUERY cache deny redir cache allow all # squidGuard url_rewrite url_rewrite_program /usr/bin/squidGuard -c /etc/squid3/squidGuard/squidGuard.conf # Store-ID running and rules store_id_program /usr/lib/squid3/mynet-storeid.pl store_id_children 20 startup=10 idle=5 concurrency=30 store_id_access deny !getmethod store_id_access deny redir store_id_access deny rewrite_prog # deny url rewrite trick access by store id store_id_access deny dontrewrite store_id_access allow rewritedoms store_id_access deny all # Squid listens to port 3128 http_port 3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB key=/etc/squid3/cert/private.pem cert=/etc/squid3/cert/public.pem always_direct allow all ssl_bump none bank ssl_bump client-first all sslproxy_cert_error allow all # Or may be deny all according to your company policy # sslproxy_cert_error deny all sslproxy_flags DONT_VERIFY_PEER sslcrtd_program /usr/lib/squid3/ssl_crtd -s /etc/squid3/ssl_db/ -M 4MB sslcrtd_children 20 ssl_unclean_shutdown on # cache directory cache_dir aufs /cache01 5320 12 256 max-size=128000 cache_dir aufs /cache02 5320 12 256 max-size=128000 cache_dir aufs /cache03 87115 10 256 min-size=128000 cache_dir aufs /cache04 87115 10 256 min-size=128000 cache_dir aufs /cache05 87115 10 256 min-size=128000 # Leave coredumps in the first cache dir coredump_dir /var/spool/squid3 # Cache options memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA cache_mem 128 MB maximum_object_size_in_memory 32 KB minimum_object_size 0 KB maximum_object_size 1024 MB cache_swap_low 95 cache_swap_high 99 # Refresh pattern custom # Max stale max_stale 1 week #PATTERN REFRESH refresh_pattern .*(begin|start)\=[1-9][0-9].* 0 0% 0 refresh_pattern -i (cgi-bin|mrtg|graph) 0 0% 0 refresh_pattern -i \.(php|lst|ui|ini|list)$ 0 0% 0 refresh_pattern (update.ini|Update.ini|version.list|Version.list|update.1st|update.exe|autoup.exe) 0 0% 0 refresh_pattern (hackshield|nprotect) 240 100% 420 override-expire
