[squid-users] Learn which refresh pattern matchs
Hello I been adding some refresh patterns to my squid 2.7 setup in order to save bandwidth. Since I have a lot of patterns, I am having trouble understanding with matches when. Is there any way to see given an url which one matches other than manualy searching the pattern on the file? Thanks Sebastian
[squid-users] Caching in Afghanistan
Hello there everyone, I'm currently deployed to Afghanistan and have recently set up a VSAT connection with approximately 18 users at peak. Not a large number of users however in our remote location a simple opening of a page with full user activity can slow things down to a near halt. I've been trying to do research on a caching server that would cache web images so that commonly opened websites would use LAN bandwidth rather then the VSAT bandwidth. I'll list the following setup I have for the 3 living quarter buildings, I'm not sure what exactly I need to do hardware/software wise but I was recommended to check this service out so hopefully you guys can let me know if Squid is exactly what I'm looking for: - BLDG 43 Router DHCP Disabled, 7 Users VSAT Hardware - x3 iDirect Router Modem - Cisco PoE-48 Port Switch - BLDG 42 Direct from Switch, 6 to 7 Users - BLDG 41 Unmanaged-Switch, 5 Users The VSAT - Router Modem connection is connected by a Rx and Tx Coaxial line (RG6). All other connections are Cat5e. I really do appreciate any help that is given. Thank you in advance -SGT B.
[squid-users] Squid Service Problem (Ubuntu)
I'm using Squid3 (squid/3.1.14) on Ubuntu 11.10. I've installed it via apt-get repository and it's running as a service. The problem is: When I first open the computer, Squid can't connect to any URI. But when I restart it by using service squid3 stop and service squid3 start commands, it starts working. It's always like this. Why it happens and what can I do ? Does anyone know ?
[squid-users] Squid ICAP Problem
Hello everbody. I'm using Squid3 (squid/3.1.14) on Ubuntu 11.10. I'm using it with my own-made ICAP server which is RFC 3507 compliant. The problem is: When chunked transfer encoding occurs (encapsulated http message has a body), sometimes it fails with my send function giving errno 11 (Resource temporarily unavailable). This happens when I try to upload relatively big files (around 500 KB), when I tried smaller ones, there wasn't any problem. It seems like, increasing chunk size of my ICAP server's responses temporarily solved the problem. But I need to know its reason. Is there any chunk limit or something in Squid or why this problem can occur ?
Re: [squid-users] Squid Service Problem (Ubuntu)
On 19/02/2012 1:41 a.m., gVeR SoNiC wrote: I'm using Squid3 (squid/3.1.14) on Ubuntu 11.10. I've installed it via apt-get repository and it's running as a service. The problem is: When I first open the computer, Squid can't connect to any URI. But when I restart it by using service squid3 stop and service squid3 start commands, it starts working. It's always like this. Why it happens and what can I do ? Does anyone know ? Seems to be this bug in Ubuntu packaging https://bugs.launchpad.net/ubuntu/+source/squid/+bug/97513 Amos
Re: [squid-users] Learn which refresh pattern matchs
On 18/02/2012 9:05 p.m., sebastian muniz wrote: Hello I been adding some refresh patterns to my squid 2.7 setup in order to save bandwidth. Since I have a lot of patterns, I am having trouble understanding with matches when. Is there any way to see given an url which one matches other than manualy searching the pattern on the file? Configure: debug_options 22,3 warning: this can produce a big log. Amos
Re: [squid-users] Working Squid Configuration, but needs some fat reduction
Dear Amos , Thanks again for your continues support. I hope at the end of this we can have a complete and a simple configuration for everyone that want to have a simple secure reverse proxy. I always like to do things the right way, but the SSL is the only thing standing in my way. I really don't like the 'sslflags=DONT_VERIFY_PEER' either but removing it causing many problems. As you can see from https_port 443 cert=C:/Interceptor/cert/baj.cert key=C:/Interceptor/cert/baj.key Those are the OpenSSL Pem conversion for my actual domain certificate, I did also generate a pem for the CA, and tried all possible combination of capath , cafile, clientcasslproxy_capath to insert this certificate CA but without success (getting different errors) So as you said Without it Squid attempts to validate the peer SSL certificate against the root CA Squid (via the openssl library) trusts. I'm not sure what is the connection between the two , I do have OpenSSL installed but only for conversion of the certificates (from cert, to pem) (Not even installed on my production squid !). My question, how to add the CA cert to OpenSSL or Squid ? Best Regards, Alaa Murad On Sat, Feb 18, 2012 at 9:40 AM, Amos Jeffries squ...@treenet.co.nz wrote: On 18/02/2012 8:20 a.m., ALAA MURAD wrote: Dear Amos , Thanks so much for your help, I hope this changes makes more sense. I have been following your comments and I feel squid is running better than before. The setting now feels more cleaner and squid out less errors, but mainly now I'm suffering from one thing, sometimes , I get this error : clientNegotiateSSL: Error negotiating SSL connection on FD 355: WSAENOTCONN, Socket is not connected. (10057) clientNegotiateSSL: Error negotiating SSL connection on FD 356: Resource temporarily unavailable (11) The TCP connection FD 355 is closing before SSL details can be negotiated between Squid and the client. The TCP connection FD 356 is doing negotiation but was unsuccessful due to resource unavailable SSL error. Probably broken TCP again, or SSL security system failing to access things. Are you able to identify whether those are connections between the peers? or from outside clients? Both servers are connected peer-to-peer (back-to-back) and a running ping confirms connection is up all the time! One thing to note about the above problem, that this mainly causing errors in IE , saying that the certificate has expired (this is really random and not sure what is wrong with it, waiting few minutes certificate is OK again ! ). Other than that, it's perfect ! Also I did those : 1- I have removed all http (port : 80 ) as this reverse proxy running only on SSL. Also do I need all this Safe_Ports for a site that only serves port 433 ?! Safe_ports is optional for a reverse-proxy. It is only relevant to forward-proxy and management port traffic. if you chose to do the management port, IF you choose to use one (the management requests can be done over accel ports too). 2- Removed cache unwanted logs . 3- Still confused with defaultsite=www.eservices.mycompany.com , I kinda got the point , but not sure what is the perfect alternative (removing it caused a header error in browser ). The vhost option tells Squid to use the client Host: header to identify the domain. defaultsite= is a backup for use if the client did not send Host: information at all. 4- Also my rule http_access allow all I guess that is needed in reverse proxy as I want to allow all to hit my site. No. Allowing your site is all that is needed. You earlier had http_access allow mycompanyserver which was doing the correct thing, and doing it before the Safe-ports and surrounding access controls. That (at the top) was the right way to do reverse-proxy access controls, so the forward-proxy ones do not get a chance to slow down the requests. allow all has the effect of passing all requests on to the backend server. By only allowing your site requests to go back to the server Squid can protect against DDoS with randomised domain names pointing at you. 5- The redirector, mostly output blanks (99%) , and in rare event it intercept and rewrite the url. Okay. good. for concurrent rewrite isn't why we can load many helpers and that will help in concurrency? I'm good in Threading in Java, but what I'm afraid of is to confuse squid , when printing stuff out of order in multi-threaded application. The concurrency works by Squid sending each request with a channel-ID token. The redirector can send its reply back at any time with the same ID token and Squid does not get confused. This saves memory and CPU running many helper processes, when a few multi-threaded ones can be used. Even without multi-threading it raises the amount of requests each helper can service, saving memory and user annoyance. 6-Not sure about this as it's a windows server , refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
Re: [squid-users] Working Squid Configuration, but needs some fat reduction
On 19/02/2012 5:42 a.m., ALAA MURAD wrote: Dear Amos , Thanks again for your continues support. I hope at the end of this we can have a complete and a simple configuration for everyone that want to have a simple secure reverse proxy. I always like to do things the right way, but the SSL is the only thing standing in my way. I really don't like the 'sslflags=DONT_VERIFY_PEER' either but removing it causing many problems. As you can see from https_port 443 cert=C:/Interceptor/cert/baj.cert key=C:/Interceptor/cert/baj.key Those are the OpenSSL Pem conversion for my actual domain certificate, I did also generate a pem for the CA, and tried all possible combination of capath , cafile, clientca sslproxy_capath to insert this certificate CA but without success (getting different errors) Nono. The root CA are installed into OpenSSL directly. See the google links below. So as you said Without it Squid attempts to validate the peer SSL certificate against the root CA Squid (via the openssl library) trusts. I'm not sure what is the connection between the two , I do have OpenSSL installed but only for conversion of the certificates (from cert, to pem) (Not even installed on my production squid !). Openssl libraries (libssl ?) are what do all the work. The squid.conf settings are just telling Squid what to send to openSSL for each type of SSL connection: client connections (http_port), server connections (sslproxy_*) and peer connections (cache_peer). My question, how to add the CA cert to OpenSSL or Squid ? There seems to be a few tutorials on it: http://www.google.com/search?q=add+a+custom+CA+to+openssl Amos
Re: [squid-users] Squid ICAP Problem
On 19/02/2012 1:58 a.m., gVeR SoNiC wrote: Hello everbody. I'm using Squid3 (squid/3.1.14) on Ubuntu 11.10. I'm using it with my own-made ICAP server which is RFC 3507 compliant. The problem is: When chunked transfer encoding occurs (encapsulated http message has a body), sometimes it fails with my send function giving errno 11 (Resource temporarily unavailable). This happens when Which means what when its at home? network buffers full (temporarily)? out of memory? disk busy doing other things? TCP connection broken? Finding that out will likely lead you to the cause. I try to upload relatively big files (around 500 KB), when I tried smaller ones, there wasn't any problem. It seems like, increasing chunk size of my ICAP server's responses temporarily solved the problem. But I need to know its reason. *raising* them? that is strange. From what to what? Is there any chunk limit or something in Squid or why this problem can occur ? Not that I'm aware of. Amos
Re: [squid-users] Learn which refresh pattern matchs
El 18/02/2012 13:33, Amos Jeffries escribió: On 18/02/2012 9:05 p.m., sebastian muniz wrote: Is there any way to see given an url which one matches other than manualy searching the pattern on the file? Configure: debug_options 22,3 warning: this can produce a big log. Amos THanks! Will try that.
Re: [squid-users] Caching in Afghanistan
On 18/02/2012 23:56, jbrodi...@gci.net wrote: Hello there everyone, I'm currently deployed to Afghanistan and have recently set up a VSAT connection with approximately 18 users at peak. Not a large number of users however in our remote location a simple opening of a page with full user activity can slow things down to a near halt. I've been trying to do research on a caching server that would cache web images so that commonly opened websites would use LAN bandwidth rather then the VSAT bandwidth. I'll list the following setup I have for the 3 living quarter buildings, I'm not sure what exactly I need to do hardware/software wise but I was recommended to check this service out so hopefully you guys can let me know if Squid is exactly what I'm looking for: - BLDG 43 Router DHCP Disabled, 7 Users VSAT Hardware - x3 iDirect Router Modem - Cisco PoE-48 Port Switch - BLDG 42 Direct from Switch, 6 to 7 Users - BLDG 41 Unmanaged-Switch, 5 Users The VSAT - Router Modem connection is connected by a Rx and Tx Coaxial line (RG6). All other connections are Cat5e. I really do appreciate any help that is given. Thank you in advance -SGT B. Hi, I would say Squid will help a lot. I would put down a full Linux box, put bind,dhcpd (for 18 people this might be an overkill) and Squid on there. I would setup the box in a transparent proxy mode. I would put the bind server in caching mode. Hardware wise, you won't need much. Take what you can get your hands on. More memory, fast disks (and spindles over space) is good. Given the size/speed of the link and the number of users, a desktop type PC will even do the trick. Hope that helps ! Pieter
[squid-users] Solving the mysteries of the icap client..
Hi, This is my first post on the squid-users mailing list and as an introduction i'll start with a few questions on the squid icap client :) 1. Why was the balancing mechanism removed from the icap client ( starting from 3.x i think, didn't read the changelogs recently ) ? Now i have to delegate icap balancing to a local HAProxy instance, which is less convenient even if HAProxy works great. 2. when having multiple ICAP services configured in squid, and considering that each of them can potentially alter the content of requests/responses, is there a way to decide if the services must be chained, or queried asynchronously ? the good thing would be to be able to set this behavior for each icap service configured. That's all for today, but no doubts i will have plenty of other questions in the near future :) Best regards, C.
Re: [squid-users] reverse proxy config
Hi Amos Am 18.02.2012 02:29, schrieb Amos Jeffries: On 18/02/2012 3:09 a.m., Erich Titl wrote: Hi Folks cache_peer_access pfaeffikon-ssl allow sites_server_104 Note: cache_peer_access and cache_peer_domain are alternative metods of deciding whether to service the request there. You can drop one of them. http_access allow sites_server_104 Here is an excerpt of the cache.log file 2012/02/17 14:47:07 kid1| Accepting HTTP Socket connections at local=[::]:8080 remote=[::] FD 15 flags=9 2012/02/17 14:47:07 kid1| Accepting reverse-proxy HTTPS Socket connections at local=[::]:443 remote=[::] FD 16 flags=9 2012/02/17 14:47:07 kid1| Configuring Parent pfaeffikon.gever.asp.ruf.ch/8083/0 2012/02/17 14:47:07 kid1| Configuring Parent To me it looks like the cache peer is not accessed correcly. Maybe. That config says the peer will only service http://m278.asp.ruf.ch:443/* URLs. And does so without encryption on the squid-peer link. The goal is to terminate https requests on the proxy and forwarding the requests for pfaeffikon.gever.asp.ruf.ch to a peer called m278.asp.ruf.ch on port 8083 I see the requests arriving at the squid host, but it appears that it does not use the host specified in the peer parameters but uses the name of the original request host. Yes. Exactly so. The client is asking for https://pfaeffikon.gever.asp.ruf.ch/. Squid will attempt to service that request. You have told Squid to only accept requests for the domain m278.asp.ruf.ch. Amos Of course you were right, I needed to understand the syntax a bit better. I had the settings the wrong way around. Thanks Erich
Re: [squid-users] Solving the mysteries of the icap client..
On 19/02/2012 9:40 a.m., Clement Game wrote: Hi, This is my first post on the squid-users mailing list and as an introduction i'll start with a few questions on the squid icap client :) 1. Why was the balancing mechanism removed from the icap client ( starting from 3.x i think, didn't read the changelogs recently ) ? Now i have to delegate icap balancing to a local HAProxy instance, which is less convenient even if HAProxy works great. Squid 2.x and older have *no* ICAP support, so I'm not sure what you are talking about there. Something cannot be removed if it was never yet supported. Squid-3.0 was the first to add ICAP support, and as such it was fairly limited and only did what the sponsors needed it to do. As time progressed more ICAP feature support was added. As of this writing the most current stable Squid release is 3.1.19. As for load balancing. AKAIK, due to network state overheads being worse than CPU overheads there is better performance when sending traffic to one backend (which already has the state on hand) than splitting it between two or more (fresh state lookups for each). ICAP in Squid utilizes this behaviour by loading one service processor up to its capacity as determined by the OPTIONS reponse Max-Connections header. With a backup mechanism measuring and balancing on response time in case someone set that limit too high or not at all. 2. when having multiple ICAP services configured in squid, and considering that each of them can potentially alter the content of requests/responses, is there a way to decide if the services must be chained, or queried asynchronously ? the good thing would be to be able to set this behavior for each icap service configured. You can. If its been a while since you read the documentation you may want to try catching up on the 3.1 ICAP developments before you get any more questions. http://wiki.squid-cache.org/Features/ICAP The way to decide you ask about is the static/dynamic chaining that documentation talks about. Amos
[squid-users] ld: Undefined symbols: _res_9_init compiling squid 3.2.0.15 with --disable-internal-dns option on Mac OS X 10.4.11
I had never attempted building with the --disable-internal-dns option before. Today, in the hope of enabling cache_dns_program, I tried it with squid-3.2.0.15-20120218-r11508. It barfed building /usr/local/squid/libexec/dnsserver: libtool: link: /usr/bin/ar cru .libs/libsquid.a .libs/comm.o .libs/CommCalls.o .libs/DescriptorSet.o .libs/SquidConfig.o libtool: link: ranlib .libs/libsquid.a libtool: link: ( cd .libs rm -f libsquid.la ln -s ../libsquid.la libsquid.la ) g++ -DHAVE_CONFIG_H -DDEFAULT_CONFIG_FILE=\/usr/local/squid/etc/squid.conf\ -DDEFAULT_SQUID_DATA_DIR=\/usr/local/squid/share\ -DDEFAULT_SQUID_CONFIG_DIR=\/usr/local/squid/etc\ -I.. -I../include -I../lib -I../src -I../include -I/usr/include -I/usr/include -I../libltdl -I../src -I../libltdl -I/usr/include -I/usr/include -I/usr/include -I/usr/include -Wall -Wpointer-arith -Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -c -o dnsserver.o dnsserver.cc /bin/sh ../libtool --tag=CXX --mode=link g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -g -o dnsserver dnsserver.o -L../compat -lcompat-squid libtool: link: g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -g -o dnsserver dnsserver.o -Wl,-bind_at_load -L/private/var/tmp/folders.501/TemporaryItems/squid-3.2.0.15-20120218-r11508/compat -lcompat-squid /usr/libexec/gcc/i686-apple-darwin8/4.0.1/ld: Undefined symbols: _res_9_init collect2: ld returned 1 exit status make[3]: *** [dnsserver] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all] Error 2 make: *** [all-recursive] Error 1 I tried switching to compiling with gcc 4.2.1. Same problem. ls -al /usr/lib/libresolv.dylib lrwxr-xr-x 1 root wheel 17 Aug 4 2007 /usr/lib/libresolv.dylib - libresolv.9.dylib cat /usr/include/resolv.h ... #define res_initres_9_init ... nm /usr/lib/libresolv.dylib | grep res_9_init 9351e8d1 T _res_9_init After searching online, this appears to be an issue peculiar to Mac OS X 10.3 or 10.4. I'm on 10.4.11. I tried a variety of workarounds: 1. Changing Makefile's LIBS = to LIBS = -lkvm -lm -lresolv -lz 2. Changing Makefile's LDFLAGS= to LDFLAGS=-lresolv 3. export LIBS=-lresolv 4. export LDFLAGS=-lresolv The only option that worked gets squid compiling with --disable-internal-dns is #4. I'm curious why the other three weren't effective. -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de
Re: [squid-users] ld: Undefined symbols: _res_9_init compiling squid 3.2.0.15 with --disable-internal-dns option on Mac OS X 10.4.11
On 19/02/2012 4:08 p.m., gewe...@gmx.net wrote: I had never attempted building with the --disable-internal-dns option before. Today, in the hope of enabling cache_dns_program, I tried it with squid-3.2.0.15-20120218-r11508. It barfed building /usr/local/squid/libexec/dnsserver: snip After searching online, this appears to be an issue peculiar to Mac OS X 10.3 or 10.4. I'm on 10.4.11. I tried a variety of workarounds: 1. Changing Makefile's LIBS = to LIBS = -lkvm -lm -lresolv -lz 2. Changing Makefile's LDFLAGS= to LDFLAGS=-lresolv 3. export LIBS=-lresolv 4. export LDFLAGS=-lresolv The only option that worked gets squid compiling with --disable-internal-dns is #4. I'm curious why the other three weren't effective. The dnsserver make rules are a little broken. It gets passed the default *FLAGS lists but not LIBS. Thank you for finding this. Was it Makefile or src/Makefile you changed for (1) and (2)? dsserver is built in src/Makefile. Amos
