Hi,
I have been following this thread of mails and I
have a problem with the number of redirectors too.
I am using squid-2.5.STABLE5 configured to work
with caching disabled.
Just curious: Any reason to disable caching?
> I am using a redirector that
does some url filtering using a local database.
What is the purpose? Filtering offending content? Selfmade redirector? What kind of database?
I have timed the redirector - it takes anywhere between 42 msecs to 2000 msecs at times to process. I am running squid on a system with the configuration - Linux-2.4.18-14, P4, 512 MB RAM. We have other applications like a web server, etc that are also running on this system.
2000msecs? Thats a lot...
You had mentioned in ur earlier mail that ur
system is so configured that 80% of the requests are
handled by the first redirector, 10% by the second and
so on. Could you kindly elaborate as to how this
done - or is it the way squid works? If this is the
case, then adding more redirectors shd not solve my
problem.
The number of requests handled by each redirector as described in the posts is more a phenomenon, not a configuration issue.
- Squidguard is very fast and IMHO the request are not distributed round robin, but to the first idle redirector
- light load - the first redirector is always ready to serve the request, all the others have never a chance to answer.
- increased load - sometime the first redirector is busy, some requests are answered by the 2nd redirector
- even higher load - sometime both 1st and 2nd redirectors are busy, the 3th gets a chance to do some work - and so on with the others if load increases.
- load peak - all redirectors get something to do, some request are queued and the warning message occurs, but never a FATAL error as the queue lenght never gets high (long?) enough.
I think your problem is different as your redirectors are quite slow, the queues are growing to long which causes the fatal error.
Increasing the number of redirectors should help a bit in your case, but probably the database is to slow to handle all the requests. If the database is the problem, increasing the no of redirector means increasing the processing time as well - this means the number of request which can be processed in a given amount of time will not increase much.
I tried conducting some simple load tests with
squid using the redirectors. 1 redirector worked fine for 5 simultaneous browser
clients(that is w/o throwing a FATAL error and
restarting), 2 redirectors worked fine for 14 browser clients
but subsequent tests showed that even with 5
redirector clients,20 browser clients cud not be
handled simultaneously. I don't want to enable
redirctor bypass though.
I am failing to understand this behaviour. I would
be thankful if u cud spare some time to explain what
cud be happening here and tell me a solution for it.
Regards and TIA,
Deepa
--- Hendrik_Voigtl�nder <[EMAIL PROTECTED]> wrote: > Hi,
E250 with how many processor of what type?=== message truncated ===
Probably you have an performance problem with the
sleepycat berkeley-db.
If all your processors are busy all the time
increasing the number of redirectors won't help.
In this case I would switch over to Linux with an
Intel machine. We have replaced our old E450 with an HP/Compaq ML370
(decent machine, but not high end) with a significant improve in squid
perfomance. I gave up on compiling squidguard on solaris, to much hassle and
to much load probably as well for the 168MHz(!) processors.
With debian no problem at all, 'apt-get install
squidguard' and done...
I really like the idea of using multiple cheap
machine with loadbalancing and failover, but IMHO you need to use
automatic proxy configuration to achieve this. I would use server
hardware for this, but something cheaper than HP/Compaq, for instance
Supermicro.
Get cachemgr.cgi running, it is really useful to
look at squid & squidguards status.
# TAG: redirector_bypass # When this is 'on', a request will not go through the # redirector if all redirectors are busy. If this is 'off' # and the redirector queue grows too large, Squid will exit # with a FATAL error and ask you to increase the number of # redirectors. You should only enable this if the redirectors # are not critical to your caching system. If you use # redirectors for access control, and you enable this option, # then users may have access to pages that they should not # be allowed to request. # redirector_bypass on
As you may have noticed it is impossible to filter
100% of all unwanted stuff, bypassing in high load situations won't make
things much worse.
Keep an eye on the redirector stats in cachemgr how
many requests are actually bypassing squidguard. In our setup it is
less than 1%.
Regards, Hendrik.
Merid Tilahun wrote:
Thanx Hendrik I am running squid on solaris 8, sun enterprise
250
machine. I have more that 500 users connect at
peak
hour. I never got around to configure cachemanager.cgi,
I
will look in to that. I use squidguard to filter porn, and it seems to
be
working but it is affecting the servicetime badly.
I
run around 20 redirector processes, I have been increasing constantly. I deactivated squidguard for a while and I got not messages, but I need squidGuard to block the porn. What is redirector bypass and how do i enable it?
--- Hendrik Voigtlaender
<[EMAIL PROTECTED]>
wrote:
Hi,
I just checked my logs: We get 40...50req/sec for about 10 hours a day
with
nearly no traffic during the night, this levels out to a daily
average
of 1000req/min. Number of users are reported with about
1000...1500,
but I guess that we have never more than 150 clients connect at the
same
time.
We have 15 redirectors running, average redirector
service time is not measurable, servicetimes are good. In cachemgr.cgi
you can check how many request are handled by each redirector.
In our setup, more than 80% of the requests are
handled by the first one. The second gets about 10%, all the other are
used only during peak periods. Redirectors #15 is idle nearly the whole
time.
In top only a a few (2...3) squidguards are
showing
enough activity to be noticed, but not much. Machine is Intel 2,4GHz,
2x36GB SCSI cache_dir, 2GB RAM, Linux debian woody, squid &
squidguard out of distro.
I played around with the number of redirectors but
never got rid of this message (all busy, increase blabla). I think it is
not critical if this warning appears on and off, but not to often. 15
is
IMHO more than sufficient with this load.
What are you using squidguard for? If you try to
block pr0n, you can as well enable redirector bypass. The blacklist will
never catch all sites anyway, it doesnt matter if a couple of more
request
bypass squidguard. This will eliminate the warning for sure.
How many redirectors are you using ? I would not
install squidguard but probably deativate it in squid.conf for testing.
I can imagine that too many redirectors are eating
up the ressources badly needed by squid, especially on a slow/small
machine.
Can you post some specs?
Regards, Hendrik Voigtl�nder
Merid Tilahun wrote:
Hi all: I have istalled and configured squid and it was working fine until now. I installed squidGuard and when
squid
starts working with it I get the user warning
messages
Too many queued redirector requests. I have
increased
the number of redirector processes but I am still getting this messages. Plus the retrieval time
for
sites is too long. Does any one know a way out?
or
shall I just uninstall squidguard? My squid get around 850 req/min ( got this from
MRTG).
Thank you in advance Regards
__________________________________
Do you Yahoo!?
Friends. Fun. Try the all-new Yahoo! Messenger.
http://messenger.yahoo.com/
________________________________________________________________________ Yahoo! India Matrimony: Find your partner online. http://yahoo.shaadi.com/india-matrimony/
