Re: [squid-users] cache_dir and max-size

2007-08-10 Thread Chris Robertson

RW wrote:

If I split my cache like this:

cache_dir aufs /usr/local/squid/small_obj 1700 16 256 max-size=80
cache_dir aufs /usr/local/squid/big_obj200 16 256

Presumably nothing bigger than max-size will end-up in the first
cache, but how often will the smaller objects end-up in the second
cache?  Assuming that both cache-directories have filled.
  


The answer to this question depends heavily on how busy your cache is.  
The default store_dir_select_algorithm is "least-load".  On a lightly 
used server, the first defined directory will be utilized the most.  As 
the server is more heavily trafficed the other directories will be 
tapped more. 

Some discussion has passed about a min-size argument to cache_dir (to 
keep small objects out of the non-COSS cache_dirs), but I don't know 
where that led.


Chris


Re: [squid-users] Redirect POST Requests

2007-08-10 Thread Chris Robertson

Santiago Del Castillo wrote:

Hi!

Is possible to handle GET and POST requests separately? I want GET  to
go to port 80 and POST to port 81 in the same origin server.

I'm using squid 2.6.STABLE13 as reverse-proxy.

Thanks!
  


cache_peer 192.168.0.10 parent 80 0 no-query originserver 
name=EverythingButPost

cache_peer 192.168.0.10 parent 81 0 no-query originserver name=Post
acl POST method POST
cache_peer_access Post allow POST
cache_peer_access Post deny all

Note: This is a back of the envelope calculation.  I have no idea if 
this will actually work.


Chris


Re: [squid-users] File Descriptors causing an issue in OpenBSD

2007-08-10 Thread Preetish
Hi All,

  Recompilng the kernel with MAXFILES=8192 worked. I even had
to add the line :openfiles-max=infinity:\

to /etc/login.def in the daemon section. Well now the File Descriptors
has increased and even the internet speed is good ( i ll know it
better by tomorrow). I have kept my cache to 10 Gb right now.Thanks to
everyone :)

Cheers
Preetish


Re: [squid-users] cache_peer

2007-08-10 Thread Chris Robertson

[EMAIL PROTECTED] wrote:

Hello,
I want to set up a squid server on machine A(P).
This Machine A(P) is behind a firewall and it has an access to a corporate
proxy B(P).
B(P) has some access control lists. And A(P) is allowed to use the B(P).
When a browser is run on A(P) to connect to any web site using the proxy
B(P) it is ok.

So a machine C is not allowed for B(P).
Running a browser on C Using A(P) as a proxy results
Access Denied (403) which comes from B(P)...

I want to use B(P) as a parent proxy for A(P). But A(P) shows the client C
as the request owner to B(P). So B(P) refuses.

Any help.

My current configuration includes - on A(P) squid.conf
cache_peer B(P) parent 8080 0
...
forwarded_for off


mail2web LIVE – Free email based on Microsoft® Exchange technology -
http://link.mail2web.com/LIVE
  


Likely you'll need to kill the "Via" header as well. Try...

header_access Via deny all
header_access X-Forwarded-For deny all

...and see if that helps.

Chris


Re: [squid-users] Cant access internal webserver when using squid 3128

2007-08-10 Thread Chris Robertson

SSCR Internet Admin wrote:

Hi,

My network is intercepting port 80 to 3128 (transparent proxy) in accessing
the internet.  Lately, I have some internal webserver and is now redirecting
trafic from outside to that internal webserver.  If use squid, (ie.,
configuring proxy on firefox), it seems that I am blocked or denied by
squid.  If I use transparent proxy, everything is working perfectly good.  


On what part on squid.conf I can tweak on this matter? Thanks and more power


TIA
  


Check the http_access rules.  Also make sure that you can access this 
internal server from the Squid box (using a browser, curl, wget or the 
like).


Chris


[squid-users] Re: cache replacement

2007-08-10 Thread RW
On Fri, 10 Aug 2007 12:30:44 +0500
"Shabbir Ahmed" <[EMAIL PROTECTED]> wrote:

> 
> i would like to know how squid replaces old items in cache by default.
> 
> like there isa site called abc.com squid cached it today and next day 
> abc.com made some changes what is the criteria squid ll cache changes
> and users ll get the newer pages.
>

When a client requests an object, squid looks at the object's expiry
time to determine whether it's stale or fresh. If there isn't an expiry
time a set of rules are used instead. Fresh objects are fetched out of
cache. If an object is stale, a conditional request is  made to the
webserver, which replies either with a new version of the object, or a
code saying the exiting object is still valid. 

It's also possible for browsers to force an update, or force a check
for updates.



Re: [squid-users] Problem with Sibling

2007-08-10 Thread Chris Robertson

Shekhar Gupta wrote:

All,

I have configured 4 proxy to work as sibling relation and specified
the ICP port as 3130 , however when i try to do a telnet it always
fails , so i think the cache performance is not getting optimized .
Any clue for this ?


Regards,
Shekhar
  


ICP is performed using UDP transmissions.  Telnet is TCP.

From one of your cache servers (assuming you have left cache_object 
requests allowed from localhost) try...


squidclient cache_object://localhost/server_list

...to get a view of how the ICP communications are working.

Chris


Re: [squid-users] Squid allow only 2 max connections per IP

2007-08-10 Thread Chris Robertson

eXtremer wrote:

Here is my config:

#Recommended minimum configuration:
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl SSL_ports port 443 563 444
acl Safe_ports port 80  # http
#acl Safe_ports port 21 # ftp
acl Safe_ports port 443 563 # https, snews
#acl Safe_ports port 70 # gopher
#acl Safe_ports port 210# wais
#acl Safe_ports port 1025-65535 # unregistered ports
#acl Safe_ports port 280# http-mgmt
#acl Safe_ports port 488# gss-http
#acl Safe_ports port 591# filemaker
#acl Safe_ports port 777# multiling http
acl CONNECT method CONNECT

#-Conectiuni maxime per IP-
acl maxcon maxconn 2

#--Reteaua 145--
acl 145a src "/etc/squid/allow/145a.allow"  
acl 145b src "/etc/squid/allow/145b.allow"


acl 145c src 192.168.41.200/32
  
#---Restrictions---


#---Restrictie ptr conectiuni maxime
http_access deny maxcon all
  


The "all" here should be the first acl referenced on the line or it will 
prevent your deny_info message from being shown.


e.g. "http_access deny all maxcon"


deny_info ERR_MAXCON maxcon

#---Restrictie ptr toti in afara de sala 145---
http_access deny all !145a !145b !145c !localhost

#Recommended minimum configuration:
http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports !SSL_ports
http_access deny CONNECT !SSL_ports
--

This is a part of my squid.conf
I don't know but maxcon is not working in my case.
If it's like this : http_access deny maxcon all <= then not even one
connection is allowed.
  


Hmmm... Have you tested this with a single connection downloader (such 
as squid-client) or by using a browser to download a non-referencing 
object (such as an image file)?  This looks like the proper usage of the 
maxconn acl.  You might try upping the debugging (see the debug_options 
on squid.conf), and watch what your cache.log reports).



If it's like this: http_access allow all !maxcon <= then all connection are
allowed.
  


This would not be such a good idea given the way your http_access lines 
are set up, as it would prevent any of the Safe_port and SSL_port checks 
later in the list.



Somebody tell me how to configure in such a way that maxconn feature will
wrk in my case,
waiting for a reply, 10x in advance.

P.S.: client_db is ON.
  


Chris


[squid-users] squid 2.6 ntlm active directory problem

2007-08-10 Thread Wilson Galafassi
hello.

i'm using samba 3.0.24-3 and squid-2.6.STABLE13-1 with fc6.

my smb.conf
[global]
workgroup = DOMAIN
netbios name = NETSERVER
server string = PROXY SERVER
load printers = no
log file = /var/log/samba/log.%m
max log size = 500
realm = DOMAIN.COM
security = ads
auth methods = winbind
password server = server.ddomain.com
winbind separator = +
encrypt passwords = yes
winbind cache time = 15
winbind enum users = yes
winbind enum groups = yes
winbind use default domain = yes
idmap uid = 1-2
idmap gid = 1-2
local master = no
os level = 233
domain master = no
preferred master = no
domain logons = no
wins server = 10.0.0.249
dns proxy = no
ldap ssl = no


my squid.conf
http_port 3128

hierarchy_stoplist cgi-bin ?

acl QUERY urlpath_regex cgi-bin \?
cache deny QUERY

acl apache rep_header Server ^Apache
broken_vary_encoding allow apache
access_log /var/log/squid/access.log squid

auth_param ntlm program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-ntlmssp
auth_param ntlm children 10

auth_param basic program /usr/bin/ntlm_auth --helper-protocol=squid-2.5-basic
auth_param basic children 10
auth_param basic realm Proxy Server
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive off
authenticate_cache_garbage_interval 10 seconds

authenticate_ttl 0 seconds

refresh_pattern ^ftp:   144020% 10080
refresh_pattern ^gopher:14400%  1440
refresh_pattern .   0   20% 4320
negative_ttl 5 minutes

acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8

acl SSL_ports port 443
acl Safe_ports port 80  # http
acl Safe_ports port 21  # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70  # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http

acl CONNECT method CONNECT

http_access allow manager localhost
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_reply_access allow all
icp_access allow all

external_acl_type nt_group %LOGIN /usr/lib/squid/wbinfo_group.pl
acl AllowedWindowsGroups external nt_group internet
http_access allow AllowedWindowsGroups
http_access deny all

the command wbinfo -u and -g works fine. no problems to join the domain.
my problem is some users when connecting msn messenger specially don't
register in the acces.log only the username of the user. sometimes
access to other sites don't log only the username too.

some sugestion to fix this?

Thanks
Wilson Galafassi


[squid-users] cache_dir and max-size

2007-08-10 Thread RW
If I split my cache like this:

cache_dir aufs /usr/local/squid/small_obj 1700 16 256 max-size=80
cache_dir aufs /usr/local/squid/big_obj200 16 256

Presumably nothing bigger than max-size will end-up in the first
cache, but how often will the smaller objects end-up in the second
cache?  Assuming that both cache-directories have filled.






[squid-users] Redirect POST Requests

2007-08-10 Thread Santiago Del Castillo
Hi!

Is possible to handle GET and POST requests separately? I want GET  to
go to port 80 and POST to port 81 in the same origin server.

I'm using squid 2.6.STABLE13 as reverse-proxy.

Thanks!

-- 
Santiago del Castillo
System Administrator
FNBOX Ventures Inc.
ARG: +54.11.5258.4202
[EMAIL PROTECTED]
http://www.fnbox.com


Re: [squid-users] page not being cached... is this right?

2007-08-10 Thread Henrik Nordstrom

> HTTP/1.x 200 OK
> Server: Apache-Coyote/1.1
> Set-Cookie: prtl_2048=2052; Expires=Sat, 11-Aug-2007 00:58:16 GMT
> Content-Type: text/html;charset=UTF-8
> Date: Fri, 10 Aug 2007 14:58:20 GMT
> X-Cache: MISS from pc-am.siege.ours.local
> X-Cache-Lookup: MISS from pc-am.siege.ours.local:3128
> Via: 1.0 pc-am.siege.ours.local:3128 (squid/2.6.STABLE14)
> Connection: close

Missing any freshness information (Expires / Cache-Control: max-age) so
this won't get cached by default.

Note: The Set-Cookie header do not prevent caching. But it will be
automatically stripped from any cache hits to the same object.

Recommended reading:

The results of the Cacheability Engine when used on your site
http://www.mnot.net/cacheability/

Caching Tutorial for Web Authors and Webmasters
http://www.mnot.net/cache_docs/

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Squid performance

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 19:03 +0400, Alexey Ivlev wrote:

> How many requests per second can squid handle?

Without cache and not too complex ACLs about 700-2000 depending on the
CPU power. It uses a single CPU only so multicore CPUs or SMP don't
improve the situation much.

On multicore/SMP servers it's possible to run more than one instance however.

Note: Hyperthreading does not add anything for Squid, sometimes even degrades 
performance.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 11:24 -0300, Alexandre Correa wrote:
> using aufs works fine..
> 
> my server receives about 4000 to 6000 req/min !!

quite modest load. Not a very high load.

> file system of hard disk is reiserfs4 !!
> 
> USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
> squid32748  1.1 35.0 1493184 1446748 ? Sl   Aug09   8:17 (squid) -D -s
> 
> the server is very clean, running squid + QUAGGA/OSPF only... i/o loop
> using epoll..
> 
> I thought because diskd has an external procces it uses more threads
> .. later i will make more tests using diskd and post the new results
> !!

diskd uses one single-threaded external process per cache_dir.

aufs uses posix threads for I/O, a number of them in parallell, number
increasing somewhat by the number of cache dirs..

Regards
Henrik



signature.asc
Description: This is a digitally signed message part


Re: [squid-users] endless growing swap.state after reboot

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 13:55 -0300, Michel Santos wrote:

> just to get it straight
> 
> when it fails I shut squid down again

yes.

> I wipe out the cache_dirs and recreate them?

yes, this to verify that the cause is swap.state.

> I copy the former original (first) backup swap.state back in place

yes.

> I start squid with this former swap.state but empty cache_dirs

yes.

> Is it that exactly?

yes.


But before you do that we perhaps should do the same, but without
erasing the cache directories.

> swap.state should shrink at this stage, eliminating it's reference when
> not finding the file right?

only if the rebuild is successful, in which case this test failed..

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


[squid-users] Can't compile Squid 2.6 on Solaris 10

2007-08-10 Thread John Yatsko, Jr.

Making install in lib
Making install in scripts
test -z "/usr/local/squid/bin" || 
/SQUID/squid-2.6.STABLE14/cfgaux/install-sh -"

//usr/sbin/install -c 'RunCache' '/usr/local/squid/bin/RunCache'
cp: cannot access /usr/local/squid/bin/RunCache
install: cp /usr/local/squid/bin/RunCache RunCache/RunCache failed
*** Error code 2
The following command caused the error:
list='RunCache'; for p in $list; do \
if test -f "$p"; then d=; else d="./"; fi; \
if test -f $d$p; then \
  f=`echo "$p" | sed 's|^.*/||;s,x,x,'`; \
  echo " //usr/sbin/install -c '$d$p' '/usr/local/squid/bin/$f'"; \
  //usr/sbin/install -c "$d$p" "/usr/local/squid/bin/$f"; \
else :; fi; \
done
make: Fatal error: Command failed for target `install-binSCRIPTS'
Current working directory /SQUID/squid-2.6.STABLE14/scripts
*** Error code 1
The following command caused the error:
make  install-exec-am install-data-am
make: Fatal error: Command failed for target `install-am'
Current working directory /SQUID/squid-2.6.STABLE14/scripts
*** Error code 1
The following command caused the error:
failcom='exit 1'; \
for f in x $MAKEFLAGS; do \
case $f in \
  *=* | --[!k]*);; \
  *k*) failcom='fail=yes';; \
esac; \
done; \
dot_seen=no; \
target=`echo install-recursive | sed s/-recursive//`; \
list='lib  scripts src icons errors doc helpers tools'; for subdir in $list; 
do\

echo "Making $target in $subdir"; \
if test "$subdir" = "."; then \
  dot_seen=yes; \
  local_target="$target-am"; \
else \
  local_target="$target"; \
fi; \
(cd $subdir && make  $local_target) \
|| eval $failcom; \
done; \
if test "$dot_seen" = "no"; then \
make  "$target-am" || exit 1; \
fi; test -z "$fail"
make: Fatal error: Command failed for target `install-recursive'


Thank you,

John Yatsko, Jr.
Technology Assistant
Erie County Public Library
160 East Front St
Erie PA 16507
(814) 451-7307 



Re: [squid-users] endless growing swap.state after reboot

2007-08-10 Thread Michel Santos

Henrik Nordstrom disse na ultima mensagem:

>> then I start squid with one of the above versions and squid starts
>> rebuilding swap.state
>>
>> when it starts failing we get what you want?
>
> That you try the same again, by shutting down Squid, then clear the
> cache and restore the backed up swap.state files and start Squid again.
> Hopefully the problem will manifest itself again, if so then there is an
> frozen state which produces the problem, and which can be debugged
> further to isolate what goes wrong.
>

just to get it straight

when it fails I shut squid down again

I wipe out the cache_dirs and recreate them?

I copy the former original (first) backup swap.state back in place

I start squid with this former swap.state but empty cache_dirs

Is it that exactly?

swap.state should shrink at this stage, eliminating it's reference when
not finding the file right?



Michel


...





Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.




Re: [squid-users] page not being cached... is this right?

2007-08-10 Thread Pablo García
Anton, in my experience, this header is what it's preventing the squid
from caching the page.

Set-Cookie: prtl_2048=2052; Expires=Sat, 11-Aug-2007 00:58:16 GMT

Regards, Pablo

On 8/10/07, Anton Melser <[EMAIL PROTECTED]> wrote:
> Hi,
> I have installed squid 2.6stable14 (windows binary linked from the
> site), and am getting a fair proportion of what should be cached
> cached, but not the most important things!
> I have deactivated the default setting to ignore urls with ? in it,
> and am getting all those pages/images except this page with cache
> hits. However, those have cache-expires headers and are cache-control
> public... Am I missing something with the below headers that means it
> won't be cached?
> Thanks for your time!
>
> http://www.mysite.test:3128/a/b/?nav_cat=128&lang=en_US
>
> GET /a/b/?nav_cat=128&lang=en_US HTTP/1.1
> Host: www.mysite.test:3128
> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.6)
> Gecko/20070725 Firefox/2.0.0.6
> Accept: 
> text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
> Accept-Language: fr,en;q=0.8,fr-fr;q=0.5,en-us;q=0.3
> Accept-Encoding: gzip,deflate
> Accept-Charset: UTF-8,*
> Keep-Alive: 300
> Connection: keep-alive
> Cookie: prtl_2048=2052; prtl_2048=2052;
> __utmz=196985131.1183130363.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
> __utmb=196985131;
> __utma=196985131.1319757679.1183130363.1186756561.1186757873.6;
> JSESSIONID=82F28C32D70FD2B8E87CF5F93F3B392A; __utmc=196985131
>
> HTTP/1.x 200 OK
> Server: Apache-Coyote/1.1
> Set-Cookie: prtl_2048=2052; Expires=Sat, 11-Aug-2007 00:58:16 GMT
> Content-Type: text/html;charset=UTF-8
> Date: Fri, 10 Aug 2007 14:58:20 GMT
> X-Cache: MISS from pc-am.siege.ours.local
> X-Cache-Lookup: MISS from pc-am.siege.ours.local:3128
> Via: 1.0 pc-am.siege.ours.local:3128 (squid/2.6.STABLE14)
> Connection: close
> --
>
> store.log
> 1186757048.978 RELEASE -1  ED746D4F06EEF75757282B4CA0B25510
> 200 1186757048-1-1 text/html -1/42930 GET
> http://www.mysite.test/a/b/?
>
> access.log
> 1186757998.370   5032 127.0.0.1 TCP_MISS/200 43227 GET
> http://www.mysite.test/a/b/? - FIRST_UP_PARENT/server_1 text/html
>


Re: [squid-users] endless growing swap.state after reboot

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 12:18 -0300, Michel Santos wrote:

> so let's mount the scenario
> 
> I shutdown squid letting rc.shutdown killing the squid process before it
> had time to close correctly the cache_dirs
> 
> then I backup swap.state

correct, including what other files there may be next to it..

> or do I backup before shutting down?

no, we want the swap.state files which is triggering the problem,
assuming it's swap.state content somehow triggering this problem...

> then I start squid with one of the above versions and squid starts
> rebuilding swap.state
> 
> when it starts failing we get what you want?

That you try the same again, by shutting down Squid, then clear the
cache and restore the backed up swap.state files and start Squid again.
Hopefully the problem will manifest itself again, if so then there is an
frozen state which produces the problem, and which can be debugged
further to isolate what goes wrong.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


[squid-users] page not being cached... is this right?

2007-08-10 Thread Anton Melser
Hi,
I have installed squid 2.6stable14 (windows binary linked from the
site), and am getting a fair proportion of what should be cached
cached, but not the most important things!
I have deactivated the default setting to ignore urls with ? in it,
and am getting all those pages/images except this page with cache
hits. However, those have cache-expires headers and are cache-control
public... Am I missing something with the below headers that means it
won't be cached?
Thanks for your time!

http://www.mysite.test:3128/a/b/?nav_cat=128&lang=en_US

GET /a/b/?nav_cat=128&lang=en_US HTTP/1.1
Host: www.mysite.test:3128
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.6)
Gecko/20070725 Firefox/2.0.0.6
Accept: 
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: fr,en;q=0.8,fr-fr;q=0.5,en-us;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 300
Connection: keep-alive
Cookie: prtl_2048=2052; prtl_2048=2052;
__utmz=196985131.1183130363.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none);
__utmb=196985131;
__utma=196985131.1319757679.1183130363.1186756561.1186757873.6;
JSESSIONID=82F28C32D70FD2B8E87CF5F93F3B392A; __utmc=196985131

HTTP/1.x 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: prtl_2048=2052; Expires=Sat, 11-Aug-2007 00:58:16 GMT
Content-Type: text/html;charset=UTF-8
Date: Fri, 10 Aug 2007 14:58:20 GMT
X-Cache: MISS from pc-am.siege.ours.local
X-Cache-Lookup: MISS from pc-am.siege.ours.local:3128
Via: 1.0 pc-am.siege.ours.local:3128 (squid/2.6.STABLE14)
Connection: close
--

store.log
1186757048.978 RELEASE -1  ED746D4F06EEF75757282B4CA0B25510
200 1186757048-1-1 text/html -1/42930 GET
http://www.mysite.test/a/b/?

access.log
1186757998.370   5032 127.0.0.1 TCP_MISS/200 43227 GET
http://www.mysite.test/a/b/? - FIRST_UP_PARENT/server_1 text/html


Re: [squid-users] endless growing swap.state after reboot

2007-08-10 Thread Michel Santos

Henrik Nordstrom disse na ultima mensagem:
> On tor, 2007-08-09 at 14:25 -0300, Michel Santos wrote:
>
>> ok the first is easy, the latter you mean what, you want the file?
>
> Unfortunately the file is a bit platform dependent, but I want you to
> hold on to the file and check if the problem can be reproduced by simply
> placing it back in the cache dir.
>

so let's mount the scenario

I shutdown squid letting rc.shutdown killing the squid process before it
had time to close correctly the cache_dirs

then I backup swap.state

or do I backup before shutting down?

then I start squid with one of the above versions and squid starts
rebuilding swap.state

when it starts failing we get what you want?


Michel

...





Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.




[squid-users] Squid performance

2007-08-10 Thread Alexey Ivlev
Hello, all!



We are creating large proxy instalation with content filtering.

Squid is defined as a proxy. Cache is not needed.



How many requests per second can squid handle?



Sure it depends on cpu,memory,hdd - suppose we can take the best hardware.

Anyway this will be cluster with network balancing, so the question is about 
maximum performance for one squid.



Will IBM RISC CPU encrease performance?



Recommendations about hardware and software will be much appreciated.



Thanks in advance.


Re: [squid-users] error during make

2007-08-10 Thread Alex Rousskov
On Fri, 2007-08-10 at 12:43 +0700, zen wrote:

> core# g++ -DHAVE_CONFIG_H -I. -I. -I../include -I../include -I../include 
> -Werror -Wall -Wpointer-arith -Wwrite-strings -Wcomments -D_REENTRANT -g 
> -O2 -MT MemPool.o -MD -MP -MF ".deps/MemPool.Tpo" -E -o MemPool.E MemPool.cc
> core# fgrep -3 mallopt MemPool.E
> core#

OK. So there is no mallopt in your preprocessed MemPool.E, as expected.

Let's see whether g++ will suddenly find it if you compile MemPool.E.
Run the same compilation command, but with a -c option at the end and
adjusted output/input file names:

g++ -DHAVE_CONFIG_H -I. -I. -I../include -I../include -I../include -Werror 
-Wall -Wpointer-arith -Wwrite-strings -Wcomments -D_REENTRANT -g -O2 -MT 
MemPool.o -MD -MP -MF ".deps/MemPool.Tpo" -c -o MemPool.o MemPool.E

If it fails, email me your MemPool.E along with the g++ output.

If it succeeds, we would have a case of two pretty much identical
compilations producing different results. If you recall, nearly the same
command failed when you ran "make" (we just ran the preprocessing step
manually). If this happens, try running "
make -k distclean;
./configure ... && make
again.

Thank you,

Alex.




[squid-users] cache_peer

2007-08-10 Thread [EMAIL PROTECTED]
Hello,
I want to set up a squid server on machine A(P).
This Machine A(P) is behind a firewall and it has an access to a corporate
proxy B(P).
B(P) has some access control lists. And A(P) is allowed to use the B(P).
When a browser is run on A(P) to connect to any web site using the proxy
B(P) it is ok.

So a machine C is not allowed for B(P).
Running a browser on C Using A(P) as a proxy results
Access Denied (403) which comes from B(P)...

I want to use B(P) as a parent proxy for A(P). But A(P) shows the client C
as the request owner to B(P). So B(P) refuses.

Any help.

My current configuration includes - on A(P) squid.conf
cache_peer B(P) parent 8080 0
...
forwarded_for off


mail2web LIVE – Free email based on Microsoft® Exchange technology -
http://link.mail2web.com/LIVE




Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Alexandre Correa
using aufs works fine..

my server receives about 4000 to 6000 req/min !!

file system of hard disk is reiserfs4 !!

USER   PID %CPU %MEMVSZ   RSS TTY  STAT START   TIME COMMAND
squid32748  1.1 35.0 1493184 1446748 ? Sl   Aug09   8:17 (squid) -D -s

the server is very clean, running squid + QUAGGA/OSPF only... i/o loop
using epoll..

I thought because diskd has an external procces it uses more threads
.. later i will make more tests using diskd and post the new results
!!

regards !!

AlexandrE
On 8/10/07, Michel Santos <[EMAIL PROTECTED]> wrote:
>
> Alexandre Correa disse na ultima mensagem:
> > after reading this email, i switched from aufs to diskd to see
> > performance of them under high load ..
> >
> > with aufs, squid never used more than 10% of cpu and response time is
> > very low (5ms to 150ms).. with diskd cpu usage goes to 50% +- and
> > median response time up to 900ms !!
> >
> > i´m running CentOS 5.0 with kernel 2.6.22, quad opteron 64 bits with
> > 4gb ram and hd are SAS 15.000 rpm
> >
>
>
> don't know anything about Centos but when a Quad Opteron does not handle
> the load you obviously have something wrong in your config, either squid
> or OS settings
>
>
> Michel
>
>
>
> ...
>
>
>
>
> 
> Datacenter Matik http://datacenter.matik.com.br
> E-Mail e Data Hosting Service para Profissionais.
> 
>
>


-- 

Sds.
Alexandre J. Correa
Onda Internet / OPinguim.net
http://www.ondainternet.com.br
http://www.opinguim.net


Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Michel Santos

Alexandre Correa disse na ultima mensagem:
> after reading this email, i switched from aufs to diskd to see
> performance of them under high load ..
>
> with aufs, squid never used more than 10% of cpu and response time is
> very low (5ms to 150ms).. with diskd cpu usage goes to 50% +- and
> median response time up to 900ms !!
>
> i´m running CentOS 5.0 with kernel 2.6.22, quad opteron 64 bits with
> 4gb ram and hd are SAS 15.000 rpm
>


don't know anything about Centos but when a Quad Opteron does not handle
the load you obviously have something wrong in your config, either squid
or OS settings


Michel



...





Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.




Re: [squid-users] endless growing swap.state after reboot

2007-08-10 Thread Henrik Nordstrom
On tor, 2007-08-09 at 14:25 -0300, Michel Santos wrote:

> ok the first is easy, the latter you mean what, you want the file?

Unfortunately the file is a bit platform dependent, but I want you to
hold on to the file and check if the problem can be reproduced by simply
placing it back in the cache dir.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: Re[4]: [squid-users] Digest auth trouble

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 15:18 +0500, Sergey Svyatkin wrote:
> Hello, Henrik.
> 
> You wrote at 10/08/2007 15:09:53:
> 
> HN> Please file a bug report http://www.squid-cache.org/bugs/
> 
> ? I have already sent... Bug 2046.

Thanks. Wasn't clear from your message that you had done this.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Alexandre Correa
after reading this email, i switched from aufs to diskd to see
performance of them under high load ..

with aufs, squid never used more than 10% of cpu and response time is
very low (5ms to 150ms).. with diskd cpu usage goes to 50% +- and
median response time up to 900ms !!

i´m running CentOS 5.0 with kernel 2.6.22, quad opteron 64 bits with
4gb ram and hd are SAS 15.000 rpm


attached graphic of cpu usage !!


On 8/10/07, Michel Santos <[EMAIL PROTECTED]> wrote:
>
> Henrik Nordstrom disse na ultima mensagem:
> > On fre, 2007-08-10 at 06:50 -0300, Michel Santos wrote:
>
> >> what is what you agree to be broken beyond the shutdown issue?
> >
> > Bug #761 unstable under high load when using diskd cache_dir
> >
> > diskd falls over under load due to internal design problems in how it
> > maintains callback queues. Duane fixed most of it quite recently so it's
> > no longer near as bad as it has been, but there is still stuff to do.
> > The problems was first reported 5 years ago.
> >
>
> indeed the cpu load went extremly down after this changes, I won on much
> machines more then 30-40%, or better 70/80% cpu load felt down to 30-40%
> overall. That was very good
>
> but I could get araound of it before and still do using at least 2 or
> better 4 or more diskd processes
>
> >> ok you mean threads instead of pthreads right?
> >
> > I don't know the FreeBSD thread packages very well to call them by name.
> > I only know there is two posix threads implementations. One userspace
> > which is what has been around for a long time and can not support aufs
> > with any reasonable performance, and a new one in more current releases
> > using kernel threads which is quite capable of supporting aufs.
>
> it it pthread versus thr (kernel threads) and who is interested, it's easy
> to do on 6.2 by creating /etc/libmap.conf or adding if exist, no further
> compile thing is necessary
>
> [/usr/local/squid/sbin/squid]
> libpthread.so.2 libthr.so.2
> libpthread.so   libthr.so
>
>
>
> Michel
> ...
>
>
>
>
> 
> Datacenter Matik http://datacenter.matik.com.br
> E-Mail e Data Hosting Service para Profissionais.
> 
>
>


-- 

Sds.
Alexandre J. Correa
Onda Internet / OPinguim.net
http://www.ondainternet.com.br
http://www.opinguim.net
<>

Re: [squid-users] File Descriptors causing an issue in OpenBSD

2007-08-10 Thread Tek Bahadur Limbu
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Fri, 10 Aug 2007 01:17:21 +0530
Preetish <[EMAIL PROTECTED]> wrote:

> > >Odd.. are you sure you are really running the new binary, and that the
> > >ulimit setting is done correctly in the start script?
> 
> #Squid startup/shutdown
> 
> if [ -z $1 ] ; then
> echo -n "Syntax is: $0 start stop"
> exit
> fi
> 
> if [ $1 != start -a $1 != stop ]; then
> echo -n "Wrong command"
> exit
> fi
> 
> if [ -x /usr/local/sbin/squid ]; then
> if [ $1 = 'start' ] ; then
> echo -n 'Running Squid: ';ulimit -HSn 8192;
> /usr/local/sbin/squid
> else
> echo -n 'Killing Squid: ';  /usr/local/sbin/squid
> -k shutdown
> fi
> else
> echo -n 'Squid not found'
> fi
> 
> 
> d> What do you get when you issue the following 2 commands:
> > limits
> No command limit.
> > and
> >
> > ulimit -n
> 
> 1024

Hi Preetish,

That shows that you have only 1024 file descriptors available on your system. 
In my FreeBSD machines, I usually don't have to adjust file descriptors because 
the defaults are more than I need (7000 - 14000). 


> 
> > kern.maxfiles
> > kern.maxfilesperproc
> 
> i did
> sysctl -w  kern.maxfiles=8192
> sysctl -w  kern.maxfilesperproc=8192 ---> this gives a error

I guess you don't have the kern.maxfilesperproc variable.

What do you have for your kern.maxusers variable?

If nothing helps, you may have to re-compile your kernel with the following 
added parameter:

option   MAXFILES=8192

But still, I think that there are other ways to increase your file descriptors 
besides re-compiling your kernel.

You can ask for help in the openbsd mailing list regarding your problem.

> 
> Then i even made changes the Options in /etc/login.def
> {{
> default:\
> :path=/usr/bin /bin /usr/sbin /sbin /usr/X11R6/bin /usr/local/bin:\
> :umask=022:\
> :datasize-max=512M:\
> :datasize-cur=512M:\
> :maxproc-max=512:\
> :maxproc-cur=64:\
> :openfiles-cur=8192:\
> :stacksize-cur=4M:\
> :localcipher=blowfish,6:\
> :ypcipher=old:\
> :tc=auth-defaults:\
> :tc=auth-ftp-defaults:
> }}
> 
> and
> 
> {{
> daemon:\
> :ignorenologin:\
> :datasize=infinity:\
> :maxproc=infinity:\
> :openfiles-cur=8192:\
> :stacksize-cur=8M:\
> :localcipher=blowfish,8:\
> :tc=default:
> }}
> 
> and after doing all these changes i uninstalled squid completely and
> all its file and everything .Then recompiled it and installed it
> againBut DAMM it gave me the same number of file descriptors. So
> now i have reduced the cache to 10 GB. I found a Squid Definitive
> guide where he said to recompile the kernel after editing the kernel
> configuration file .

Reducing just the size of your cache may not be able to help you much with your 
file descriptors limit. 

> 
> 
> Squid Object Cache: Version 2.6.STABLE13
> Start Time: Thu, 09 Aug 2007 19:09:36 GMT
> Current Time:   Thu, 09 Aug 2007 19:11:13 GMT
> Connection information for squid:
> Number of clients accessing cache:  321
> Number of HTTP requests received:   2649
> Number of ICP messages received:0
> Number of ICP messages sent:0
> Number of queued ICP replies:   0
> Request failure ratio:   0.00
> Average HTTP requests per minute since start:   1638.4
> Average ICP messages per minute since start:0.0
> Select loop called: 34876 times, 2.782 ms avg
> Cache information for squid:
> Request Hit Ratios: 5min: 15.1%, 60min: 15.1%
> Byte Hit Ratios:5min: 29.4%, 60min: 29.4%
> Request Memory Hit Ratios:  5min: 9.7%, 60min: 9.7%
> Request Disk Hit Ratios:5min: 44.4%, 60min: 44.4%
> Storage Swap size:  23806 KB
> Storage Mem size:   2516 KB
> Mean Object Size:   7.57 KB
> Requests given to unlinkd:  0
> Median Service Times (seconds)  5 min60 min:
> HTTP Requests (All):   0.68577  0.68577
> Cache Misses:  1.24267  1.24267
> Cache Hits:0.00179  0.00179
> Near Hits: 0.68577  0.68577
> Not-Modified Replies:  0.00091  0.00091
> DNS Lookups:   0.00190  0.00190
> ICP Queries:   0.0  0.0


- From your above data, your service response times which are under 1.5 seconds 
are good figures for a satellite link. Before, it was 15 seconds! Considering 
that, your proxy server is much faster now. But since your data above is only 2 
minutes old, you have to monitor in regularly for a longer period of time. 

Starting with a cache_dir size of 10GB is good start. You can later increase 
it's size based upon your needs and demands.


Thanking you...



> 
> 
> :(((
> 
> Preetish
> 


- -- 

With best regards an

Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Michel Santos

Henrik Nordstrom disse na ultima mensagem:
> On fre, 2007-08-10 at 06:50 -0300, Michel Santos wrote:

>> what is what you agree to be broken beyond the shutdown issue?
>
> Bug #761 unstable under high load when using diskd cache_dir
>
> diskd falls over under load due to internal design problems in how it
> maintains callback queues. Duane fixed most of it quite recently so it's
> no longer near as bad as it has been, but there is still stuff to do.
> The problems was first reported 5 years ago.
>

indeed the cpu load went extremly down after this changes, I won on much
machines more then 30-40%, or better 70/80% cpu load felt down to 30-40%
overall. That was very good

but I could get araound of it before and still do using at least 2 or
better 4 or more diskd processes

>> ok you mean threads instead of pthreads right?
>
> I don't know the FreeBSD thread packages very well to call them by name.
> I only know there is two posix threads implementations. One userspace
> which is what has been around for a long time and can not support aufs
> with any reasonable performance, and a new one in more current releases
> using kernel threads which is quite capable of supporting aufs.

it it pthread versus thr (kernel threads) and who is interested, it's easy
to do on 6.2 by creating /etc/libmap.conf or adding if exist, no further
compile thing is necessary

[/usr/local/squid/sbin/squid]
libpthread.so.2 libthr.so.2
libpthread.so   libthr.so



Michel
...





Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.




Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Michel Santos
Adrian Chadd disse na ultima mensagem:
> On Thu, Aug 09, 2007, Michel Santos wrote:
>
>> > the bug, I am curious what others have been using or prefer as their
>> > alternative to diskd and why?
>>
>> diskd for sure is the fastest specially on SMP machines but there are
>> not
>> so much people sharing my opinion ...
>
> Just supply real-world numbers showing which is faster.
>

oook, let's agree first what fast means fast here since fast can be
relative depending on who sense the speed and what he is used to right ...

when I say speed I mean especially response time which then often depends
on local network and wan connection latency and server quality (hardware)
so then it's kind of hard to measure that all together. Like you know
well, often squid might be blamed for performance problems and in the end
it was something else.

But then perhaps a req/hit relationsship satisfies your curiousity? Then
have a look at the image attached which shows a average server I have.

> Remember - the overlap between the people doing the development and the
> people saving/making money using Squid is almost 0..
>

hum, may be may be not. Problem here is that most people have one or two
servers (if) and eventually do not have enough real life data to reflect
the hundreds of different situations we find in the wild. Also a corporate
or home frontend proxy running nat and controlling internet access
probably is not exactly a performance relative comparism since such a
machine never comes to it's limits nor has much to do in means of cache
functions

"people saving/making" money I guess are for you those who sell their
consultant services but for me would be those who use squid for spending
less or getting more out of their internet connection - or shorter -
interested in it's cache funcionality only

so you see a bunch of different purposes and basics which are not easy to
compare in general statements as you are used to

technically speaking we do have 4 fs as choice and to not forget, this
thread is dedicated to freebsd and I have no idea about linux and less
about windows

so then first we discard ufs as good, stable and standard and we discard
coss because of it's kind of excessive startup time of 1-3 hours ... ;)

then we have left aufs and diskd for performance geeks

aufs is good but not good enough it starts choking same way as ufs under
load and this happens on the exat same hardware as the diskd I tell you
next. IMO this is happening because of missing real SMP support. may be
this is wrong and other things are making the difference here but don't
forget on freebsd our choice is ufs2 and eventually this does not work
exactly as extN on Linux

diskd probably is not very much used since it needs SHM/IPC tuning and
that is not as easy as it seems so my guess most people do not even try it
(no offense). Diskd by it's own runs several processes, one per cache_dir
what makes it naturally more SMP friendly as any other fs squid offers.

diskd also is lightning fast when configured well, specially under load
and I like to remember terabytes of databases using the same technology
with success since years so it can not be so bad ...

then resuming, for me, diskd is my choice on loaded servers and choked
links because it is faster for my application as a transparent frontend
cache on the only network router in an ISP environment. I am using diskd
since it came out and sure I ever tried the other options but none came
close.




Michel

...





Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.
<>

Re[4]: [squid-users] Digest auth trouble

2007-08-10 Thread Sergey Svyatkin
Hello, Henrik.

You wrote at 10/08/2007 15:09:53:

HN> Please file a bug report http://www.squid-cache.org/bugs/

? I have already sent... Bug 2046.

-- 
WBR,
 Sergey Svyatkin  mailto:[EMAIL PROTECTED]



Re: Re[2]: [squid-users] Digest auth trouble

2007-08-10 Thread Henrik Nordstrom
Please file a bug report

http://www.squid-cache.org/bugs/


On fre, 2007-08-10 at 11:30 +0500, Sergey Svyatkin wrote:


> HN> Please get a stack trace and file a bug report.
> 
> See this:
> 
> [EMAIL PROTECTED] /usr/local/squid/cache]# gdb squid squid.core
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-marcel-freebsd"...
> Core was generated by `squid'.
> Program terminated with signal 6, Aborted.
> Reading symbols from /lib/libcrypt.so.3...done.
> Loaded symbols for /lib/libcrypt.so.3
> Reading symbols from /lib/libm.so.4...done.
> Loaded symbols for /lib/libm.so.4
> Reading symbols from /lib/libc.so.6...done.
> Loaded symbols for /lib/libc.so.6
> Reading symbols from /libexec/ld-elf.so.1...done.
> Loaded symbols for /libexec/ld-elf.so.1
> #0  0x28218ecb in kill () from /lib/libc.so.6
> (gdb) where
> #0  0x28218ecb in kill () from /lib/libc.so.6
> #1  0x28218e68 in raise () from /lib/libc.so.6
> #2  0x28217b78 in abort () from /lib/libc.so.6
> #3  0x281f3db8 in __assert () from /lib/libc.so.6
> #4  0x080d456f in hash_remove_link (hid=0x80e29aa, hl=0x28229d80) at 
> hash.c:277
> #5  0x080d1143 in authDigestNoncePurge (nonce=0x9724b00)
> at digest/auth_digest.c:426
> #6  0x080d213b in authenticateDigestNonceCacheCleanup (data=0x0)
> at digest/auth_digest.c:281
> #7  0x0807e9c0 in eventRun () at event.c:148
> #8  0x0809e353 in main (argc=3, argv=0xbfbfec78) at main.c:832
> (gdb) quit
> 
> [EMAIL PROTECTED] /usr/local/squid/cache]# uname -a
> FreeBSD proxy.svgc.ru 6.2-RELEASE FreeBSD 6.2-RELEASE #1: Tue Jun  5 12:59:59 
> SAMST 2007
>  [EMAIL PROTECTED]:/usr/src/sys/i386/compile/PROXY  i386
> 


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 06:50 -0300, Michel Santos wrote:

> please be a little bit more specific about "comitting resources", what do
> you exactly mean?

Gettin a developer working on fixing the bugs.

> what is what you agree to be broken beyond the shutdown issue?

Bug #761 unstable under high load when using diskd cache_dir

diskd falls over under load due to internal design problems in how it
maintains callback queues. Duane fixed most of it quite recently so it's
no longer near as bad as it has been, but there is still stuff to do.
The problems was first reported 5 years ago.

> ok you mean threads instead of pthreads right?

I don't know the FreeBSD thread packages very well to call them by name.
I only know there is two posix threads implementations. One userspace
which is what has been around for a long time and can not support aufs
with any reasonable performance, and a new one in more current releases
using kernel threads which is quite capable of supporting aufs.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 16:54 +0800, Adrian Chadd wrote:

> And I'm saying it shouldn't, thats not how stuff was intended, and
> the fact that stuff "kind of sometimes mostly maybe" works is busted.

It doesn't. Squid never accepts to do authentication in interception
mode. Any attempt to do so will result in the following getting logged
in access.log:

aclAuthenticated: authentication not applicable on transparently
intercepted requests.

and the http_access line ignored.

> People keep -wanting- to try it though.

Indeed.

> We're in agreement!

Good.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Michel Santos

Henrik Nordstrom disse na ultima mensagem:
> On tor, 2007-08-09 at 10:18 -0700, Nicole wrote:
>> As some have pointed out, it's a shame diskd is horked, since it seemed
>> to be nice and fast.
>
> Well, it's been broken for several years now, an no one has been willing
> to commit any resources to get it fixed.
>

please be a little bit more specific about "comitting resources", what do
you exactly mean?


what is what you agree to be broken beyond the shutdown issue?


>> However, since I have not heard of any progress on fixing
>> the bug, I am curious what others have been using or prefer as their
>> alternative to diskd and why?
>
> aufs is seen as the best alternative currently, with FreeBSD also
> supporting kernel threads.
>
> Note: running aufs without kernel threads is a dead end and won't
> perform well, you might just as well run with the ufs cache_dir type
> then.
>

ok you mean threads instead of pthreads right?


Michel
...





Datacenter Matik http://datacenter.matik.com.br
E-Mail e Data Hosting Service para Profissionais.




Re: [squid-users] few questions around multiple cache_dirs

2007-08-10 Thread Henrik Nordstrom
On tor, 2007-08-09 at 14:08 -0700, Neil Harkins wrote:

> * "x-squid-internal/vary" stubs appear to be able to wind up on a
> different cache_dir than the object itself. Is this a bug?

It's not a bug, it's a design artefact. The stub and the object is
separate from each other, so there is only 1/N probability they will end
up on the same cache_dir just as for any other two objects (assuming
none of the max-/min-size options is used).

The risk of loosing the object due to loss of another cache_dir is not
considered important.

> * how does squid determine which of several cache_dirs has an object
> after a restart...

By reading the swap.state files, these contains the per-cache_dir object
indexes + transaction log.

> lookups performed, where N is the # of cache_dirs? Does an unclean
> shutdown/interrupted flush to swap.state completely invalidate all
> objects in a cache_dir,

varies. 

> Also, if entirely in memory, is it exempt from cache_mem limits?

cache_mem is only object storage in memory, not the meta data.

> * although i admittedly can't reproduce now, i earlier saw object
> files in the aufs cache_dir occasionally getting renamed(rewritten?)
> in the same cache_dir, incrementing the filename by 1 on each of
> multiple successive identical requests (same client). any idea what
> could account for this behavior?

Most likely the client forced a refresh of the object using
Control-Reload or similar.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] Opinions sought on best storage type for FreeBSD

2007-08-10 Thread Henrik Nordstrom
On tor, 2007-08-09 at 10:18 -0700, Nicole wrote:
> As some have pointed out, it's a shame diskd is horked, since it seemed
> to be nice and fast.

Well, it's been broken for several years now, an no one has been willing
to commit any resources to get it fixed.

> However, since I have not heard of any progress on fixing
> the bug, I am curious what others have been using or prefer as their
> alternative to diskd and why?

aufs is seen as the best alternative currently, with FreeBSD also
supporting kernel threads.

Note: running aufs without kernel threads is a dead end and won't
perform well, you might just as well run with the ufs cache_dir type
then.

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Henrik Nordstrom
On fre, 2007-08-10 at 09:18 +0800, Adrian Chadd wrote:
> On Thu, Aug 09, 2007, Henrik Nordstrom wrote:
> > On m??n, 2007-08-06 at 18:26 +0800, Adrian Chadd wrote:
> > 
> > > Look at how a browser talks directly to an origin server when presenting
> > > (HTTP Basic) authentication credentials, and what a proxy ends up doing
> > > with those.
> > 
> > What about it?
> 
> It doesn't work reliably? :)

Doesn't it? When?

Regards
Henrik


signature.asc
Description: This is a digitally signed message part


Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Neil A. Hillard
Hi,

Adrian Chadd wrote:
> On Fri, Aug 10, 2007, Neil A. Hillard wrote:
>> Hi,
>>
>> Adrian Chadd wrote:
>>> On Fri, Aug 10, 2007, Neil A. Hillard wrote:
>>>
> It doesn't work reliably? :)
 Doesn't it?  You'll have to cite specific examples.  I can't think of
 one problem I've had that's related to basic auth not working as it
 should (as long as you don't count configuration faux pas!)
>>> Transparent interception with proxy basic authentication?
>> Not valid - it was never designed to do that.  We repeat the question -
>> if the browser doesn't know a proxy is there then why should it
>> authenticate to it?
> 
> And I'm saying it shouldn't, thats not how stuff was intended, and
> the fact that stuff "kind of sometimes mostly maybe" works is busted.
> People keep -wanting- to try it though.
> 
> We're in agreement!

OK, matter settled!  I pity the next person to ask this question! :-)


Neil.

-- 
Neil Hillard[EMAIL PROTECTED]
AgustaWestland  http://www.whl.co.uk/

Disclaimer: This message does not necessarily reflect the
views of Westland Helicopters Ltd.


Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Adrian Chadd
On Fri, Aug 10, 2007, Neil A. Hillard wrote:
> Hi,
> 
> Adrian Chadd wrote:
> > On Fri, Aug 10, 2007, Neil A. Hillard wrote:
> > 
> >>> It doesn't work reliably? :)
> >> Doesn't it?  You'll have to cite specific examples.  I can't think of
> >> one problem I've had that's related to basic auth not working as it
> >> should (as long as you don't count configuration faux pas!)
> > 
> > Transparent interception with proxy basic authentication?
> 
> Not valid - it was never designed to do that.  We repeat the question -
> if the browser doesn't know a proxy is there then why should it
> authenticate to it?

And I'm saying it shouldn't, thats not how stuff was intended, and
the fact that stuff "kind of sometimes mostly maybe" works is busted.
People keep -wanting- to try it though.

We're in agreement!



Adrian



Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Neil A. Hillard
Hi,

Adrian Chadd wrote:
> On Fri, Aug 10, 2007, Neil A. Hillard wrote:
> 
>>> It doesn't work reliably? :)
>> Doesn't it?  You'll have to cite specific examples.  I can't think of
>> one problem I've had that's related to basic auth not working as it
>> should (as long as you don't count configuration faux pas!)
> 
> Transparent interception with proxy basic authentication?

Not valid - it was never designed to do that.  We repeat the question -
if the browser doesn't know a proxy is there then why should it
authenticate to it?


Neil.

-- 
Neil Hillard[EMAIL PROTECTED]
AgustaWestland  http://www.whl.co.uk/

Disclaimer: This message does not necessarily reflect the
views of Westland Helicopters Ltd.


Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Adrian Chadd
On Fri, Aug 10, 2007, Neil A. Hillard wrote:

> > It doesn't work reliably? :)
> 
> Doesn't it?  You'll have to cite specific examples.  I can't think of
> one problem I've had that's related to basic auth not working as it
> should (as long as you don't count configuration faux pas!)

Transparent interception with proxy basic authentication?




Adrian



[squid-users] Cant access internal webserver when using squid 3128

2007-08-10 Thread SSCR Internet Admin
Hi,

My network is intercepting port 80 to 3128 (transparent proxy) in accessing
the internet.  Lately, I have some internal webserver and is now redirecting
trafic from outside to that internal webserver.  If use squid, (ie.,
configuring proxy on firefox), it seems that I am blocked or denied by
squid.  If I use transparent proxy, everything is working perfectly good.  

On what part on squid.conf I can tweak on this matter? Thanks and more power


TIA


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.



Re: [squid-users] TRANSEPARENT PROXY WITH SQUID 2.6

2007-08-10 Thread Adrian Chadd


http_port 3128 transparent

That should do it.



Adrian

On Fri, Aug 10, 2007, Indunil Jayasooriya wrote:
> Hi,
> 
> I am running squid as a TRANSEPARENT PROXY WITH SQUID 2.5 on CENTOS 4.5.
> 
> Pls see below for squid.con file.
> 
> 
>  http_port 3128
>  cache_mem 64 MB
>  cache_dir ufs /var/spool/squid 100 16 256
>   cache_access_log /var/log/squid/access.log
>  cache_log /var/log/squid/cache.log
>  cache_store_log /var/log/squid/store.log
> 
> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> 
> # Example rule allowing access from your local networks. Adapt
> # to list your (internal) IP networks from where browsing should
> # be allowed
> 
> acl mynet src 192.168.101.0/24
> http_access allow mynet
> 
> httpd_accel_host virtual
>  httpd_accel_port 80
> httpd_accel_with_proxy on
> httpd_accel_uses_host_header on
> 
> 
> I am ruunnig itables on the SAME BOX. I have added below rules for squid.
> 
> #Redirecting traffic destined to port 80 to port 3128
>  iptables -t nat -A PREROUTING -p tcp -i eth2 --dport 80 -j REDIRECT
> --to-port 3128
> 
> #For squid traffic to Accept
> iptables -A INPUT -i eth2 -d 192.168.101.254 -p tcp -s
> 192.168.101.0/24 --dport 3128 -j ACCEPT
> 
> Evrything works fine.
> 
> Now, I am ging to upgrade CENTOS 4.5 to CENTOS 5. It comes with SQUID 2.6.
> 
> But, squid.conf file in SQUID 2.6 has no below lines.
> 
> httpd_accel_host virtual
>  httpd_accel_port 80
>  httpd_accel_with_proxy on
>  httpd_accel_uses_host_header on
> 
> So, I want to know how to setup squid as a TRANSEPARENT PROXY, when it
> comes to SQUID 2.6.
> 
> any idea?
> -- 
> Thank you
> Indunil Jayasooriya

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level bandwidth-capped VPSes available in WA -


[squid-users] TRANSEPARENT PROXY WITH SQUID 2.6

2007-08-10 Thread Indunil Jayasooriya
Hi,

I am running squid as a TRANSEPARENT PROXY WITH SQUID 2.5 on CENTOS 4.5.

Pls see below for squid.con file.


 http_port 3128
 cache_mem 64 MB
 cache_dir ufs /var/spool/squid 100 16 256
  cache_access_log /var/log/squid/access.log
 cache_log /var/log/squid/cache.log
 cache_store_log /var/log/squid/store.log

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS

# Example rule allowing access from your local networks. Adapt
# to list your (internal) IP networks from where browsing should
# be allowed

acl mynet src 192.168.101.0/24
http_access allow mynet

httpd_accel_host virtual
 httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on


I am ruunnig itables on the SAME BOX. I have added below rules for squid.

#Redirecting traffic destined to port 80 to port 3128
 iptables -t nat -A PREROUTING -p tcp -i eth2 --dport 80 -j REDIRECT
--to-port 3128

#For squid traffic to Accept
iptables -A INPUT -i eth2 -d 192.168.101.254 -p tcp -s
192.168.101.0/24 --dport 3128 -j ACCEPT

Evrything works fine.

Now, I am ging to upgrade CENTOS 4.5 to CENTOS 5. It comes with SQUID 2.6.

But, squid.conf file in SQUID 2.6 has no below lines.

httpd_accel_host virtual
 httpd_accel_port 80
 httpd_accel_with_proxy on
 httpd_accel_uses_host_header on

So, I want to know how to setup squid as a TRANSEPARENT PROXY, when it
comes to SQUID 2.6.

any idea?
-- 
Thank you
Indunil Jayasooriya


Re: [squid-users] username and password in TRANSPARENT mode

2007-08-10 Thread Neil A. Hillard
Hi,

Adrian Chadd wrote:
> On Thu, Aug 09, 2007, Henrik Nordstrom wrote:
>> On m??n, 2007-08-06 at 18:26 +0800, Adrian Chadd wrote:
>>
>>> Look at how a browser talks directly to an origin server when presenting
>>> (HTTP Basic) authentication credentials, and what a proxy ends up doing
>>> with those.
>> What about it?
> 
> It doesn't work reliably? :)

Doesn't it?  You'll have to cite specific examples.  I can't think of
one problem I've had that's related to basic auth not working as it
should (as long as you don't count configuration faux pas!)


Neil.

-- 
Neil Hillard[EMAIL PROTECTED]
AgustaWestland  http://www.whl.co.uk/

Disclaimer: This message does not necessarily reflect the
views of Westland Helicopters Ltd.


[squid-users] few questions around multiple cache_dirs

2007-08-10 Thread Neil Harkins
Hi. I'm in the early stages of designing and testing a config with
multiple aufs cache_dirs on squid-2.6.STABLE3 as httpd accel for a lot
of content, and have a few questions based on what I've observed thus
far:

* "x-squid-internal/vary" stubs appear to be able to wind up on a
different cache_dir than the object itself. Is this a bug? Or a
tradeoff in favor of performance in the cache_dir being available 99%
of the time case, rather than storing the stubs on the same cache_dir
so a failure of a disk containing one or the other doesn't invalidate
the object? (note: I'm using max-size, which may have contributed to
the splitting, as the stubs are small and the objects large).

* how does squid determine which of several cache_dirs has an object
after a restart... is the complete url->cachefile mapping stored in
swap.state and each completely loaded into memory at startup, or are N
lookups performed, where N is the # of cache_dirs? Does an unclean
shutdown/interrupted flush to swap.state completely invalidate all
objects in a cache_dir, or does it attempt to "fsck" the objects?
Also, if entirely in memory, is it exempt from cache_mem limits?

* although i admittedly can't reproduce now, i earlier saw object
files in the aufs cache_dir occasionally getting renamed(rewritten?)
in the same cache_dir, incrementing the filename by 1 on each of
multiple successive identical requests (same client). any idea what
could account for this behavior?

thanks,
-neil


[squid-users] cache replacement

2007-08-10 Thread Shabbir Ahmed


i would like to know how squid replaces old items in cache by default.

like there isa site called abc.com squid cached it today and next day 
abc.com made some changes what is the criteria squid ll cache changes and 
users ll get the newer pages.



Thanku,