Re: [squid-users] optimization suggestions?
On 01.10 08:41, Victor Medina wrote: I have changed the disk replacement policy to LFUDA I will also give a try to Async-UFS and COSS. But I cannot do it on week days due to traffic issues with costumers. ( I need to compile a new SQUID, and I will use the opportunity to upgrade squid and samba to the latest stable release, in one shot! =) you can easily switch to aufs (if you have compiled squid with it). how many requests do you have per second? also check disk use by iostat -d 1 to see if disk is the short place On the other side, after seeing my config, do you agree with the parameters i have used? any comments on memory, disk space? file system used(REISER btw.)? reisersfs and XFS are OK, memory is OK, I didn't say more because I don't know more about your system - dedicated disk(s) etc etc -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. You have the right to remain silent. Anything you say will be misquoted, then used against you.
Re: [squid-users] optimization suggestions?
On 30.09 09:08, Victor Medina wrote: I have a SQUID server running on a Piii 1100ghz server with 1024 RAM, SCSI disks. We have almost 100 users. And a 700Kbps ADSL Connection. The cache directory is in it's own scsi drive. I am seeing some very slow response in the proxy server. The incoming traffic in the internet connection is almost exactly the same as the outgoing traffic in the internal network, which basically means that the traffic from the internal proxy client is the same as the downloaded stuff. (am i right?) I have System Health graphics from both interfaces, I can send them to private mails if you want to take a look. DNS is being handled by a BIND server on the same machine. Most of the sites, that users can access are specified by acl's so they are very predictable, most users have _only_ access to certain sites. Even though there are a awful lot of users for a 700Kbps link, the site are almost always the same. The cache dir is 7gb in size. SQUID has plenty of RAM. File system being used is RAISER, and it's a linux machine running kernel 2.6 under SuSE 9.1. Anybody would be so kind and suggest any optimization that could somehow improve performance? I will copy the relevant information about my squid.conf. what I'd advise you - using LFUDA cache_replacement_policy (memory replacement policy may stay GSDF) - using bigger cache (if your 700 kbps link is loaded with HTTP requests, your users are probably downloading more than 7GB per week) - play with maximum_object_size and another question is, if squid eats a lot of CPU and memory - unless it's badly configured, it should not. -- Matus UHLAR - fantomas, [EMAIL PROTECTED] ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Save the whales. Collect the whole set.
Re: [squid-users] optimization suggestions?
Hi! I have changed the disk replacement policy to LFUDA I will also give a try to Async-UFS and COSS. But I cannot do it on week days due to traffic issues with costumers. ( I need to compile a new SQUID, and I will use the opportunity to upgrade squid and samba to the latest stable release, in one shot! =) On the other side, after seeing my config, do you agree with the parameters i have used? any comments on memory, disk space? file system used(REISER btw.)? Best Regards Victor Medina On Fri, 2004-10-01 at 07:41, Matus UHLAR - fantomas wrote: On 30.09 09:08, Victor Medina wrote: I have a SQUID server running on a Piii 1100ghz server with 1024 RAM, SCSI disks. We have almost 100 users. And a 700Kbps ADSL Connection. The cache directory is in it's own scsi drive. I am seeing some very slow response in the proxy server. The incoming traffic in the internet connection is almost exactly the same as the outgoing traffic in the internal network, which basically means that the traffic from the internal proxy client is the same as the downloaded stuff. (am i right?) I have System Health graphics from both interfaces, I can send them to private mails if you want to take a look. DNS is being handled by a BIND server on the same machine. Most of the sites, that users can access are specified by acl's so they are very predictable, most users have _only_ access to certain sites. Even though there are a awful lot of users for a 700Kbps link, the site are almost always the same. The cache dir is 7gb in size. SQUID has plenty of RAM. File system being used is RAISER, and it's a linux machine running kernel 2.6 under SuSE 9.1. Anybody would be so kind and suggest any optimization that could somehow improve performance? I will copy the relevant information about my squid.conf. what I'd advise you - using LFUDA cache_replacement_policy (memory replacement policy may stay GSDF) - using bigger cache (if your 700 kbps link is loaded with HTTP requests, your users are probably downloading more than 7GB per week) - play with maximum_object_size and another question is, if squid eats a lot of CPU and memory - unless it's badly configured, it should not.
[squid-users] optimization suggestions?
Hello all! I have a SQUID server running on a Piii 1100ghz server with 1024 RAM, SCSI disks. We have almost 100 users. And a 700Kbps ADSL Connection. The cache directory is in it's own scsi drive. I am seeing some very slow response in the proxy server. The incoming traffic in the internet connection is almost exactly the same as the outgoing traffic in the internal network, which basically means that the traffic from the internal proxy client is the same as the downloaded stuff. (am i right?) I have System Health graphics from both interfaces, I can send them to private mails if you want to take a look. DNS is being handled by a BIND server on the same machine. We are using ntlm auth via samba. We compiled our own SQUID server. CFLAGS=-O3 -march=i686 -mcpu=i686 SQUID was compiled as: Squid Cache: Version 2.5.STABLE5 configure options: --prefix=/opt/EPAWebCachingSuite-1.0-i686/ --sysconfdir=/opt/EPAWebCachingSuite-1.0-i686/etc/squid --with-dl --enable-snmp --enable-carp --enable-useragent-log '--enable-auth=basic digest ntlm' '--enable-basic-auth-helpers=MSNT SMB getpwnam multi-domain-NTLM winbind' '--enable-ntlm-auth-helpers=SMB no_check winbind' --enable-digest-auth-helpers=password --enable-ntlm-fail-open --enable-referer-log --enable-htcp --enable-underscores --enable-stackt races --enable-delay-pools --enable-ssl --enable-cache-digests --with-samba-sources=/home/vmedina/SOURCES/samba-2.2.9/ --enable-x-accelerator-vary --disable-ident-lookups --enable-truncate --enable-removal-policies=heap --enable-xmalloc-statistics --enable-linux-netfilter --enable-stacktraces Most of the sites, that users can access are specified by acl's so they are very predictable, most users have _only_ access to certain sites. Even though there are a awful lot of users for a 700Kbps link, the site are almost always the same. The cache dir is 7gb in size. SQUID has plenty of RAM. File system being used is RAISER, and it's a linux machine running kernel 2.6 under SuSE 9.1. Anybody would be so kind and suggest any optimization that could somehow improve performance? I will copy the relevant information about my squid.conf. icp_port 0 hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? cache_mem 192 MB cache_replacement_policy heap GDSF memory_replacement_policy heap GDSF cache_dir ufs /workload/cache 7168 16 256 cache_access_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/access.log cache_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/cache.log cache_store_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/store.log error_directory /opt/EPAWebCachingSuite-1.0-i686//share/errors/Spanish # DEFINICION DE LA AUTENTICACION CON NTLM auth_param ntlm program /opt/EPAWebCachingSuite-1.0-i686/libexec/wb_ntlmauth auth_param ntlm children 25 auth_param ntlm max_challenge_reuses 0 auth_param ntlm max_challenge_lifetime 15 minutes auth_param basic program /opt/EPAWebCachingSuite-1.0-i686/libexec/wb_auth auth_param basic children 25 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 6 hours # NO TOCAR! cache_mgr root cache_effective_user proxy cache_effective_group proxy logfile_rotate 0 log_icp_queries off cachemgr_passwd P6RsP6RsP6Rs all buffered_logs on delay_initial_bucket_level 50 delay_pools 0 Any suggestions or comments are welcome Thanxs! Victor.
Re: [squid-users] optimization suggestions?
On Thu, 30 Sep 2004, Victor Medina wrote: SQUID was compiled as: Squid Cache: Version 2.5.STABLE5 configure options: --prefix=/opt/EPAWebCachingSuite-1.0-i686/ --sysconfdir=/opt/EPAWebCachingSuite-1.0-i686/etc/squid --with-dl --enable-snmp --enable-carp --enable-useragent-log '--enable-auth=basic digest ntlm' '--enable-basic-auth-helpers=MSNT SMB getpwnam multi-domain-NTLM winbind' '--enable-ntlm-auth-helpers=SMB no_check winbind' --enable-digest-auth-helpers=password --enable-ntlm-fail-open --enable-referer-log --enable-htcp --enable-underscores --enable-stackt races --enable-delay-pools --enable-ssl --enable-cache-digests --with-samba-sources=/home/vmedina/SOURCES/samba-2.2.9/ --enable-x-accelerator-vary --disable-ident-lookups --enable-truncate --enable-removal-policies=heap --enable-xmalloc-statistics --enable-linux-netfilter --enable-stacktraces This does not answer your question, but why this excessive list of configure options? Rule of thumb: Only use a configure option if you know you need it. For most options there is reasons why they are not enabled by default. Regards Henrik
Re: [squid-users] optimization suggestions?
jejeje! quite right! but basically a repeat a SuSE setup. Which is fairly similar to this one. Any suggestion based on my config for a leaner squid? =) On Thu, 2004-09-30 at 09:52, Henrik Nordstrom wrote: On Thu, 30 Sep 2004, Victor Medina wrote: SQUID was compiled as: Squid Cache: Version 2.5.STABLE5 configure options: --prefix=/opt/EPAWebCachingSuite-1.0-i686/ --sysconfdir=/opt/EPAWebCachingSuite-1.0-i686/etc/squid --with-dl --enable-snmp --enable-carp --enable-useragent-log '--enable-auth=basic digest ntlm' '--enable-basic-auth-helpers=MSNT SMB getpwnam multi-domain-NTLM winbind' '--enable-ntlm-auth-helpers=SMB no_check winbind' --enable-digest-auth-helpers=password --enable-ntlm-fail-open --enable-referer-log --enable-htcp --enable-underscores --enable-stackt races --enable-delay-pools --enable-ssl --enable-cache-digests --with-samba-sources=/home/vmedina/SOURCES/samba-2.2.9/ --enable-x-accelerator-vary --disable-ident-lookups --enable-truncate --enable-removal-policies=heap --enable-xmalloc-statistics --enable-linux-netfilter --enable-stacktraces This does not answer your question, but why this excessive list of configure options? Rule of thumb: Only use a configure option if you know you need it. For most options there is reasons why they are not enabled by default. Regards Henrik
Re: [squid-users] optimization suggestions?
Hello! I'll have to recompile, but i am willing to =) First.. one thing, is COSS fs productionr eady? If it is not, what about stability? =) What about aufs?? I am currently using ufs. Victor. On Thu, 2004-09-30 at 09:59, Costas Zacharopoulos wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday 30 September 2004 16:08, Victor Medina wrote: If you want much better performance, change the cache_dir type to coss. If you want to do it, tell me to tell you some hits:) Hello all! I have a SQUID server running on a Piii 1100ghz server with 1024 RAM, SCSI disks. We have almost 100 users. And a 700Kbps ADSL Connection. The cache directory is in it's own scsi drive. I am seeing some very slow response in the proxy server. The incoming traffic in the internet connection is almost exactly the same as the outgoing traffic in the internal network, which basically means that the traffic from the internal proxy client is the same as the downloaded stuff. (am i right?) I have System Health graphics from both interfaces, I can send them to private mails if you want to take a look. DNS is being handled by a BIND server on the same machine. We are using ntlm auth via samba. We compiled our own SQUID server. CFLAGS=-O3 -march=i686 -mcpu=i686 SQUID was compiled as: Squid Cache: Version 2.5.STABLE5 configure options: --prefix=/opt/EPAWebCachingSuite-1.0-i686/ --sysconfdir=/opt/EPAWebCachingSuite-1.0-i686/etc/squid --with-dl --enable-snmp --enable-carp --enable-useragent-log '--enable-auth=basic digest ntlm' '--enable-basic-auth-helpers=MSNT SMB getpwnam multi-domain-NTLM winbind' '--enable-ntlm-auth-helpers=SMB no_check winbind' --enable-digest-auth-helpers=password --enable-ntlm-fail-open --enable-referer-log --enable-htcp --enable-underscores --enable-stackt races --enable-delay-pools --enable-ssl --enable-cache-digests --with-samba-sources=/home/vmedina/SOURCES/samba-2.2.9/ --enable-x-accelerator-vary --disable-ident-lookups --enable-truncate --enable-removal-policies=heap --enable-xmalloc-statistics --enable-linux-netfilter --enable-stacktraces Most of the sites, that users can access are specified by acl's so they are very predictable, most users have _only_ access to certain sites. Even though there are a awful lot of users for a 700Kbps link, the site are almost always the same. The cache dir is 7gb in size. SQUID has plenty of RAM. File system being used is RAISER, and it's a linux machine running kernel 2.6 under SuSE 9.1. Anybody would be so kind and suggest any optimization that could somehow improve performance? I will copy the relevant information about my squid.conf. icp_port 0 hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? cache_mem 192 MB cache_replacement_policy heap GDSF memory_replacement_policy heap GDSF cache_dir ufs /workload/cache 7168 16 256 cache_access_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/access.log cache_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/cache.log cache_store_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/store.log error_directory /opt/EPAWebCachingSuite-1.0-i686//share/errors/Spanish # DEFINICION DE LA AUTENTICACION CON NTLM auth_param ntlm program /opt/EPAWebCachingSuite-1.0-i686/libexec/wb_ntlmauth auth_param ntlm children 25 auth_param ntlm max_challenge_reuses 0 auth_param ntlm max_challenge_lifetime 15 minutes auth_param basic program /opt/EPAWebCachingSuite-1.0-i686/libexec/wb_auth auth_param basic children 25 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 6 hours # NO TOCAR! cache_mgr root cache_effective_user proxy cache_effective_group proxy logfile_rotate 0 log_icp_queries off cachemgr_passwd P6RsP6RsP6Rs all buffered_logs on delay_initial_bucket_level 50 delay_pools 0 Any suggestions or comments are welcome Thanxs! Victor. -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.3 (GNU/Linux) iD8DBQFBXBFim87SXUGUjPsRAm5CAKCbJAjSPylneqSiXwePJE8xdgNC5QCeJ3RE cw3S4AXReiHQzGCiJsSG7YM= =jafO -END PGP SIGNATURE-
Re: [squid-users] optimization suggestions?
Hello! Further investigations on the performance issues.. i found this. I took two access.logs of two different weeks. and counted how many tcp-hits and how many tcp-misses I had. In group one, one week worth of logs I found this: 10855 TCP_HITS that's almost 8% 108127 TCP_MISS that's almost 88% In group two, one week worth of logs, i found this: 8024 TCP_HITS, that's almost 10% 69695 TCP_MISS, that's almost 88% Or.. i have a very difficult traffic to cache, or i am doing something incredibly wrong and stupid (my first choice! =) or.. something is going wrong with my squid or my link. =) as always comments are welcome. Best Regards my friends! Victor.
Re: [squid-users] optimization suggestions?
On Thu, 30 Sep 2004, Victor Medina wrote: I'll have to recompile, but i am willing to =) First.. one thing, is COSS fs productionr eady? If it is not, what about stability? =) What about aufs?? I am currently using ufs. If you have more than 30 req/s then looking into aufs/diskd is recommended. But you will notice this from requests taking a long time and your Internet not fully utilized.. Regards Henrik
Re: [squid-users] optimization suggestions?
I am reading about coss file system right now! =) OK, i can try both options, i can recompile with coss and aufs/diskd. My squid haven't had a crash in 4 months or so... So rebuilding the cache in case of a crash is not an issue. One thing is, how do i know how many request/seg i am serving at a given moment? Best Regards. Vic. PS: who is writing the coss file system? Squid developer or some external individuals? On Thu, 2004-09-30 at 11:35, Costas Zacharopoulos wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday 30 September 2004 17:55, Victor Medina wrote: I tested both aufs coss. There is a big distance in performance! You can recompile squid while running, but you have to use exactly the same options as the scrutch install. Have a look at config.status and if it's there with the first time configuration options edit it, add the coss schema and ./config.status - -recheck:) But you have to know how to configure coss before enable it! Hello! I'll have to recompile, but i am willing to =) First.. one thing, is COSS fs productionr eady? If it is not, what about stability? =) What about aufs?? I am currently using ufs. Victor. On Thu, 2004-09-30 at 09:59, Costas Zacharopoulos wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thursday 30 September 2004 16:08, Victor Medina wrote: If you want much better performance, change the cache_dir type to coss. If you want to do it, tell me to tell you some hits:) Hello all! I have a SQUID server running on a Piii 1100ghz server with 1024 RAM, SCSI disks. We have almost 100 users. And a 700Kbps ADSL Connection. The cache directory is in it's own scsi drive. I am seeing some very slow response in the proxy server. The incoming traffic in the internet connection is almost exactly the same as the outgoing traffic in the internal network, which basically means that the traffic from the internal proxy client is the same as the downloaded stuff. (am i right?) I have System Health graphics from both interfaces, I can send them to private mails if you want to take a look. DNS is being handled by a BIND server on the same machine. We are using ntlm auth via samba. We compiled our own SQUID server. CFLAGS=-O3 -march=i686 -mcpu=i686 SQUID was compiled as: Squid Cache: Version 2.5.STABLE5 configure options: --prefix=/opt/EPAWebCachingSuite-1.0-i686/ --sysconfdir=/opt/EPAWebCachingSuite-1.0-i686/etc/squid --with-dl --enable-snmp --enable-carp --enable-useragent-log '--enable-auth=basic digest ntlm' '--enable-basic-auth-helpers=MSNT SMB getpwnam multi-domain-NTLM winbind' '--enable-ntlm-auth-helpers=SMB no_check winbind' --enable-digest-auth-helpers=password --enable-ntlm-fail-open --enable-referer-log --enable-htcp --enable-underscores --enable-stackt races --enable-delay-pools --enable-ssl --enable-cache-digests --with-samba-sources=/home/vmedina/SOURCES/samba-2.2.9/ --enable-x-accelerator-vary --disable-ident-lookups --enable-truncate --enable-removal-policies=heap --enable-xmalloc-statistics --enable-linux-netfilter --enable-stacktraces Most of the sites, that users can access are specified by acl's so they are very predictable, most users have _only_ access to certain sites. Even though there are a awful lot of users for a 700Kbps link, the site are almost always the same. The cache dir is 7gb in size. SQUID has plenty of RAM. File system being used is RAISER, and it's a linux machine running kernel 2.6 under SuSE 9.1. Anybody would be so kind and suggest any optimization that could somehow improve performance? I will copy the relevant information about my squid.conf. icp_port 0 hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex cgi-bin \? cache_mem 192 MB cache_replacement_policy heap GDSF memory_replacement_policy heap GDSF cache_dir ufs /workload/cache 7168 16 256 cache_access_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/access.log cache_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/cache.log cache_store_log /opt/EPAWebCachingSuite-1.0-i686/var/logs/squid/store.log error_directory /opt/EPAWebCachingSuite-1.0-i686//share/errors/Spanish # DEFINICION DE LA AUTENTICACION CON NTLM auth_param ntlm program /opt/EPAWebCachingSuite-1.0-i686/libexec/wb_ntlmauth auth_param ntlm children 25 auth_param ntlm max_challenge_reuses 0 auth_param ntlm max_challenge_lifetime 15 minutes auth_param basic program /opt/EPAWebCachingSuite-1.0-i686/libexec/wb_auth auth_param basic children 25 auth_param basic realm Squid proxy-caching web server auth_param basic credentialsttl 6 hours # NO TOCAR! cache_mgr root cache_effective_user proxy cache_effective_group proxy
Re: [squid-users] optimization suggestions?
On Thu, 30 Sep 2004, Victor Medina wrote: I am reading about coss file system right now! =) coss is highly experimental and not exacly production quality.. regards Henrik
[squid-users] optimization
I'm looking for sugested web sites that talk specifically about optimization of squid with hardware/OS combinations. We have a server up and running but are building a new one. We have read the man pages and have found a lot of helpful info on the squid site and FAQs. However there has to be someplace out there that discusses the effect of different hardware, OS and settings combinations in specific terms. The only results we have found have been from the 1990's. Also at this point we're not looking for sites that compare squid to other cacheing options only squid on this hardware/os/etc to squid on some other combination. Thanks for any pointers. Neil
