Re: [squid-users] squid 3.1.0.13 performance results ready - reverse proxy - (2.6.x vs 3.1.x) - need help
GaneshKumar Natarajan wrote: Thanks Amos. Is this with the gzip feature already enabled? NO. gzip is not enabled in 3.1 and also client doesnt send accept-encoding. the request are typically the same that was sent to 2.6 version. Is the web server agent sending chunked replies? NO both could be noticeably slower as the entire object needs to be re-formatted. we have 32 GB, but we use only 50% of it. how much we could increase cache_mem ? It depends highly o the traffic load and object sizes. Best-practice is to start small'ish and increase it gradually while watching that RAM does not get full or start to swap. swapping is very bad for performance. also i will try heap LRU as you suggested for memory_replacement_policy. let me know, if you need any other options to try. I will run the test again and post some results to you. I can't think of anything that would be useful for a general benchmark. Thanks. Amos Ganesh On Tue, Oct 20, 2009 at 6:44 PM, Amos Jeffries squ...@treenet.co.nz wrote: On Tue, 20 Oct 2009 13:43:05 -0400, GaneshKumar Natarajan itisg...@gmail.com wrote: We wanted to evaluate 3.1.0.13 squid to move from our current squid version of 2.6.x ( stable 4 + few custom changes ) We did the following performance test from a Avalance setup. 1. preload objects in squid cache. 2. 3500 transactions/sec with 90-10 hit-miss ratio. 3. mean size of object 23 kb. 4. ran it for 30 minutes. ( 5 min ramp up to load 3500, 20 min with load 3500, 5 min to cool down ) Average response time Results we got. 2.6.x version = 22 milli second 3.1.0.13 = 274 milli second. ( the graph increases over period of time... ) This is a bit strange. The other benchmarks I've seen (2.6STABLE5 vs 3.0STABLE2) show a small lag increase of around 10% for small objects and a large 10x decrease for MB sized objects. But not a 10x increase. This is one of the first benchmarks received for 3.1 so its hard to say where its coming from. Is this with the gzip feature already enabled? Is the web server agent sending chunked replies? both could be noticeably slower as the entire object needs to be re-formatted. 3.1 does not yet do collapsed forwarding (planned for merge 3.2 if anyone gets time), that might also be having an effect. --- similarly, we did for large objects with 40 transaction/sec, mean object size 1.8 MB. 2.6.x = 91 ms, squid 3.1.0.13 = 109 ms. this is somewhat ok.. --- We wanted to move to 3.1.0.13 to make use of gzip+ecap feature and other 3.1 features, but this performance results is disappointing. The OS and squid.conf parameters for small file objects are typically the same for both 2.6 and 3.1 setup. [ to mention a few: cache_mem = 16 GB ( we have 32 GB max ), max_object_size_in_memory = 1 MB refer config file below ] Questions: 1. Is there any paramater am missing for 3.1 squid, which would help to improve performance for high loads? cache_mem would have been the key one. 2. Or Is squid 3.1 really not ready yet for high load situations for small objects? Any performance related work going on, any dates/versions to expect ? Has not yet had serious testing for loads. I've only seen two quality independent benchmarks since 2.5. Adrian did a lot of benchmarking and tuning, then only plugged the results back into 2.7, leaving 3.x out in the cold. The 12-18 months of work for 3.2 is geared at pushing the bar up again trying to surpass 2.7. am giving the squid.conf entries 3.1 (its the same for 2.6 also ). let me know, if you need any other details. Regards, Ganesh OS -- linux RH4 -release 8 Linux 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux SQUID 3.1.0.13 Squid.conf entries for Small file objects (note: the following squid parameters were the same for 2.6 squid.) http_port 80 vhost vport=80 acl port80 port 80 icp_port 0 udp_incoming_address 0.0.0.0 udp_outgoing_address 255.255.255.255 icp_query_timeout 0 maximum_icp_query_timeout 2000 mcast_icp_query_timeout 2000 dead_peer_timeout 10 seconds hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex \? acl CGI urlpath_regex cgi-bin acl readCommunityString snmp_community icds-nms acl LMS src 192.168.2.4 snmp_access allow readCommunityString all acl apache rep_header Server ^Apache cache_swap_low 95 cache_swap_high 98 maximum_object_size 100 MB minimum_object_size 0 KB maximum_object_size_in_memory 1 MB The above may be limiting the 3.1 large object results. 3.1 no longer has the huge object speed limitations that 2.x does, so this can be increased provided the RAM can cope. ipcache_size 2048 ipcache_low 95 ipcache_high 98 cache_replacement_policy lru memory_replacement_policy lru heap types are better here regardless of the squid version. cache_log /squid/logs/cache.log cache_store_log none log_ip_on_direct on debug_options ALL,1 client_netmask 255.255.255.255 dns_timeout 10 seconds refresh_pattern ^ftp:
[squid-users] squid 3.1.0.13 performance results ready - reverse proxy - (2.6.x vs 3.1.x) - need help
We wanted to evaluate 3.1.0.13 squid to move from our current squid version of 2.6.x ( stable 4 + few custom changes ) We did the following performance test from a Avalance setup. 1. preload objects in squid cache. 2. 3500 transactions/sec with 90-10 hit-miss ratio. 3. mean size of object 23 kb. 4. ran it for 30 minutes. ( 5 min ramp up to load 3500, 20 min with load 3500, 5 min to cool down ) Average response time Results we got. 2.6.x version = 22 milli second 3.1.0.13 = 274 milli second. ( the graph increases over period of time... ) --- similarly, we did for large objects with 40 transaction/sec, mean object size 1.8 MB. 2.6.x = 91 ms, squid 3.1.0.13 = 109 ms. this is somewhat ok.. --- We wanted to move to 3.1.0.13 to make use of gzip+ecap feature and other 3.1 features, but this performance results is disappointing. The OS and squid.conf parameters for small file objects are typically the same for both 2.6 and 3.1 setup. [ to mention a few: cache_mem = 16 GB ( we have 32 GB max ), max_object_size_in_memory = 1 MB refer config file below ] Questions: 1. Is there any paramater am missing for 3.1 squid, which would help to improve performance for high loads? 2. Or Is squid 3.1 really not ready yet for high load situations for small objects? Any performance related work going on, any dates/versions to expect ? am giving the squid.conf entries 3.1 (its the same for 2.6 also ). let me know, if you need any other details. Regards, Ganesh OS -- linux RH4 -release 8 Linux 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux SQUID 3.1.0.13 Squid.conf entries for Small file objects (note: the following squid parameters were the same for 2.6 squid.) http_port 80 vhost vport=80 acl port80 port 80 icp_port 0 udp_incoming_address 0.0.0.0 udp_outgoing_address 255.255.255.255 icp_query_timeout 0 maximum_icp_query_timeout 2000 mcast_icp_query_timeout 2000 dead_peer_timeout 10 seconds hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex \? acl CGI urlpath_regex cgi-bin acl readCommunityString snmp_community icds-nms acl LMS src 192.168.2.4 snmp_access allow readCommunityString all acl apache rep_header Server ^Apache cache_swap_low 95 cache_swap_high 98 maximum_object_size 100 MB minimum_object_size 0 KB maximum_object_size_in_memory 1 MB ipcache_size 2048 ipcache_low 95 ipcache_high 98 cache_replacement_policy lru memory_replacement_policy lru cache_log /squid/logs/cache.log cache_store_log none log_ip_on_direct on debug_options ALL,1 client_netmask 255.255.255.255 dns_timeout 10 seconds refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern . 1440100%1440ignore-reload quick_abort_min -1 KB quick_abort_max 16 KB quick_abort_pct 95 negative_ttl 1 minutes positive_dns_ttl 1 hour negative_dns_ttl 1 minute range_offset_limit -1 MB connect_timeout 5 seconds peer_connect_timeout 5 seconds read_timeout 60 seconds request_timeout 10 seconds persistent_request_timeout 10 minutes pconn_timeout 120 seconds shutdown_lifetime 30 seconds acl manager proto cache_object acl localhost src 127.0.0.1/32 acl SSL_ports port 443 563 acl Safe_ports port 80 acl OBJECT method OBJECT acl CONNECT method CONNECT acl PURGE method PURGE acl Safe_methods method GET POST HEAD PUT acl Safe_protos proto HTTP http_access allow manager localhost1 http_access allow manager localhost http_access deny manager http_access allow Safe_methods http_access allow PURGE localhost1 http_access allow PURGE localhost http_access allow OBJECT localhost http_access allow OBJECT localhost1 http_access deny PURGE http_access deny OBJECT http_access deny !Safe_ports http_access deny !Safe_protos http_access deny CONNECT !SSL_ports http_access deny all http_reply_access allow all reply_header_max_size 20 KB cache_mgr webmaster cache_effective_user icds announce_host dummy.net announce_port 3131 forwarded_for on icp_hit_stale on cachemgr_passwd passw0rd info stats/objects client_db off maximum_single_addr_tries 1 snmp_port 161 offline_mode off uri_whitespace encode nonhierarchical_direct on prefer_direct off strip_query_terms off coredump_dir none redirector_bypass off client_persistent_connections on server_persistent_connections on cache_dir aufs /squid/cache0 158522 29 830 cache_dir aufs /squid/cache1 252949 29 830 cache_dir aufs /squid/cache2 252949 29 830 cache_dir aufs /squid/cache3 252949 29 830 cache_dir aufs /squid/cache4 252949 29 830 cache_dir aufs /squid/cache5 252949 29 830 request_body_max_size 100 KB request_header_max_size 8 KB minimum_expiry_time 0 seconds read_ahead_gap 400 KB cache_mem 16083 MB acl 1001 dstdomain www1.acm.com acl 1002 dstdomain www2.acm.com acl 1003 dstdomain www3.acm.com ... acl 1025 dstdomain www25.acm.com cache_peer xxx parent 8000 0 no-query originserver forceddomain=www.acm.com cache_peer_access 10.0.1.4 allow 1001
Re: [squid-users] squid 3.1.0.13 performance results ready - reverse proxy - (2.6.x vs 3.1.x) - need help
On Tue, 20 Oct 2009 13:43:05 -0400, GaneshKumar Natarajan itisg...@gmail.com wrote: We wanted to evaluate 3.1.0.13 squid to move from our current squid version of 2.6.x ( stable 4 + few custom changes ) We did the following performance test from a Avalance setup. 1. preload objects in squid cache. 2. 3500 transactions/sec with 90-10 hit-miss ratio. 3. mean size of object 23 kb. 4. ran it for 30 minutes. ( 5 min ramp up to load 3500, 20 min with load 3500, 5 min to cool down ) Average response time Results we got. 2.6.x version = 22 milli second 3.1.0.13 = 274 milli second. ( the graph increases over period of time... ) This is a bit strange. The other benchmarks I've seen (2.6STABLE5 vs 3.0STABLE2) show a small lag increase of around 10% for small objects and a large 10x decrease for MB sized objects. But not a 10x increase. This is one of the first benchmarks received for 3.1 so its hard to say where its coming from. Is this with the gzip feature already enabled? Is the web server agent sending chunked replies? both could be noticeably slower as the entire object needs to be re-formatted. 3.1 does not yet do collapsed forwarding (planned for merge 3.2 if anyone gets time), that might also be having an effect. --- similarly, we did for large objects with 40 transaction/sec, mean object size 1.8 MB. 2.6.x = 91 ms, squid 3.1.0.13 = 109 ms. this is somewhat ok.. --- We wanted to move to 3.1.0.13 to make use of gzip+ecap feature and other 3.1 features, but this performance results is disappointing. The OS and squid.conf parameters for small file objects are typically the same for both 2.6 and 3.1 setup. [ to mention a few: cache_mem = 16 GB ( we have 32 GB max ), max_object_size_in_memory = 1 MB refer config file below ] Questions: 1. Is there any paramater am missing for 3.1 squid, which would help to improve performance for high loads? cache_mem would have been the key one. 2. Or Is squid 3.1 really not ready yet for high load situations for small objects? Any performance related work going on, any dates/versions to expect ? Has not yet had serious testing for loads. I've only seen two quality independent benchmarks since 2.5. Adrian did a lot of benchmarking and tuning, then only plugged the results back into 2.7, leaving 3.x out in the cold. The 12-18 months of work for 3.2 is geared at pushing the bar up again trying to surpass 2.7. am giving the squid.conf entries 3.1 (its the same for 2.6 also ). let me know, if you need any other details. Regards, Ganesh OS -- linux RH4 -release 8 Linux 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux SQUID 3.1.0.13 Squid.conf entries for Small file objects (note: the following squid parameters were the same for 2.6 squid.) http_port 80 vhost vport=80 acl port80 port 80 icp_port 0 udp_incoming_address 0.0.0.0 udp_outgoing_address 255.255.255.255 icp_query_timeout 0 maximum_icp_query_timeout 2000 mcast_icp_query_timeout 2000 dead_peer_timeout 10 seconds hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex \? acl CGI urlpath_regex cgi-bin acl readCommunityString snmp_community icds-nms acl LMS src 192.168.2.4 snmp_access allow readCommunityString all acl apache rep_header Server ^Apache cache_swap_low 95 cache_swap_high 98 maximum_object_size 100 MB minimum_object_size 0 KB maximum_object_size_in_memory 1 MB The above may be limiting the 3.1 large object results. 3.1 no longer has the huge object speed limitations that 2.x does, so this can be increased provided the RAM can cope. ipcache_size 2048 ipcache_low 95 ipcache_high 98 cache_replacement_policy lru memory_replacement_policy lru heap types are better here regardless of the squid version. cache_log /squid/logs/cache.log cache_store_log none log_ip_on_direct on debug_options ALL,1 client_netmask 255.255.255.255 dns_timeout 10 seconds refresh_pattern ^ftp: 144020% 10080 refresh_pattern ^gopher:14400% 1440 refresh_pattern . 1440100%1440ignore-reload quick_abort_min -1 KB quick_abort_max 16 KB quick_abort_pct 95 negative_ttl 1 minutes positive_dns_ttl 1 hour negative_dns_ttl 1 minute range_offset_limit -1 MB connect_timeout 5 seconds peer_connect_timeout 5 seconds read_timeout 60 seconds request_timeout 10 seconds persistent_request_timeout 10 minutes pconn_timeout 120 seconds shutdown_lifetime 30 seconds acl manager proto cache_object acl localhost src 127.0.0.1/32 acl SSL_ports port 443 563 acl Safe_ports port 80 acl OBJECT method OBJECT acl CONNECT method CONNECT acl PURGE method PURGE acl Safe_methods method GET POST HEAD PUT acl Safe_protos proto HTTP http_access allow manager localhost1 http_access allow manager localhost http_access deny manager http_access allow Safe_methods http_access allow PURGE
Re: [squid-users] squid 3.1.0.13 performance results ready - reverse proxy - (2.6.x vs 3.1.x) - need help
Thanks Amos. Is this with the gzip feature already enabled? NO. gzip is not enabled in 3.1 and also client doesnt send accept-encoding. the request are typically the same that was sent to 2.6 version. Is the web server agent sending chunked replies? NO both could be noticeably slower as the entire object needs to be re-formatted. we have 32 GB, but we use only 50% of it. how much we could increase cache_mem ? also i will try heap LRU as you suggested for memory_replacement_policy. let me know, if you need any other options to try. I will run the test again and post some results to you. Ganesh On Tue, Oct 20, 2009 at 6:44 PM, Amos Jeffries squ...@treenet.co.nz wrote: On Tue, 20 Oct 2009 13:43:05 -0400, GaneshKumar Natarajan itisg...@gmail.com wrote: We wanted to evaluate 3.1.0.13 squid to move from our current squid version of 2.6.x ( stable 4 + few custom changes ) We did the following performance test from a Avalance setup. 1. preload objects in squid cache. 2. 3500 transactions/sec with 90-10 hit-miss ratio. 3. mean size of object 23 kb. 4. ran it for 30 minutes. ( 5 min ramp up to load 3500, 20 min with load 3500, 5 min to cool down ) Average response time Results we got. 2.6.x version = 22 milli second 3.1.0.13 = 274 milli second. ( the graph increases over period of time... ) This is a bit strange. The other benchmarks I've seen (2.6STABLE5 vs 3.0STABLE2) show a small lag increase of around 10% for small objects and a large 10x decrease for MB sized objects. But not a 10x increase. This is one of the first benchmarks received for 3.1 so its hard to say where its coming from. Is this with the gzip feature already enabled? Is the web server agent sending chunked replies? both could be noticeably slower as the entire object needs to be re-formatted. 3.1 does not yet do collapsed forwarding (planned for merge 3.2 if anyone gets time), that might also be having an effect. --- similarly, we did for large objects with 40 transaction/sec, mean object size 1.8 MB. 2.6.x = 91 ms, squid 3.1.0.13 = 109 ms. this is somewhat ok.. --- We wanted to move to 3.1.0.13 to make use of gzip+ecap feature and other 3.1 features, but this performance results is disappointing. The OS and squid.conf parameters for small file objects are typically the same for both 2.6 and 3.1 setup. [ to mention a few: cache_mem = 16 GB ( we have 32 GB max ), max_object_size_in_memory = 1 MB refer config file below ] Questions: 1. Is there any paramater am missing for 3.1 squid, which would help to improve performance for high loads? cache_mem would have been the key one. 2. Or Is squid 3.1 really not ready yet for high load situations for small objects? Any performance related work going on, any dates/versions to expect ? Has not yet had serious testing for loads. I've only seen two quality independent benchmarks since 2.5. Adrian did a lot of benchmarking and tuning, then only plugged the results back into 2.7, leaving 3.x out in the cold. The 12-18 months of work for 3.2 is geared at pushing the bar up again trying to surpass 2.7. am giving the squid.conf entries 3.1 (its the same for 2.6 also ). let me know, if you need any other details. Regards, Ganesh OS -- linux RH4 -release 8 Linux 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:33:05 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux SQUID 3.1.0.13 Squid.conf entries for Small file objects (note: the following squid parameters were the same for 2.6 squid.) http_port 80 vhost vport=80 acl port80 port 80 icp_port 0 udp_incoming_address 0.0.0.0 udp_outgoing_address 255.255.255.255 icp_query_timeout 0 maximum_icp_query_timeout 2000 mcast_icp_query_timeout 2000 dead_peer_timeout 10 seconds hierarchy_stoplist cgi-bin ? acl QUERY urlpath_regex \? acl CGI urlpath_regex cgi-bin acl readCommunityString snmp_community icds-nms acl LMS src 192.168.2.4 snmp_access allow readCommunityString all acl apache rep_header Server ^Apache cache_swap_low 95 cache_swap_high 98 maximum_object_size 100 MB minimum_object_size 0 KB maximum_object_size_in_memory 1 MB The above may be limiting the 3.1 large object results. 3.1 no longer has the huge object speed limitations that 2.x does, so this can be increased provided the RAM can cope. ipcache_size 2048 ipcache_low 95 ipcache_high 98 cache_replacement_policy lru memory_replacement_policy lru heap types are better here regardless of the squid version. cache_log /squid/logs/cache.log cache_store_log none log_ip_on_direct on debug_options ALL,1 client_netmask 255.255.255.255 dns_timeout 10 seconds refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern . 1440 100% 1440 ignore-reload quick_abort_min -1 KB quick_abort_max 16 KB quick_abort_pct 95 negative_ttl 1 minutes
