problems collapsing large responses

2010-02-15 Thread Mark Nottingham
I'm seeing some strange behaviour when using collapsed_forwarding on large 
responses in squid-2.7 and squid2-HEAD.

Two separate symptoms:
  1) large responses not being cached when collapsed
  2) large responses not being completely sent; i.e., part of the response is 
sent, then it 'locks up'

#2 is more worrisome.

To recreate:
  - compile a squid-2.7 or HEAD, configure with collapsed_forwarding on
  - serve content with a script like this:

---8---
#!/usr/bin/env python

import sys, time
time.sleep(2)
print Status: 200 OK
print Content-Type: text/plain
print Cache-Control: max-age=45
print Vary: Accept-Encoding
print
for i in range(1024):
 print abcdefghij * 12
---8---

and drive traffic like this:
  httperf --server localhost --port 3128 --hog --num-calls 1 --num-conns 10 
--rate 2 --uri `cat urls.txt`
or this:
  http_load -rate 2 -seconds 20 -proxy localhost:3128 urls.txt
assuring that the cache is empty first.

See access.log as well as load generation results.

Observations:
  - there seems to be a threshold response size of somewhere around 110K that 
triggers this
  - does not appear to rely on value of maximum_object_size_in_memory
  - does not appear to be specific to disk or null disk caching
  - problem #2 seems to be caused by a Vary header
  - does not appear to be related to VARY_RESTART; clientCacheHit: Vary MATCH!

Does this seem familiar to anyone? I'll file a bug, but thought I'd check and 
see if it was a known issue.

Cheers,


--
Mark Nottingham   m...@yahoo-inc.com




Re: problems collapsing large responses

2010-02-15 Thread Mark Nottingham
A bit more;

  - it *does* hit VARY_REFRESH first
  - threshold for the vary problem is much lower
  - evident since (at least) Squid 2.7STABLE1


On 15/02/2010, at 8:59 PM, Mark Nottingham wrote:

 I'm seeing some strange behaviour when using collapsed_forwarding on large 
 responses in squid-2.7 and squid2-HEAD.
 
 Two separate symptoms:
  1) large responses not being cached when collapsed
  2) large responses not being completely sent; i.e., part of the response is 
 sent, then it 'locks up'
 
 #2 is more worrisome.
 
 To recreate:
  - compile a squid-2.7 or HEAD, configure with collapsed_forwarding on
  - serve content with a script like this:
 
 ---8---
 #!/usr/bin/env python
 
 import sys, time
 time.sleep(2)
 print Status: 200 OK
 print Content-Type: text/plain
 print Cache-Control: max-age=45
 print Vary: Accept-Encoding
 print
 for i in range(1024):
 print abcdefghij * 12
 ---8---
 
 and drive traffic like this:
  httperf --server localhost --port 3128 --hog --num-calls 1 --num-conns 10 
 --rate 2 --uri `cat urls.txt`
 or this:
  http_load -rate 2 -seconds 20 -proxy localhost:3128 urls.txt
 assuring that the cache is empty first.
 
 See access.log as well as load generation results.
 
 Observations:
  - there seems to be a threshold response size of somewhere around 110K that 
 triggers this
  - does not appear to rely on value of maximum_object_size_in_memory
  - does not appear to be specific to disk or null disk caching
  - problem #2 seems to be caused by a Vary header
  - does not appear to be related to VARY_RESTART; clientCacheHit: Vary MATCH!
 
 Does this seem familiar to anyone? I'll file a bug, but thought I'd check and 
 see if it was a known issue.
 
 Cheers,
 
 
 --
 Mark Nottingham   m...@yahoo-inc.com
 
 

--
Mark Nottingham   m...@yahoo-inc.com