Making PEER_TCP_MAGIC_COUNT a cache_peer option

2008-06-06 Thread Mark Nottingham

http://www.squid-cache.org/bugs/show_bug.cgi?id=2377

Squid will consider a peer dead when it has ten connect failures; this  
is hardcoded in as PEER_TCP_MAGIC_COUNT.
The attached patch makes it a per-peer configuration option,  
defaulting to PEER_TCP_MAGIC_COUNT.



--
Mark Nottingham   [EMAIL PROTECTED]




Re: Round-Robin becomes unbalanced when a peer dies and comes back

2008-06-06 Thread Adrian Chadd
On Fri, Jun 06, 2008, Mark Nottingham wrote:
 Patch attached to bug.

.. why haven't you just been given CVS commit access already? :)



Adrian



Re: Round-Robin becomes unbalanced when a peer dies and comes back

2008-06-06 Thread Adrian Chadd
On Fri, Jun 06, 2008, Mark Nottingham wrote:
 Patch attached to bug.

.. looks fine to me.

Squid admins: Can we just please hook Mark up with CVS repository access
so he can commit these sorts of patches after review? :) I don't like
seeing stuff fall between the cracks because we're too busy.



Adrian

-- 
- Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support -
- $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -


Re: Making PEER_TCP_MAGIC_COUNT a cache_peer option

2008-06-06 Thread Amos Jeffries

Adrian Chadd wrote:

On Fri, Jun 06, 2008, Mark Nottingham wrote:

http://www.squid-cache.org/bugs/show_bug.cgi?id=2377

Squid will consider a peer dead when it has ten connect failures; this  
is hardcoded in as PEER_TCP_MAGIC_COUNT.
The attached patch makes it a per-peer configuration option,  
defaulting to PEER_TCP_MAGIC_COUNT.


Looks fine - but you need to add some code to the configuration file
-generator- so it spits out the option when building the cache_peer line.



What do you mean Adran? I thought it would just be a default setting 
added at the top of the cache_peer parsing function. Overridden if the 
option is used.


Amos
--
Please use Squid 2.7.STABLE1 or 3.0.STABLE6


Re: Round-Robin becomes unbalanced when a peer dies and comes back

2008-06-06 Thread Kinkie
I'm not an admin, but I agree.

  K



On 6/6/08, Adrian Chadd [EMAIL PROTECTED] wrote:
 On Fri, Jun 06, 2008, Mark Nottingham wrote:
 Patch attached to bug.

 .. looks fine to me.

 Squid admins: Can we just please hook Mark up with CVS repository access
 so he can commit these sorts of patches after review? :) I don't like
 seeing stuff fall between the cracks because we're too busy.



 Adrian

 --
 - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid
 Support -
 - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -



-- 
/kinkie


Re: header-to-header latency

2008-06-06 Thread Henrik Nordstrom
On fre, 2008-06-06 at 15:38 +1000, Mark Nottingham wrote:

 Logging the client-side request done point is an option, but it's of  
 limited value, because most requests don't have a body, and as  
 mentioned, the server can start sending the response before the  
 request is done. Not against it, just don't see *much* value.

What is of value is the delay between client request done and response
headers start, with a filter to ignore negative entries where the server
responds too early (or alternatively switch to measure request header
done to response header starts in such cases)

Regards
Henrik



Re: X-Vary-Options patch

2008-06-06 Thread Mark Nottingham

Squid devs: Did this make its way into 2.7?

Tim: AIUI, the following header:
  X-Vary-Options: Accept-Encoding; list-contains=gzip
will bucket the cache for this URI into two entries; those whose  
Accept-Encoding contains the list value gzip, and those that don't.  
Is that correct?


Also, I'm assuming the origin would still need to send
  Vary: Accept-Encoding
to work properly with downstream caches.

Cheers,


On 26/02/2008, at 3:30 PM, Adrian Chadd wrote:


G'day,

I'm happy to commit this to Squid-2.HEAD as-is. Can you throw it in
a Bugzilla report and spit me the number?

Thanks,



Adrian

On Fri, Feb 08, 2008, Tim Starling wrote:

There are two major sources of suboptimal hit rate on Wikipedia which
relate to the Vary header:

* In Accept-Encoding, we only care whether gzip is present or  
not, but

IE and Firefox use different whitespace conventions and so each get
separate entries in the cache
* We only care whether the user is logged in or not. Other cookies,  
such

as pure-JavaScript cookies used by client-side code to store
preferences, unnecessarily degrade our hit rate.

There have been other patches related to this problem, but as far  
as I'm
aware, they're all special-case, site-specific hacks. My patch adds  
an

X-Vary-Options response header (hereafter XVO), and thus gives the
origin server fine control over cache variance. In the patch, the XVO
header overrides the Vary header, so the Vary header can still be  
sent
as usual for compatibility with caches that don't support this  
feature.


The format of the XVO header is inspired by the format of the Accept
header. As in Vary, XVO is separated by commas into parts which  
relate
to different request headers. Then those parts are further  
separated by

semicolons. The first semicolon-separated part is the request header
name, and subsequent parts give name/value pairs separated by equals
signs, defining options relating to the variance of that header.

Two option names are currently defined:

list-contains: splits the request header into comma-separated parts
and varies depending on whether the resulting list contains the  
option value

string-contains: performs a simple search of the request header and
varies depending on whether it matches.

Multiple such options per header are allowed.

So for example:

X-Vary-Options: Cookie; string-contains=UserID;
string-contains=_session, Accept-Encoding; list-contains=gzip

This would vary the cache on three tests:
* whether the Cookie header contains the string UserID
* whether the Cookie header contains the string _session
* whether the Accept-Encoding header, interpreted as a comma- 
separated

list, contains the item gzip

The patch refactors all references to the Vary and X-Accelerator-Vary
headers into the functions httpHeaderHasVary() and  
httpHeaderGetVary()

in HttpHeader.c. It then adds X-Vary-Options to these functions,
interpreting it as a string rather than a list to avoid inappropriate
splitting on whitespace. It puts the resulting combined header into
X-Vary-Options instead of Vary in the base vary marker object,  
again to
avoid inappropriate list-style interpretation. httpMakeVaryMark()  
then

interprets this combined header in the way described above.

The added features of the patch are conditional, and are enabled by  
the
configure option --enable-vary-options. Autoconf and automake will  
need

to be run after applying this patch.

The interpretation of some non-standards-compliant Vary headers  
(those
containing semicolons) is changed slightly by this patch regardless  
of

--enable-vary-options.

The patch is attached and also available at:
http://noc.wikimedia.org/~tstarling/patches/vary_options_upstream.patch

For your review and consideration.

-- Tim Starling


diff -Xdiffx -ru squid-2.6.18.orig/configure.in squid-2.6.18/ 
configure.in
--- squid-2.6.18.orig/configure.in2008-01-10 23:34:23.0  
+1100

+++ squid-2.6.18/configure.in 2008-02-07 19:43:23.0 +1100
@@ -1507,6 +1507,16 @@
  fi
])

+dnl Enable vary options
+AC_ARG_ENABLE(vary_options,
+[  --enable-vary-options
+ Enable support for the X-Vary-Options  
header.],

+[ if test $enableval = yes ; then
+echo Enabling support for vary options
+AC_DEFINE(VARY_OPTIONS, 1, [Enable support for the X-Vary- 
Options header])

+  fi
+])
+
AC_ARG_ENABLE(follow-x-forwarded-for,
[  --enable-follow-x-forwarded-for
 Enable support for following the X- 
Forwarded-For
diff -Xdiffx -ru squid-2.6.18.orig/src/client_side.c squid-2.6.18/ 
src/client_side.c
--- squid-2.6.18.orig/src/client_side.c   2008-02-07  
19:28:38.0 +1100
+++ squid-2.6.18/src/client_side.c2008-02-08 14:39:38.0  
+1100

@@ -735,10 +735,7 @@
 request_t *request = http-request;
 const char *etag = httpHeaderGetStr(mem-reply-header,  
HDR_ETAG);

 const char *vary = request-vary_headers;
- int has_vary = 

Re: X-Vary-Options patch

2008-06-06 Thread Adrian Chadd
I think some of their stuff was backed out of Squid-2.7 before
release.

What we -should- do is create a wiki page to document all the crazy
stuff about Vary: and coordinate things a little better. I'd really
like to see more sensible vary handling go into Squid and there
certainly seems like there is enough interest in it now to get
work done.





Adrian

On Sat, Jun 07, 2008, Mark Nottingham wrote:
 Squid devs: Did this make its way into 2.7?
 
 Tim: AIUI, the following header:
   X-Vary-Options: Accept-Encoding; list-contains=gzip
 will bucket the cache for this URI into two entries; those whose  
 Accept-Encoding contains the list value gzip, and those that don't.  
 Is that correct?
 
 Also, I'm assuming the origin would still need to send
   Vary: Accept-Encoding
 to work properly with downstream caches.
 
 Cheers,
 
 
 On 26/02/2008, at 3:30 PM, Adrian Chadd wrote:
 
 G'day,
 
 I'm happy to commit this to Squid-2.HEAD as-is. Can you throw it in
 a Bugzilla report and spit me the number?
 
 Thanks,
 
 
 
 Adrian
 
 On Fri, Feb 08, 2008, Tim Starling wrote:
 There are two major sources of suboptimal hit rate on Wikipedia which
 relate to the Vary header:
 
 * In Accept-Encoding, we only care whether gzip is present or  
 not, but
 IE and Firefox use different whitespace conventions and so each get
 separate entries in the cache
 * We only care whether the user is logged in or not. Other cookies,  
 such
 as pure-JavaScript cookies used by client-side code to store
 preferences, unnecessarily degrade our hit rate.
 
 There have been other patches related to this problem, but as far  
 as I'm
 aware, they're all special-case, site-specific hacks. My patch adds  
 an
 X-Vary-Options response header (hereafter XVO), and thus gives the
 origin server fine control over cache variance. In the patch, the XVO
 header overrides the Vary header, so the Vary header can still be  
 sent
 as usual for compatibility with caches that don't support this  
 feature.
 
 The format of the XVO header is inspired by the format of the Accept
 header. As in Vary, XVO is separated by commas into parts which  
 relate
 to different request headers. Then those parts are further  
 separated by
 semicolons. The first semicolon-separated part is the request header
 name, and subsequent parts give name/value pairs separated by equals
 signs, defining options relating to the variance of that header.
 
 Two option names are currently defined:
 
 list-contains: splits the request header into comma-separated parts
 and varies depending on whether the resulting list contains the  
 option value
 string-contains: performs a simple search of the request header and
 varies depending on whether it matches.
 
 Multiple such options per header are allowed.
 
 So for example:
 
 X-Vary-Options: Cookie; string-contains=UserID;
 string-contains=_session, Accept-Encoding; list-contains=gzip
 
 This would vary the cache on three tests:
 * whether the Cookie header contains the string UserID
 * whether the Cookie header contains the string _session
 * whether the Accept-Encoding header, interpreted as a comma- 
 separated
 list, contains the item gzip
 
 The patch refactors all references to the Vary and X-Accelerator-Vary
 headers into the functions httpHeaderHasVary() and  
 httpHeaderGetVary()
 in HttpHeader.c. It then adds X-Vary-Options to these functions,
 interpreting it as a string rather than a list to avoid inappropriate
 splitting on whitespace. It puts the resulting combined header into
 X-Vary-Options instead of Vary in the base vary marker object,  
 again to
 avoid inappropriate list-style interpretation. httpMakeVaryMark()  
 then
 interprets this combined header in the way described above.
 
 The added features of the patch are conditional, and are enabled by  
 the
 configure option --enable-vary-options. Autoconf and automake will  
 need
 to be run after applying this patch.
 
 The interpretation of some non-standards-compliant Vary headers  
 (those
 containing semicolons) is changed slightly by this patch regardless  
 of
 --enable-vary-options.
 
 The patch is attached and also available at:
 http://noc.wikimedia.org/~tstarling/patches/vary_options_upstream.patch
 
 For your review and consideration.
 
 -- Tim Starling
 
 diff -Xdiffx -ru squid-2.6.18.orig/configure.in squid-2.6.18/ 
 configure.in
 --- squid-2.6.18.orig/configure.in2008-01-10 23:34:23.0  
 +1100
 +++ squid-2.6.18/configure.in 2008-02-07 19:43:23.0 +1100
 @@ -1507,6 +1507,16 @@
   fi
 ])
 
 +dnl Enable vary options
 +AC_ARG_ENABLE(vary_options,
 +[  --enable-vary-options
 + Enable support for the X-Vary-Options  
 header.],
 +[ if test $enableval = yes ; then
 +echo Enabling support for vary options
 +AC_DEFINE(VARY_OPTIONS, 1, [Enable support for the X-Vary- 
 Options header])
 +  fi
 +])
 +
 AC_ARG_ENABLE(follow-x-forwarded-for,
 [  --enable-follow-x-forwarded-for
  Enable support 

Re: Rev 8939: Merge from trunk in file:///home/rousskov/programs/bazaar/repos/squid/ecap/

2008-06-06 Thread Amos Jeffries

Alex Rousskov wrote:

At file:///home/rousskov/programs/bazaar/repos/squid/ecap/



Ready for some merging?

Amos
--
Please use Squid 2.7.STABLE1 or 3.0.STABLE6