Re: [PATCH] HTTP Compliance: improve age calculation

2010-09-28 Thread Amos Jeffries

On 28/09/10 18:04, Alex Rousskov wrote:

On 09/27/2010 11:01 PM, Alex Rousskov wrote:

HTTP Compliance: improve entity age calculation.

Account for response delay in entity age calculation as described in RFC
2616 section 13.2.3.

Co-Advisor test cases:
test_case/rfc2616/ageCalc-none-7-none
test_case/rfc2616/ageCalc-5400-4-5


Now with the patch attached.

Alex.
P.S. I wish Thunderbird would check the subject for the
attachment-present keywords and not just the body!


+1.

Amos
--
Please be using
  Current Stable Squid 2.7.STABLE9 or 3.1.8
  Beta testers wanted for 3.2.0.2


Build failed in Hudson: 3.HEAD-i386-OpenBSD #588

2010-09-28 Thread noc
See http://build.squid-cache.org/job/3.HEAD-i386-OpenBSD/588/changes

Changes:

[Alex Rousskov rouss...@measurement-factory.com] HTTP Compliance: improve age 
calculation.

Account for response delay in age calculation as described in RFC 2616 section
13.2.3.

Co-Advisor test cases:
test_case/rfc2616/ageCalc-none-7-none
test_case/rfc2616/ageCalc-5400-4-5

[Alex Rousskov rouss...@measurement-factory.com] HTTP Compliance: Reply with 
an error if required validation fails.

RFC 2616 says that proxy MUST not use stale entries that have s-maxage,
proxy-revalidate, or must-revalidate cache-directive.

Add new fail_on_validation_err request flag to store result from
refreshCheck().  It is needed to avoid refreshLimits() recalculation in
clientReplyContext::handleIMSReply().

Split LOG_TCP_REFRESH_FAIL into LOG_TCP_REFRESH_FAIL_OLD (stale reply sent)
and LOG_TCP_REFRESH_FAIL_ERR (error forwarded). However, both are still logged
as TCP_REFRESH_FAIL for backward-compatibility with external scripts and such.
We may decide to start logging more detailed codes later.

Co-Advisor test cases:
test_case/rfc2616/noSrv-hit-must-reval-s-maxage-resp
test_case/rfc2616/noSrv-hit-must-reval-proxy-revalidate-resp
test_case/rfc2616/noSrv-hit-must-reval-must-revalidate-resp

[Alex Rousskov rouss...@measurement-factory.com] Cleanup: Remove old_rep2 
from clientReplyContext::handleIMSReply().

StoreEntry::getReply() returns pointer to HttpReply that belongs to MemObject.
It does not create a new object. Hence, outer old_rep, inner old old_rep, and
inner new old_rep2 in handleIMSReply() are equal.

No changes in Squid behavior are expected.

--
[...truncated 3109 lines...]
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: strcat() is almost always misused, please use strlcat()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: sprintf() is often misused, please use snprintf()
Making all in POP3
sed -e 's,[...@]perl[@],/usr/bin/perl,g' 
../../../../helpers/basic_auth/POP3/basic_pop3_auth.pl.in basic_pop3_auth || 
(/bin/rm -f -f basic_pop3_auth ; exit 1)
Making all in RADIUS
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include
-I../../../../helpers/basic_auth/RADIUS-Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -MT 
basic_radius_auth.o -MD -MP -MF .deps/basic_radius_auth.Tpo -c -o 
basic_radius_auth.o ../../../../helpers/basic_auth/RADIUS/basic_radius_auth.cc; 
 then mv -f .deps/basic_radius_auth.Tpo .deps/basic_radius_auth.Po; else rm 
-f .deps/basic_radius_auth.Tpo; exit 1; fi
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include
-I../../../../helpers/basic_auth/RADIUS-Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -MT radius-util.o 
-MD -MP -MF .deps/radius-util.Tpo -c -o radius-util.o 
../../../../helpers/basic_auth/RADIUS/radius-util.cc;  then mv -f 
.deps/radius-util.Tpo .deps/radius-util.Po; else rm -f 
.deps/radius-util.Tpo; exit 1; fi
/bin/sh ../../../libtool --tag=CXX --mode=link ccache g++ -Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT  -g -O2  -g -o 
basic_radius_auth  basic_radius_auth.o  radius-util.o -L../../../lib -lmiscutil 
 ../../../compat/libcompat.la-lm 
libtool: link: ccache g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments 
-Werror -pipe -D_REENTRANT -g -O2 -g -o basic_radius_auth basic_radius_auth.o 
radius-util.o  
-Lhttp://build.squid-cache.org/job/3.HEAD-i386-OpenBSD/ws/btlayer-00-default/squid-3.HEAD-BZR/_build/lib
 -lmiscutil ../../../compat/.libs/libcompat.a -lm
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: vsprintf() is often misused, please use vsnprintf()
basic_radius_auth.o(.text+0x128): In function `main':
../../../../helpers/basic_auth/RADIUS/basic_radius_auth.cc:471: warning: 
strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: strcat() is almost always misused, please use strlcat()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: sprintf() is often misused, please use snprintf()
Making all in fake
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include   -Wall 
-Wpointer-arith -Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 
-MT fake.o -MD -MP -MF .deps/fake.Tpo -c -o fake.o 
../../../../helpers/basic_auth/fake/fake.cc;  then mv -f .deps/fake.Tpo 
.deps/fake.Po; else rm -f .deps/fake.Tpo; exit 1; fi
/bin/sh ../../../libtool --tag=CXX --mode=link ccache g++ -Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT  -g -O2  -g -o 

[PATCH] HTTP Compliance: do not cache replies to no-store requests

2010-09-28 Thread Alex Rousskov

HTTP Compliance: do not cache replies to requests with CC/no-store.

Per RFC 2616, do not store any part of response to requests with 
no-store Cache-Control directive.


We may still serve no-store requests from the cache because RFC 2616 
does not seem to prohibit that. This may change if HTTPbis or developers 
decide to prohibit no-store hits.


Co-Advisor test case:
test_case/rfc2616/ccReqDirMsg-no-store-basic
HTTP Compliance: do not cache replies to requests with CC/no-store.

Per RFC 2616, do not store any part of response to requests with no-store
Cache-Control directive.

We may still serve no-store requests from the cache because RFC 2616 does not
seem to prohibit that. This may change if HTTPbis or developers decide to
prohibit no-store hits.

Co-Advisor test case:
test_case/rfc2616/ccReqDirMsg-no-store-basic

=== modified file 'src/http.cc'
--- src/http.cc	2010-09-14 07:37:38 +
+++ src/http.cc	2010-09-22 15:56:41 +
@@ -354,40 +354,46 @@ HttpStateData::cacheableReply()
 const char *v;
 #if USE_HTTP_VIOLATIONS
 
 const refresh_t *R = NULL;
 
 /* This strange looking define first looks up the refresh pattern
  * and then checks if the specified flag is set. The main purpose
  * of this is to simplify the refresh pattern lookup and USE_HTTP_VIOLATIONS
  * condition
  */
 #define REFRESH_OVERRIDE(flag) \
 ((R = (R ? R : refreshLimits(entry-mem_obj-url))) , \
 (R  R-flags.flag))
 #else
 #define REFRESH_OVERRIDE(flag) 0
 #endif
 
 if (surrogateNoStore)
 return 0;
 
+// RFC 2616: do not cache replies to responses with no-store CC directive
+if (request  request-cache_control 
+EBIT_TEST(request-cache_control-mask, CC_NO_STORE) 
+!REFRESH_OVERRIDE(ignore_no_store))
+return 0;
+
 if (!ignoreCacheControl) {
 if (EBIT_TEST(cc_mask, CC_PRIVATE)) {
 if (!REFRESH_OVERRIDE(ignore_private))
 return 0;
 }
 
 if (EBIT_TEST(cc_mask, CC_NO_CACHE)) {
 if (!REFRESH_OVERRIDE(ignore_no_cache))
 return 0;
 }
 
 if (EBIT_TEST(cc_mask, CC_NO_STORE)) {
 if (!REFRESH_OVERRIDE(ignore_no_store))
 return 0;
 }
 }
 
 if (request-flags.auth || request-flags.auth_sent) {
 /*
  * Responses to requests with authorization may be cached



Build failed in Hudson: 3.HEAD-i386-OpenBSD #589

2010-09-28 Thread noc
See http://build.squid-cache.org/job/3.HEAD-i386-OpenBSD/589/changes

Changes:

[Automatic source maintenance squid...@squid-cache.org] SourceFormat 
Enforcement

[Alex Rousskov rouss...@measurement-factory.com] HTTP Compliance: improve age 
calculation.

Account for response delay in age calculation as described in RFC 2616 section
13.2.3.

Co-Advisor test cases:
test_case/rfc2616/ageCalc-none-7-none
test_case/rfc2616/ageCalc-5400-4-5

--
[...truncated 3103 lines...]
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: strcat() is almost always misused, please use strlcat()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: sprintf() is often misused, please use snprintf()
Making all in POP3
sed -e 's,[...@]perl[@],/usr/bin/perl,g' 
../../../../helpers/basic_auth/POP3/basic_pop3_auth.pl.in basic_pop3_auth || 
(/bin/rm -f -f basic_pop3_auth ; exit 1)
Making all in RADIUS
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include
-I../../../../helpers/basic_auth/RADIUS-Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -MT 
basic_radius_auth.o -MD -MP -MF .deps/basic_radius_auth.Tpo -c -o 
basic_radius_auth.o ../../../../helpers/basic_auth/RADIUS/basic_radius_auth.cc; 
 then mv -f .deps/basic_radius_auth.Tpo .deps/basic_radius_auth.Po; else rm 
-f .deps/basic_radius_auth.Tpo; exit 1; fi
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include
-I../../../../helpers/basic_auth/RADIUS-Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 -MT radius-util.o 
-MD -MP -MF .deps/radius-util.Tpo -c -o radius-util.o 
../../../../helpers/basic_auth/RADIUS/radius-util.cc;  then mv -f 
.deps/radius-util.Tpo .deps/radius-util.Po; else rm -f 
.deps/radius-util.Tpo; exit 1; fi
/bin/sh ../../../libtool --tag=CXX --mode=link ccache g++ -Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT  -g -O2  -g -o 
basic_radius_auth  basic_radius_auth.o  radius-util.o -L../../../lib -lmiscutil 
 ../../../compat/libcompat.la-lm 
libtool: link: ccache g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments 
-Werror -pipe -D_REENTRANT -g -O2 -g -o basic_radius_auth basic_radius_auth.o 
radius-util.o  
-Lhttp://build.squid-cache.org/job/3.HEAD-i386-OpenBSD/ws/btlayer-00-default/squid-3.HEAD-BZR/_build/lib
 -lmiscutil ../../../compat/.libs/libcompat.a -lm
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: vsprintf() is often misused, please use vsnprintf()
basic_radius_auth.o(.text+0x128): In function `main':
../../../../helpers/basic_auth/RADIUS/basic_radius_auth.cc:471: warning: 
strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: strcat() is almost always misused, please use strlcat()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: sprintf() is often misused, please use snprintf()
Making all in fake
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include   -Wall 
-Wpointer-arith -Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 
-MT fake.o -MD -MP -MF .deps/fake.Tpo -c -o fake.o 
../../../../helpers/basic_auth/fake/fake.cc;  then mv -f .deps/fake.Tpo 
.deps/fake.Po; else rm -f .deps/fake.Tpo; exit 1; fi
/bin/sh ../../../libtool --tag=CXX --mode=link ccache g++ -Wall -Wpointer-arith 
-Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT  -g -O2  -g -o 
basic_fake_auth  fake.o -L../../../lib -lmiscutil  ../../../compat/libcompat.la 
libtool: link: ccache g++ -Wall -Wpointer-arith -Wwrite-strings -Wcomments 
-Werror -pipe -D_REENTRANT -g -O2 -g -o basic_fake_auth fake.o  
-Lhttp://build.squid-cache.org/job/3.HEAD-i386-OpenBSD/ws/btlayer-00-default/squid-3.HEAD-BZR/_build/lib
 -lmiscutil ../../../compat/.libs/libcompat.a
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: vsprintf() is often misused, please use vsnprintf()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: strcpy() is almost always misused, please use strlcpy()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: strcat() is almost always misused, please use strlcat()
/usr/local/lib/gcc/i386-unknown-openbsd4.7/4.2.4/../../../libestdc++.so.11.0: 
warning: sprintf() is often misused, please use snprintf()
Making all in getpwnam
if ccache g++ -DHAVE_CONFIG_H  -I../../../.. -I../../../../include 
-I../../../../src  -I../../../include -I/usr/local/include   -Wall 
-Wpointer-arith -Wwrite-strings -Wcomments -Werror -pipe -D_REENTRANT -g -O2 
-MT basic_getpwnam_auth.o -MD -MP