[squid-dev] Jenkins build is back to normal : trunk-matrix » clang,d-ubuntu-utopic #292
See http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/292/ ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
[squid-dev] Preliminary results of gperf+std::unordered_map
Hi, the preliminary polygraph results for using a std::unordered_map with a perfect hash (generated by gperf) to do RegisteredHeader - HttpHdr lookups are in, and are quite disappointing. The hash is as effective as it can be: two table lookups and simple calculations. Yet when using it squid loses about 10RPS (on about 2k) over std::map (-0.5% performance); I guess that for a small hash such as the one we're using, the effort to walk the hash structure I'll next try to go the full length and get rid of std::unordered_map and using gperf for all the work (it stores data in a single const array), and I'll validate the numbers, but if things won't change significantly, I'll just stick with std::map. -- Francesco ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
[squid-dev] Build failed in Jenkins: trunk-polygraph #802
See http://build.squid-cache.org/job/trunk-polygraph/802/ -- Started by an SCM change Building remotely on polygraph (12.04 amd64-Ubuntu Ubuntu amd64-Ubuntu-12.04 Ubuntu-12.04 amd64) in workspace http://build.squid-cache.org/job/trunk-polygraph/ws/ $ bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ info result: bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ returned 0. Command output: 14212 kin...@squid-cache.org-20150806185529-133uj7jtqm3yfr74 stderr: [trunk-polygraph] $ bzr update M src/HttpHdrCc.cc M src/HttpHdrCc.cci All changes applied successfully. Updated to revision 14212 of branch http://bzr.squid-cache.org/bzr/squid3/trunk [trunk-polygraph] $ bzr switch http://bzr.squid-cache.org/bzr/squid3/trunk/ Tree is up to date at revision 14212. Switched to branch: http://bzr.squid-cache.org/bzr/squid3/trunk/ [trunk-polygraph] $ bzr revert $ bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ info result: bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ returned 0. Command output: 14212 kin...@squid-cache.org-20150806185529-133uj7jtqm3yfr74 stderr: [trunk-polygraph] $ bzr log -v -r revid:kin...@squid-cache.org-20150806185529-133uj7jtqm3yfr74..revid:kin...@squid-cache.org-20150806185529-133uj7jtqm3yfr74 --long --show-ids Getting local revision... $ bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ info result: bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ returned 0. Command output: 14212 kin...@squid-cache.org-20150806185529-133uj7jtqm3yfr74 stderr: RevisionState revno:14212 revid:kin...@squid-cache.org-20150806185529-133uj7jtqm3yfr74 [trunk-polygraph] $ /bin/sh -xe /tmp/hudson2265326743884544292.sh + cd /home/jenkins/squidperf + python SquidBasicPerf.py --audited http://build.squid-cache.org/job/trunk-polygraph/355/artifact/logs/test.lx --jjid 802 --svnurl http://bzr.squid-cache.org/bzr/squid3/trunk/ --jobname trunk-polygraph Test is failed Build step 'Execute shell' marked build as failure Archiving artifacts [description-setter] Description set: ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
[squid-dev] [PATCH] squid SSL subsystem did not initialized correctly
Hi all,
Currently SSL subsystem did not initialized correctly in squid
trunk. This is because of the
Security::ProxyOutgoingConfig.encryptTransport which is always false so
the client SSL CTX object never builds. As a result squid may not start
if SSL is configured. I am attaching a small patch I am using in my
squid trees to work with SSL.
=== modified file 'src/cache_cf.cc'
--- src/cache_cf.cc 2015-08-04 21:04:09 +
+++ src/cache_cf.cc 2015-08-06 09:49:07 +
@@ -848,47 +848,46 @@
#endif
}
} else {
Config2.effectiveUserID = geteuid();
Config2.effectiveGroupID = getegid();
}
if (NULL != Config.effectiveGroup) {
struct group *grp = getgrnam(Config.effectiveGroup);
if (NULL == grp) {
fatalf(getgrnam failed to find groupid for effective group '%s',
Config.effectiveGroup);
return;
}
Config2.effectiveGroupID = grp-gr_gid;
}
-if (Security::ProxyOutgoingConfig.encryptTransport) {
-debugs(3, DBG_IMPORTANT, Initializing https:// proxy context);
-Config.ssl_client.sslContext = Security::ProxyOutgoingConfig.createClientContext(false);
-if (!Config.ssl_client.sslContext) {
-debugs(3, DBG_CRITICAL, ERROR: Could not initialize https:// proxy context);
-self_destruct();
-}
+debugs(3, DBG_IMPORTANT, Initializing https:// proxy context);
+Security::ProxyOutgoingConfig.encryptTransport = true;
+Config.ssl_client.sslContext = Security::ProxyOutgoingConfig.createClientContext(false);
+if (!Config.ssl_client.sslContext) {
+debugs(3, DBG_CRITICAL, ERROR: Could not initialize https:// proxy context);
+self_destruct();
}
for (CachePeer *p = Config.peers; p != NULL; p = p-next) {
// default value for ssldomain= is the peer host/IP
if (p-secure.sslDomain.isEmpty())
p-secure.sslDomain = p-host;
if (p-secure.encryptTransport) {
debugs(3, DBG_IMPORTANT, Initializing cache_peer p-name TLS context);
p-sslContext = p-secure.createClientContext(true);
if (!p-sslContext) {
debugs(3, DBG_CRITICAL, ERROR: Could not initialize cache_peer p-name TLS context);
self_destruct();
}
}
}
#if USE_OPENSSL
for (AnyP::PortCfgPointer s = HttpPortList; s != NULL; s = s-next) {
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev
[squid-dev] Build failed in Jenkins: trunk-polygraph #801
See http://build.squid-cache.org/job/trunk-polygraph/801/ -- Started by an SCM change Building remotely on polygraph (12.04 amd64-Ubuntu Ubuntu amd64-Ubuntu-12.04 Ubuntu-12.04 amd64) in workspace http://build.squid-cache.org/job/trunk-polygraph/ws/ Cleaning workspace... $ bzr checkout --lightweight http://bzr.squid-cache.org/bzr/squid3/trunk/ http://build.squid-cache.org/job/trunk-polygraph/ws/ Getting local revision... $ bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ info result: bzr revision-info -d http://build.squid-cache.org/job/trunk-polygraph/ws/ returned 0. Command output: 14211 squid...@squid-cache.org-20150806121211-bchtf6rkgle4m0wc stderr: RevisionState revno:14211 revid:squid...@squid-cache.org-20150806121211-bchtf6rkgle4m0wc [trunk-polygraph] $ /bin/sh -xe /tmp/hudson2290702500532202276.sh + cd /home/jenkins/squidperf + python SquidBasicPerf.py --audited http://build.squid-cache.org/job/trunk-polygraph/355/artifact/logs/test.lx --jjid 801 --svnurl http://bzr.squid-cache.org/bzr/squid3/trunk/ --jobname trunk-polygraph Test is failed Build step 'Execute shell' marked build as failure Archiving artifacts [description-setter] Description set: ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
[squid-dev] Build failed in Jenkins: trunk-matrix » clang,d-ubuntu-utopic #291
See http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/291/ -- [...truncated 14267 lines...] am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[5]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth/digest' make[5]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth/digest' (cd negotiate make top_distdir=../../../squid-4.0.0-BZR distdir=../../../squid-4.0.0-BZR/src/auth/negotiate \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[5]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth/negotiate' make[5]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth/negotiate' (cd ntlm make top_distdir=../../../squid-4.0.0-BZR distdir=../../../squid-4.0.0-BZR/src/auth/ntlm \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[5]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth/ntlm' make[5]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth/ntlm' make[4]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/auth' (cd http make top_distdir=../../squid-4.0.0-BZR distdir=../../squid-4.0.0-BZR/src/http \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[4]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/http' (cd one make top_distdir=../../../squid-4.0.0-BZR distdir=../../../squid-4.0.0-BZR/src/http/one \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[5]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/http/one' make[5]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/http/one' make[4]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/http' (cd ip make top_distdir=../../squid-4.0.0-BZR distdir=../../squid-4.0.0-BZR/src/ip \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[4]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/ip' make[4]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/ip' (cd icmp make top_distdir=../../squid-4.0.0-BZR distdir=../../squid-4.0.0-BZR/src/icmp \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[4]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/icmp' make[4]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/icmp' (cd ident make top_distdir=../../squid-4.0.0-BZR distdir=../../squid-4.0.0-BZR/src/ident \ am__remove_distdir=: am__skip_length_check=: am__skip_mode_fix=: distdir) make[4]: Entering directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/ident' cd ../../.. /bin/bash http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/cfgaux/missing automake-1.14 --foreign src/ident/Makefile automake-1.14: error: no Automake input file found for 'src/ident/Makefile' automake-1.14: error: no input file found among supplied arguments Makefile:694: recipe for target '../../../src/ident/Makefile.in' failed make[4]: *** [../../../src/ident/Makefile.in] Error 1 make[4]: Leaving directory 'http://build.squid-cache.org/job/trunk-matrix/compiler=clang,label=d-ubuntu-utopic/ws/btlayer-00-default/squid-4.0.0-BZR/_build/src/ident' Makefile:7293: recipe for target 'distdir' failed make[3]: *** [distdir] Error 1 make[3]: Leaving directory
Re: [squid-dev] [PATCH] squid SSL subsystem did not initialized correctly
On 6/08/2015 9:54 p.m., Tsantilas Christos wrote:
Hi all,
Currently SSL subsystem did not initialized correctly in squid trunk.
This is because of the Security::ProxyOutgoingConfig.encryptTransport
which is always false so the client SSL CTX object never builds. As a
result squid may not start if SSL is configured. I am attaching a small
patch I am using in my squid trees to work with SSL.
This always-enabled code is not compatible with the possible admin
configuration:
tls_outgoing_options disable
Can you please try this instead:
Security::PeerOptions::parse(const char *token)
{
if (strncmp(token, disable, 7) == 0) {
clear();
+return;
} else if (strncmp(token, cert=, 5) == 0) {
...
} else {
debugs(3, DBG_CRITICAL, ERROR: Unknown TLS option ' ...
+return;
}
+
+encryptTransport = true;
}
If that works you can go through and also remove uses of
secure.encryptTransport = true from adaptation/ServiceConfig.cc and
cache_cf.cc where it is set next to a call to secure.parse()
... but not the other one where it is set to always-on for https_port.
If the final result still works, please commit.
Amos
___
squid-dev mailing list
squid-dev@lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-dev
Re: [squid-dev] [PATCH] HttpHeader migration (coverity-fixes branch)
Merged as revision 14210 with some minor added polish (ostream for
HttpHdrCcType and Http::HdrType).
On Wed, Aug 5, 2015 at 3:58 PM, Kinkie gkin...@gmail.com wrote:
On Wed, Aug 5, 2015 at 4:08 AM, Amos Jeffries squ...@treenet.co.nz
wrote:
On 5/08/2015 9:08 a.m., Kinkie wrote:
On Tue, Aug 4, 2015 at 2:58 PM, Amos Jeffries wrote:
On 4/08/2015 11:22 p.m., Kinkie wrote:
Hi all,
the attached patch is a build- and run-tested merge proposal for
next
round of the coverity-fixes branch, currently focusing on more
effective
header name - id lookups.
Here's the status with the current todo checklist:
* DONE 1. shift HDR_BAD_HDR to end of enum
* DONE 2. shift headers data array to http/RegistredHeaders.cc
* DONE 3. creatign LookupTable object from teh enum and array
* (with HDR_BAD_HDR as invalid value)
* DONE 4. replacing httpHeaderIdByName() uses with the lookup table
* NOT POSSIBLE 5. merge HDR_BAD_HDR and HDR_ENUM_END into one thing -
HDR_ENUM_END is overloaded meaning All headers in Manglers.
* DONE 6. replacing httpHeaderNameById with direct array lookups
* DONE 7. being looking at the other arrays removal
In working on this I found out several instances of enum abuses -
tracking
those down has been the hardest part of the effort.
HttpHeader::parse is being used to parse error page templates - thus
the
relaxed any_registered_header() checks in some methods, e.g.
HttpHdeader::addEntry().
Next steps, if there is consensus:
- moving LookupTable away from std::map to std::hash with custom
gperf-derived Hashers for extra boost
- investigating whether strongly-typed enums can be used instead of
C-style
enums in more places.
- moving away from homegrown bitfields (CBIT_TEST etc.) towards
std::vectorbool or std::vectorunsigned char, possibly via a class
bitfield or somesuch.
audit results:
* httpHdrCcCleanModule() and httpHdrScCleanModule() can both be fully
deleted now.
Yes. I chose to let them in: they are NOPs, not in the critical path,
and
may be useful in the future. Let me know if you still want them removed.
:-( more 'dead' code. This kind of thing is useful for
ctpr/dtor/functor/virtual definition. But C functions that are not even
used as functors is a waste of LOC and also compiler time dealing with
the symbols.
Its minor but still technical debt. The long-term plan is also to use
RegisteredRunners for this type of thing not C functions.
Ok. Removed all empty module cleanup functions - which are only invoked if
LEAKCHECK is enabled anyway.
* LookupTableRecord should be a class.
- custom storage types can then inherit from it, with it as the first
parent
Well, templatization can prevent that need, but sure. Should I implement
that?
Isn't nowadays a struct == all-public class though? It looks odd, but
functionally is the same
Avoid looks odd whenever possible :-) as you say, the compiler dont
care. But this is about us humans quick and easy reading it in 10 years
time.
I think its okay for useful code optimizations. Bt then the design
choice needs documenting so nobody breaks or undoes it casually.
Ok.
in src/HttpHeader.cc:
* can you move the headerLookupTable to src/http/RegisteredHeaders.cc
and the Http:: namespace as well please ?
Moved to RegisteredHeaders and renamed HDR_FOO to Http::HdrType::FOO .
Moving to a strongly-typed enum is unfortunately not feasible as eCAP
requires integer-enum equivalence; it may be that the whole change has
to
be reverted.
I thought enum class with first parameter value assigned 0
(HDR_ACCEPT = 0,) should do that. ie. strongly typed to unsigned
integer values.
No.
strongly-typed == doesn't automatically cast to int, if you use
strongly-typed you have to static_cast, while for old-style enum casting is
automatic.
in c++11 the full enum syntax (square brackets mean optional) is
enum [ class ] enumName [ : storage_specification ] {
enum_elem_1 [ = elem-1-representation ],
...
};
where storage_specification is the underlying integral data type.
Using the optional class specification disables auto-cast-to-int.
Elements can then be referenced by enumName::enum_elem_name , where the
enumName specification is mandatory for enum class, optional for
old-style enum.
BTW: I've tried changing the underlying storage to unsigned, and I get
asserts in masks calculation.
* any_registered_header() is wrong.
- it matches for HDR_OTHER which by definition is a non-registered
header
- assert_eid() is equivalent to any_valid_header()
Then we need to change name. We need two semantics:
1) any header which is valid and defined (including OTHER)
2) any header ID which will not go out-of-bounds (same as (1) +
HDR_BAD_HDR)
any suggestion?
(1) is already any_valid_header().
(2) any_HdrType_enum_value()
But where/why do we need (2) ?
Any value input or
Re: [squid-dev] [PATCH] PackableStream for cachemgr reports
I like this. On Wed, Aug 5, 2015 at 6:24 PM, Amos Jeffries squ...@treenet.co.nz wrote: Adds a PackableStream class which provides std::ostream semantics for writing data to a cachemgr report. FYI: This follows on from discussiosn back in 2011 regarding how to restructure the cachemgr internal data storage for relay between workers vs the report output formatting. Current trunk 'improved' code uses FooActionData classes to store and relay the report data internally, and classes implementing the Action API to format the report for delivery. For easy transition I have added an overload dump() method to the Action class API. New FooAction child classes should override the dump(ostream) method instead of the StoreEntry one. Writing their data to the stream. Older actions overload the now-deprecated dump(StoreEntry*) method and use some complicate nested function calls or share helper functions in complex ways which make it hard to upgrade all at once. So for now the dump(StoreEntry*) is made a stub that calls dump(ostream) if not overridden explicitly. The CacheManager class still uses dump(StoreEntry*) to get the reply payload from Action childs regardless of which dumper they implement. The basic report types are converted as part of this to be the first Actions using the stream output. The menu action in particular is slightly polished to benefit from stream abilities. Future steps along this path (in no particular order) are: * convert old Action classes to new API * refactor old C-code report generators to be Action classes - fixing display output syntax to minimal YAML as we go on both the above. It mostly is already, but some reports have wrong syntax. * update Actions to be hidden from the menu display * update Actions to support menu name aliases * update Action API to receive client desired format Amos ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev -- Francesco ___ squid-dev mailing list squid-dev@lists.squid-cache.org http://lists.squid-cache.org/listinfo/squid-dev
