Re: squid3-ipv6
Ah, Thank you Henrik. That clears a lot of confusion up. Sounds like Husni has got squid2 up to scratch and will be taking that forward. I have signed up at sourceforge using the account name amosjeffries for cvs access to the project branch. A bit about me for the record: I am CTO and NOC of a small hosting and development company based in New Zealand. I have a BSc in Software engineering (C++ and PHP Speciality), with some post-graduate work on IPv6. Aims: I am joining in an attempt to aid the push of Squid3-ipv6 up to the point of a stable release for active use within the local network here and the internet at large. Amos Jeffries Treehouse Networks Ltd. [EMAIL PROTECTED] - Original Message - From: Henrik Nordstrom [EMAIL PROTECTED] To: Amos Jeffries [EMAIL PROTECTED] Cc: squid-dev@squid-cache.org Sent: Thursday, January 04, 2007 4:19 AM Subject: Re: squid3-ipv6 Before I sign up at sourceforge and ask to join in the development I thou= ght=20 it would be good to contact you developers and find out what the current=20 state actually is. Current state of Squid3-ipv6 is that there is no maintainer working on the IPV6 code and tidying it up into pieces suitable for merging or completing the missing pieces if any. Has it actually gone stale? Currently unknown, but quite likely not much work is needed to make it compile again. and why? Lack of attention from developers working in IPv6 environments. None of the current active Squid developers have IPv6 connectivity. is it something only an experienced squid person can handle? Experience is more needed in the area of general C + ipv6 programming than Squid. We'll happily provide assistance on any squid specifics. or can someone like myself with a need and time=20 help to push it further with more coding? Absolutely. So get yourself subscribed to squid-dev and signed up for a sourceforge account and start digging into the code, please! Note: Squid-2 IPv6 apparently just got some attention. Another developer just posted information about his IPv6 squid work about the same time as you. url:http://marc.theaimsgroup.com/?l=3Dsquid-devm=3D116783428506284w=3D2 Regards Henrik
Re: Debug
Henrik Nordstrom wrote: mån 2007-01-08 klockan 17:27 +1300 skrev Amos Jeffries: The problem I am tracking is a DNS timeout when a TCP_MISS:DIRECT begins but can't figure out how to get any action-by-action or function-by-function record of what calls are being made in the area. Well. it is not so easy. Squid is effectively one big state machine centered around the comm loop. While it's waiting for an external event (network I/O mainly) there is no code only state.. I'm getting to see the picture as clearly an opaque problem. ;-) Over the last two days. The more I work on this the more I wish there was some developer documentation saying things like this clearly available in the source or on the website. So newbies like myself can quickly get a picture of where to look for the entry points to all the events without having to ask. Assuming it's an address lookup failing then the two most important entry points to look as is ipcache_nbgethostbyname and idnsGrokReply. ipcache_nbgethostbyname is the entry point to the DNS code for looking up the address of a host name. idnsGrokReply is where the raw DNS responses is processed and results handed back to the module which asked for it.. ipcache_nbgethostbyname to the point where the DNS query is ready to be sent should be normal linear code. idnsGrokReply is all linear code up to an including the callback to the module which initiated the DNS query (i.e. all the way to the forwarding code when the query came from there.). But there is one or two callbacks before the forwarding code is reached so it's perhaps not entirely obvious if only reading code. Regards Henrik Stranger, and stranger. Thank you for those entry points. I found the idnsGrokReply after much random code reading yesterday and found it doing something. Also, the error has moved further down the sequence now from DNS timeout to connection timeout (to localhost apache!). Without any code changes on my part. I'm going to have to trace the traffic in my environment I think to see exactly what its asking for before I get further into the code. :-( Amos
Re: new website issues
Adrian Chadd wrote: (for those of you who haven't seen it, the new website is taking shape @ http://new.squid-cache.org/ . The source for the website is in Squid CVS under www2 in case anyone feels like submitting me diffs.) The issue some people have seen with the splash image sitting on top of the text has to do with font sizes. The text inside the logo box grows too long to sit comfortably with the menu and pushes the menu down, screwing up the page layout. I'm not a CSS person at all. I'm learning how it works (and becoming increasingly horrified over time) so I'd appreciate some help on tidying this up before we release it to the public. All right. I was hoping you would be fine with this by yourself ;-) Just gave it a quick once-over. It looks good enough that its safe to raise a hand as a web developer. I'm willing to make it standards compliant if you want. Over-all not bad. 30% less errors-per-page than the Mozilla Site ;-) The funky text issues in the header may or may not have something to do with this: h1spana href=/Squid-/spanCache.org/a/h1 Should look like: h1a href=/spanSquid-/spanCache.org/a/h1 Or the latter half of the link gets cut and the /span could do weird spacing/wrapping stuff in IE. AYJ
Advancing IPv6 properly
I have been giving the branch a lot of thought and a little testing recently. I have come to the conclusion that the IPv6 branch under squid3 as it stands is in need of quite a makeover. I'd like your ideas on this: Creating an IPAddress class to replace all the nasty macros currently used in the (incomplete) preperation of HEAD for the future ipv6 branch merge. I am not certain at this point if the creation of such a class warrants it's own branch. One of the things I'd like guidance on is whether or not, and if so from where it would be best to branch. My reasons: - The dev guides I have read on the site read that squid3 is meant to be a C++ program (objects and explicit types _NO_ macros) but has not yet been fully ported up as yet. This would form another small step in that upgrade path. - The current ipv6 branch uses exclusively macros to enable a smooth upgrade when the ipv6 side of it is going. This is built into the nature of the branch and is partially moved up to HEAD already in preperation. Not a nice method of transition in an object-based app. - Most of the ipv6 mods still need to be finished and tested anyway, so will not suffer greatly from the shrinkage thhis would cause. Is it worth it? and would anyone with more knowledge of the future code than I have like to hazard a guess at an expected timespan for it? Amos
Re: locked out by partial cvsmerge
Adrian Chadd wrote: On Fri, Feb 23, 2007, Alex Rousskov wrote: Do we have command-line access to SourceForge CVS so that the lock file can be manually removed? What should I do to unlock myself? Shell.sourceforge.net is currently busted for some reason. I wonder whats going on. oh, grrr. It looks like the rest of us are also blocked by that lock. I just tried to merge the latest changes down into my branch and it gives me the same warning/error. cvs rtag: [02:21:29] waiting for rousskov's lock in /cvsroot/squid/squid3/lib/cppunit-1.10.0/doc I had to abort after a 30sec loop displaying that error for 10 minutes. Amos
Re: locked out by partial cvsmerge
Henrik Nordstrom wrote:
tis 2007-02-27 klockan 21:40 -0700 skrev Alex Rousskov:
Thanks for the explanation and snipped details. Unfortunately, it looks
like the problem is back and has even become worse as I see more locks
(from other developers) getting stuck now:
cvs rtag: [21:28:08] waiting for rousskov's lock in
/cvsroot/squid/squid3/lib/cppunit-1.10.0/doc
cvs rtag: [21:28:38] waiting for serassio's lock in
/cvsroot/squid/squid3/lib/cppunit-1.10.0/doc
cvs rtag: [21:29:08] waiting for amosjeffries's lock in
/cvsroot/squid/squid3/lib/cppunit-1.10.0/doc
All seem to be gone now...
Do you know whether there is something going on with SourceForge that
increases the probability of these locks getting stuck? Anything we can
do about it in the short term?
It might actually be the cvsmerge script.. the following line isn't
exactly kind to CVS..
o Check if there is any pending changes in the repository
diffl=`eecvs -q rdiff -kk -r ${mergetag} -r ${mergefrom} ${module} | head | wc
-l`
probably should remove the head from there, letting the rdiff run to completion.
Is that a hint for us all to edit our copies of cvsmerge?
Regards
Henrik
Amos
Re: redirection
Siddhesh PaiRaikar wrote: we need to block a particular site based on the content. we are planning to block the page from the function storeAppend() in store.c. wold it be the right place for blocking the page or should it be blocked elsewhere. httpAppendBody is probably a better place.. we are using squid 2.6 stable 6 and i cudnt find that fucntion in the source code.. is it a part of the later releases? also please if any one can tell us... A quick grep of my branch of squid3 shows no sign of it. Maybe its some of the new code from storework Henrik and Adrian have been doing to 2.6stable9+. what would be the easiest way to start the redirection to our server's custom page which says page blocked as in what function is responsible for starting the redirection to another page and the data structures also responsible for holding the current URL so that we can replace it with the new blocking URL of our server. Personally I use deny_info passing to a dynamic script with a few % parameters depending on the situation. Amos
DEBUG Section for IPAddress
I have so far not placed any permanent debug statements in the IPAddress object. It's come about time I should be making some more permanent ones and so need a section number. Is there some method I don't know yet for assigning a unique level or do I just pick an unused number and write it in? Amos
Squid3 ICAP / PRE6
Alex, You mentioned earlier that the ICAP integration into HEAD is going to be within the next few weeks. Exactly how close is it? I have a few bits of IPv6 I want to push up. But if you are very close I want to wait until your ICAP / PRE6 is released before I start changing HEAD on you. Amos
Re: DEBUG Section for IPAddress
Alex Rousskov wrote: On Mon, 2007-03-26 at 23:59 +1200, Amos Jeffries wrote: I have so far not placed any permanent debug statements in the IPAddress object. It's come about time I should be making some more permanent ones and so need a section number. Is there some method I don't know yet for assigning a unique level or do I just pick an unused number and write it in? If you have not received better instructions yet, just use the next unused number for the debug section and add it to your file(s) prefix just like most other source files do. The stale doc/debug-sections.txt is (or should be) generated based on the source code. Do not update that file unless you are willing to freshen it up first. Cheers, Alex. Thank you Alex, No I have not received any further instructions and assumed that was the best action to take. I picked level 36, but that turns out to be cache dir processing and I expect its in use own another branch. If by 'freshen up' you mean rebuild the file from source using a grep of the source files headers for DEBUG section and a manual massage. I did that earlier before seeing the doc/ file. The update patch resulting is attached. FYI to all: IP Handling will join IP Cache in section 14, unless there are any objections by easter. Amos --- DEBUG_levels.txt 2007-04-04 12:56:03.0 +1200 +++ doc/debug-sections.txt 2007-03-27 20:12:49.0 +1200 @@ -3,14 +3,11 @@ section 0 Client Database section 0 Debug Routines section 0 DNS Resolver -section 0 Hash Tables -section 0 UFS Store Dump section 0 WWW Client section 1 Startup and Main Loop section 2 Unlink Daemon section 3 Configuration File Parsing section 4 Error Generation -section 5 Comms section 5 Socket Functions section 6 Disk I/O Routines section 7 Multicast @@ -21,35 +18,21 @@ section 12Internet Cache Protocol section 13High Level Memory Pool Management section 14IP Cache -section 14IP Storage and Handling section 15Neighbor Routines section 16Cache Manager Objects section 17Request Forwarding section 18Cache Manager Statistics section 19Store Memory Primitives section 20Storage Manager -section 20Storage Manager Heap-based replacement -section 20Storage Manager Logging Functions -section 20Storage Manager MD5 Cache Keys -section 20Storage Manager Swapfile Metadata -section 20Storage Manager Swapfile Unpacker -section 20Storage Manager Swapin Functions -section 20Storage Manager Swapout Functions -section 20Store Rebuild Routines -section 21Integer functions section 21Misc Functions -section 21Time Functions section 22Refresh Calculation section 23URL Parsing -section 23URL Scheme parsing - +section 24WAIS Relay section 25MIME Parsing section 26Secure Sockets Layer Proxy section 27Cache Announcer section 28Access Control section 29Authenticator -section 29Negotiate Authenticator -section 29NTLM Authenticator section 30Ident (RFC 931) section 31Hypertext Caching Protocol section 32Asynchronous Disk I/O @@ -61,26 +44,20 @@ section 38Network Measurement Database section 39Cache Array Routing Protocol section 40User-Agent and Referer logging -section 40User-Agent logging section 41Event Processing section 42ICMP Pinger program section 43AIOPS -section 43Windows AIOPS section 44Peer Selection Algorithm section 45Callback Data Registry section 46Access Log -section 47Store COSS Directory Routines section 47Store Directory Routines section 48Persistent Connections -section 49SNMP Interface -section 49SNMP support +section 49SNMP section 50Log file handling section 51Filedescriptor Functions section 52URN Parsing section 53AS Number handling -section 53 Radix tree data structure implementation section 54Interprocess Communication -section 54Windows Interprocess Communication section 55HTTP Header section 56HTTP Message Body section 57HTTP Status-line @@ -89,7 +66,7 @@ section 60Packer: A uniform interface to store-like modules section 61Redirector section 62Generic Histogram -section 63Low Level Memory Pool Management +section 63- section 64HTTP Range Header section 65HTTP Cache Control Header section 66HTTP Header Tools @@ -104,38 +81,18 @@ section 75WHOIS protocol section 76Internal Squid Object handling section 77Delay Pools -section 78DNS lookups; interacts with lib/rfc1035.c -section 79Disk IO Routines -section 79Squid-side DISKD I/O functions. -section 79Squid-side Disk I/O functions. -section 79Storage Manager COSS Interface -section 79Storage Manager UFS Interface -section 80WCCP Support -section 81aio_xxx() POSIX emulation on Windows -section 81 CPU Profiling
Re: DEBUG Section for IPAddress
Gah! I diffed the files backwards. Here is the actual patch for doc/debug-sections.txt Amos --- doc/debug-sections.txt 2007-03-27 20:12:49.0 +1200 +++ DEBUG_levels.txt 2007-04-04 12:56:03.0 +1200 @@ -3,11 +3,14 @@ section 0 Client Database section 0 Debug Routines section 0 DNS Resolver +section 0 Hash Tables +section 0 UFS Store Dump section 0 WWW Client section 1 Startup and Main Loop section 2 Unlink Daemon section 3 Configuration File Parsing section 4 Error Generation +section 5 Comms section 5 Socket Functions section 6 Disk I/O Routines section 7 Multicast @@ -18,21 +21,35 @@ section 12Internet Cache Protocol section 13High Level Memory Pool Management section 14IP Cache +section 14IP Storage and Handling section 15Neighbor Routines section 16Cache Manager Objects section 17Request Forwarding section 18Cache Manager Statistics section 19Store Memory Primitives section 20Storage Manager +section 20Storage Manager Heap-based replacement +section 20Storage Manager Logging Functions +section 20Storage Manager MD5 Cache Keys +section 20Storage Manager Swapfile Metadata +section 20Storage Manager Swapfile Unpacker +section 20Storage Manager Swapin Functions +section 20Storage Manager Swapout Functions +section 20Store Rebuild Routines +section 21Integer functions section 21Misc Functions +section 21Time Functions section 22Refresh Calculation section 23URL Parsing -section 24WAIS Relay +section 23URL Scheme parsing + section 25MIME Parsing section 26Secure Sockets Layer Proxy section 27Cache Announcer section 28Access Control section 29Authenticator +section 29Negotiate Authenticator +section 29NTLM Authenticator section 30Ident (RFC 931) section 31Hypertext Caching Protocol section 32Asynchronous Disk I/O @@ -44,20 +61,26 @@ section 38Network Measurement Database section 39Cache Array Routing Protocol section 40User-Agent and Referer logging +section 40User-Agent logging section 41Event Processing section 42ICMP Pinger program section 43AIOPS +section 43Windows AIOPS section 44Peer Selection Algorithm section 45Callback Data Registry section 46Access Log +section 47Store COSS Directory Routines section 47Store Directory Routines section 48Persistent Connections -section 49SNMP +section 49SNMP Interface +section 49SNMP support section 50Log file handling section 51Filedescriptor Functions section 52URN Parsing section 53AS Number handling +section 53 Radix tree data structure implementation section 54Interprocess Communication +section 54Windows Interprocess Communication section 55HTTP Header section 56HTTP Message Body section 57HTTP Status-line @@ -66,7 +89,7 @@ section 60Packer: A uniform interface to store-like modules section 61Redirector section 62Generic Histogram -section 63- +section 63Low Level Memory Pool Management section 64HTTP Range Header section 65HTTP Cache Control Header section 66HTTP Header Tools @@ -81,18 +104,38 @@ section 75WHOIS protocol section 76Internal Squid Object handling section 77Delay Pools -section 78DNS lookups -section 79Lowlevel store I/O -section 80WCCP -section 81Cache Store +section 78DNS lookups; interacts with lib/rfc1035.c +section 79Disk IO Routines +section 79Squid-side DISKD I/O functions. +section 79Squid-side Disk I/O functions. +section 79Storage Manager COSS Interface +section 79Storage Manager UFS Interface +section 80WCCP Support +section 81aio_xxx() POSIX emulation on Windows +section 81 CPU Profiling Routines section 82External ACL section 83SSL accelerator support section 84Helper process maintenance -section 85Client side request management - after parsing, before caching -section 86ESI processing -section 87client side stream management -section 88Client side reply management - from store to stream -section 89NAT / IP Interception -section 90Store Client -section 91Http Surrogate-Control Header -section 92Store File System +section 85Client-side Request Routines +section 86ESI Expressions +section 86ESI Processing +section 87Client-side Stream routines. +section 88Client-side Reply Routines +section 89NAT / IP Interception +section 90HTTP Cache Control Header +section 90Storage Manager Client-Side Interface + +section 92Storage File System +section 93ICAP (RFC 3507) Client + + +section ? Common Removal policy +section --External DISKD process implementation. +section ?? FDE +section ? HEAP based removal policies +section ??Linked list functions +section ? LRU Removal
Re: some help with the website please
Adrian Chadd wrote: I hate to ask for help when I said I'd do it, but I'm running very short on spare time at the moment and I'd appreciate some help in finishing off the new.squid-cache.org website so its ready to be made live. Someone with server access: help, please? Adrian I have time now, but no server access. Is it doable without server access? Amos
Re: DEBUG Section for IPAddress
Alex Rousskov wrote: On Wed, 2007-04-04 at 13:02 +1200, Amos Jeffries wrote: Gah! I diffed the files backwards. Here is the actual patch for doc/debug-sections.txt Amos, The doc/mk-debugs.sh script you were using (or equivalent) needs to be fixed. It needs to grep files in subdirectories of src/. For example, it currently misses: DEBUG: section 93 ICAP (RFC 3507) Client Using 'find src -name *.cc -o -name *.h' instead of a src/*.cc mask may work. Alex. You looking at my first copy of the patch? That was badly mangled. Its there in the amended patch I re-posted: +section 90HTTP Cache Control Header +section 90Storage Manager Client-Side Interface + +section 92Storage File System +section 93ICAP (RFC 3507) Client + + +section ? Common Removal policy +section --External DISKD process implementation. +section ?? FDE +section ? HEAP based removal policies Amos
IPv6 developments for HEAD
Attached are two patches which constitute part of the core developments
for protocol-independent handling of IP addresses in squid3.
The patches apply to lib/ and include/ respectively, and do the following:
- Rollback previous alterations to DNS library code in rfc1035.c
and return it to the pure IPv4 protocol DNS lookups as defined
by RFC1035.
- Fix 0-day bug where base64 code does not provide dependencies
for utils.h properly.
- defines new protocol-independant class IPAddress for IP storage
and handling.
class IPAddress provides:
- protocol independent storage of in_addr, in6_addr, sockaddr_in,
and sockaddr_in6.
- functionality to convert between any of the contained IP formats.
Either the binary struct or string representations.
- defines a macro MAX_IPSTRLEN to indicate the length of buffers
needed to hold string representations of any address string
produced.
- functionality to explicitly check the compatible protocol needed
can be retrieved. Useful for high-level protocols such as WCCP,
ARP and ICMP which are transport-specific to determine if the
IPAddress can be one suitable for that protocol.
The class is built from basic data types and contains no dynamic
memory operations outside its own allocated block and references
no pointers. Meaning it can be safely used as a basic data type in
its own right. With the added abilities of data conversion through
assignments from any of the above mentioned storage types.
The storage of sockaddr_* information implies that both port and
address details are stored, and can be set or retrieved
independently from any IPAdress which contains them.
Functionality is provided to convert from hostent structures, but at
present IPAddress can only hold a single IP. This means the
conversion will ONLY save the first entry in the hostent list of
IPA's. Where an non-first entry of the hostent must be stored it
needs to be explicitly de-referenced and cast from its native char**
stored format inside hostent to the appropriate in_addr or in6_addr
type which can then can be passed to the IPAddress object for
conversion.
Amos Jeffries
Patch file generated Sat Apr 7 15:24:21 NZST 2007 from
CVS branch squid3-ipv6
CVS base branch HEAD
CVS repository: [EMAIL PROTECTED]:/cvsroot/squid
CVS module: squid3/include
cvs -q rdiff -u -kk -r Z-squid3-ipv6_merge_HEAD -r squid3-ipv6 squid3/include
Index: squid3/include/IPAddress.h
diff -u /dev/null squid3/include/IPAddress.h:1.1.2.8
--- /dev/null Thu Jan 1 01:00:00 1970
+++ squid3/include/IPAddress.h Thu Apr 5 02:47:56 2007
@@ -0,0 +1,230 @@
+/*
+ * $Id$
+ */
+#ifndef _INC_IPADDRESS_H
+#define _INC_IPADDRESS_H
+
+#include autoconf.h
+
+#include sys/socket.h
+#include netinet/in.h
+#include iostream
+#include netdb.h
+
+
+/// Specify the type of Address being or to be handled.
+enum IPAddressType {
+None =0, /// Nothing Special. Equates to default if used as a parameter.
+SockAddr =1, /// Full SocketAddr Details Stored. Port and Address can be used together or seperately.
+IPv4 =2, /// Pure IPv4 address stored (conversion up must be done explicitly)
+IPv6 =4, /// Pure IPv6 address Stored (no conversion to IPv4 possible)
+IPv64=6 /// Dual-Address stored (can return either IPv6 OR IPv4)
+};
+
+/// Length of buffer that needs to be allocated to old a null-terminated IP-string
+// Yuck. But there are still structures that need it to be an 'integer constant'.
+#define MAX_IPSTRLEN 75
+
+/**
+ * Holds and manipulates IPv4, IPv6, and Socket Addresses.
+ */
+class IPAddress
+{
+public:
+ /** @name Constructors and Destructor */
+ /[EMAIL PROTECTED]/
+// default constructor.
+IPAddress();
+IPAddress(const IPAddress );
+IPAddress(const struct in_addr );
+IPAddress(const struct sockaddr_in );
+#ifdef USE_IPV6
+IPAddress(const struct in6_addr );
+IPAddress(const struct sockaddr_in6 );
+#endif
+IPAddress(const hostent *);
+IPAddress(const char*);
+/// Default destructor.
+~IPAddress();
+/[EMAIL PROTECTED]/
+
+/** @name Assignment Operators */
+/[EMAIL PROTECTED]/
+IPAddress operator =(const IPAddress s);
+IPAddress operator =(struct sockaddr_in const s);
+IPAddress operator =(struct in_addr const s);
+#ifdef USE_IPV6
+IPAddress operator =(struct in6_addr const s);
+IPAddress operator =(struct sockaddr_in6 const s);
+#endif
+bool operator =(const struct hostent *s);
+bool operator =(const char *s);
+/[EMAIL PROTECTED]/
+
+/** @name Boolean Operators */
+/[EMAIL PROTECTED]/
+bool operator ==(IPAddress const s) const;
+bool operator =(IPAddress const rhs) const;
+bool operator =(IPAddress const rhs) const;
+
+public:
+ /* methods */
+
+/** Test whether content can be used as an IPv4 address
Re: IPv6 developments for HEAD
Guido Serassio wrote:
Hi Amos,
At 07.24 07/04/2007, Amos Jeffries wrote:
Attached are two patches which constitute part of the core
developments for protocol-independent handling of IP addresses in squid3.
There is a problem in your patch.
You are including many include files without checking for the existence
of them, in example:
sys/socket.h
netinet/in.h
netdb.h
Many of them are not available or are named differently on Unix and
Windows, so this patch will broke immediately the build process on too
much platforms.
So you should change your patch to use conditional including like other
source code.
Regards
Guido
Thank Guido. I wondered what the reason was behind those.
Fixed. New patches attached.
Amos
Patch file generated Sat Apr 7 23:19:12 NZST 2007 from
CVS branch squid3-ipv6
CVS base branch HEAD
CVS repository: [EMAIL PROTECTED]:/cvsroot/squid
CVS module: squid3/include
cvs -q rdiff -u -kk -r Z-squid3-ipv6_merge_HEAD -r squid3-ipv6 squid3/include
Index: squid3/include/IPAddress.h
diff -u /dev/null squid3/include/IPAddress.h:1.1.2.10
--- /dev/null Thu Jan 1 01:00:00 1970
+++ squid3/include/IPAddress.h Sat Apr 7 03:58:45 2007
@@ -0,0 +1,247 @@
+/*
+ * $Id$
+ */
+#ifndef _INC_IPADDRESS_H
+#define _INC_IPADDRESS_H
+
+#include autoconf.h
+
+#if HAVE_SYS_SOCKET_H
+#include sys/socket.h
+#endif
+#if HAVE_NETINET_IN_H
+#include netinet/in.h
+#endif
+#if HAVE_NETDB_H !defined(_SQUID_NETDB_H_) /* protect NEXTSTEP */
+#define _SQUID_NETDB_H_
+#ifdef _SQUID_NEXT_
+#include netinet/in_systm.h
+#endif
+#include netdb.h
+#endif
+
+#include iostream
+
+
+/// Specify the type of Address being or to be handled.
+enum IPAddressType {
+None =0, /// Nothing Special. Equates to default if used as a parameter.
+SockAddr =1, /// Full SocketAddr Details Stored. Port and Address can be used together or seperately.
+IPv4 =2, /// Pure IPv4 address stored (conversion up must be done explicitly)
+IPv6 =4, /// Pure IPv6 address Stored (no conversion to IPv4 possible)
+IPv64=6 /// Dual-Address stored (can return either IPv6 OR IPv4)
+};
+
+/// Length of buffer that needs to be allocated to old a null-terminated IP-string
+// Yuck. But there are still structures that need it to be an 'integer constant'.
+#define MAX_IPSTRLEN 75
+
+/**
+ * Holds and manipulates IPv4, IPv6, and Socket Addresses.
+ */
+class IPAddress
+{
+public:
+ /** @name Constructors and Destructor */
+ /[EMAIL PROTECTED]/
+// default constructor.
+IPAddress();
+IPAddress(const IPAddress );
+IPAddress(const struct in_addr );
+IPAddress(const struct sockaddr_in );
+#ifdef USE_IPV6
+IPAddress(const struct in6_addr );
+IPAddress(const struct sockaddr_in6 );
+#endif
+IPAddress(const hostent *);
+IPAddress(const char*);
+/// Default destructor.
+~IPAddress();
+/[EMAIL PROTECTED]/
+
+/** @name Assignment Operators */
+/[EMAIL PROTECTED]/
+IPAddress operator =(const IPAddress s);
+IPAddress operator =(struct sockaddr_in const s);
+IPAddress operator =(struct in_addr const s);
+#ifdef USE_IPV6
+IPAddress operator =(struct in6_addr const s);
+IPAddress operator =(struct sockaddr_in6 const s);
+#endif
+bool operator =(const struct hostent *s);
+bool operator =(const char *s);
+/[EMAIL PROTECTED]/
+
+/** @name Boolean Operators */
+/[EMAIL PROTECTED]/
+bool operator ==(IPAddress const s) const;
+bool operator =(IPAddress const rhs) const;
+bool operator =(IPAddress const rhs) const;
+
+public:
+ /* methods */
+
+/** Test whether content can be used as an IPv4 address
+ * \retval true if content was received as an IPv4 address
+ * \retval true if content was received as an IPv4-Mapped address
+ * \retval false if content was received as a non-mapped IPv6 native address.
+ */
+inline bool IsIPv4() const { return (m_Type IPv4); };
+
+/** Test whether content can be used as an IPv6 address.
+ * \retval true if --enable-ipv6 has been compiled.
+ * \retval false if --disable-ipv6 has been compiled.
+ * \retval false if --with-ipv6-split-stack has been compiled AND content is IPv4-mapped.
+ */
+inline bool IsIPv6() const {
+return (m_Type IPv6)
+#ifdef USE_IPV6_SPLITSTACK
+ !(m_Type IPv4)
+#endif
+;
+};
+
+/** Test whether content can be used as a Socket address.
+ * \retval true if address AND port are both set
+ * \retval true if content was received as a Socket address
+ * \retval false if port in unset (zero)
+ */
+inline bool IsSockAddr() const { return (m_Type SockAddr); };
+
+/** Content-neutral test for whether the specific IP case ANY_ADDR is stored.
+ * This is the default content of a new
Re: IPv6 developments for HEAD
Guido Serassio wrote: Hi Amos, At 13.21 07/04/2007, Amos Jeffries wrote: Thank Guido. I wondered what the reason was behind those. Fixed. New patches attached. Still some details on the include side: In IPAddress.cc you should first include squid.h like other sources, in this way you should no more need the inclusion of many other includes in IPAddress.cc itself and IPAddress.h. Regards Guido You are overlooking the fact that these files are is libsquid.la. squid.h is not available there and will break many compile dependancies if it is included by those files. Amos
Re: IPv6 developments for HEAD
Guido Serassio wrote: Hi Amos, At 13.52 07/04/2007, Amos Jeffries wrote: Guido Serassio wrote: Hi Amos, At 13.21 07/04/2007, Amos Jeffries wrote: Thank Guido. I wondered what the reason was behind those. Fixed. New patches attached. Still some details on the include side: In IPAddress.cc you should first include squid.h like other sources, in this way you should no more need the inclusion of many other includes in IPAddress.cc itself and IPAddress.h. Regards Guido You are overlooking the fact that these files are is libsquid.la. squid.h is not available there and will break many compile dependancies if it is included by those files. Sorry, I don't realize that IPAddress.cc is placed in the lib directory :-) So, in IPAddress.cc you should first include config.h ore some other include including config.h and not autoconf.h in IPAddress.h. Regards Guido Okay. While I'm re-arranging I suppose I should also include the GNU disclaimers license stuff too. Anything else before I make new patches? Amos
Re: IPv6 developments for HEAD
Sigh. For some weird reason my test machine built squid with the
previous patches despite a missing #include.
Here is an updated patch set, tested on two machines.
Patch file generated Sun Apr 8 01:12:39 NZST 2007 from
CVS branch squid3-ipv6
CVS base branch HEAD
CVS repository: [EMAIL PROTECTED]:/cvsroot/squid
CVS module: squid3/include
cvs -q rdiff -u -kk -r Z-squid3-ipv6_merge_HEAD -r squid3-ipv6 squid3/include
Index: squid3/include/IPAddress.h
diff -u /dev/null squid3/include/IPAddress.h:1.1.2.12
--- /dev/null Thu Jan 1 01:00:00 1970
+++ squid3/include/IPAddress.h Sat Apr 7 06:06:43 2007
@@ -0,0 +1,303 @@
+
+/*
+ * $Id$
+ *
+ * DEBUG: section 14IP Storage and Handling
+ * AUTHOR: Amos Jeffries
+ *
+ * SQUID Internet Object Cache http://squid.nlanr.net/Squid/
+ * --
+ *
+ * Squid is the result of efforts by numerous individuals from the
+ * Internet community. Development is led by Duane Wessels of the
+ * National Laboratory for Applied Network Research and funded by the
+ * National Science Foundation. Squid is Copyrighted (C) 1998 by
+ * the Regents of the University of California. Please see the
+ * COPYRIGHT file for full details. Squid incorporates software
+ * developed and/or copyrighted by other sources. Please see the
+ * CREDITS file for full details.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
+ *
+ */
+
+#ifndef _INC_IPADDRESS_H
+#define _INC_IPADDRESS_H
+
+#if HAVE_SYS_SOCKET_H
+#include sys/socket.h
+#endif
+#if HAVE_NETINET_IN_H
+#include netinet/in.h
+#endif
+#if HAVE_NETDB_H !defined(_SQUID_NETDB_H_) /* protect NEXTSTEP */
+#define _SQUID_NETDB_H_
+#ifdef _SQUID_NEXT_
+#include netinet/in_systm.h
+#endif
+#include netdb.h
+#endif
+
+#include iostream
+
+
+/// Specify the type of Address being or to be handled.
+enum IPAddressType {
+None =0, /// Nothing Special. Equates to default if used as a parameter.
+SockAddr =1, /// Full SocketAddr Details Stored. Port and Address can be used together or seperately.
+IPv4 =2, /// Pure IPv4 address stored (conversion up must be done explicitly)
+IPv6 =4, /// Pure IPv6 address Stored (no conversion to IPv4 possible)
+IPv64=6 /// Dual-Address stored (can return either IPv6 OR IPv4)
+};
+
+/// Length of buffer that needs to be allocated to old a null-terminated IP-string
+// Yuck. But there are still structures that need it to be an 'integer constant'.
+#define MAX_IPSTRLEN 75
+
+/**
+ * Holds and manipulates IPv4, IPv6, and Socket Addresses.
+ */
+
+class IPAddress
+{
+
+public:
+/** @name Constructors and Destructor */
+/[EMAIL PROTECTED]/
+// default constructor.
+IPAddress();
+IPAddress(const IPAddress );
+
+IPAddress(const struct in_addr );
+
+IPAddress(const struct sockaddr_in );
+#ifdef USE_IPV6
+
+IPAddress(const struct in6_addr );
+
+IPAddress(const struct sockaddr_in6 );
+#endif
+
+IPAddress(const hostent *);
+IPAddress(const char*);
+/// Default destructor.
+~IPAddress();
+/[EMAIL PROTECTED]/
+
+/** @name Assignment Operators */
+/[EMAIL PROTECTED]/
+IPAddress operator =(const IPAddress s);
+
+IPAddress operator =(struct sockaddr_in const s);
+
+IPAddress operator =(struct in_addr const s);
+#ifdef USE_IPV6
+
+IPAddress operator =(struct in6_addr const s);
+
+IPAddress operator =(struct sockaddr_in6 const s);
+#endif
+
+bool operator =(const struct hostent *s);
+bool operator =(const char *s);
+/[EMAIL PROTECTED]/
+
+/** @name Boolean Operators */
+/[EMAIL PROTECTED]/
+bool operator ==(IPAddress const s) const;
+bool operator =(IPAddress const rhs) const;
+bool operator =(IPAddress const rhs) const;
+
+public:
+/* methods */
+
+/** Test whether content can be used as an IPv4 address
+ * \retval true if content was received as an IPv4 address
+ * \retval true if content was received as an IPv4-Mapped address
+ * \retval false if content was received as a non-mapped IPv6 native address.
+ */
+inline bool IsIPv4() const { return (m_Type IPv4); };
+
+/** Test whether content can be used as an IPv6 address.
+ * \retval true if --enable-ipv6 has been compiled.
+ * \retval false
Re: IPv6 developments for HEAD
Alex Rousskov wrote: On Wed, 2007-04-11 at 11:41 +1200, [EMAIL PROTECTED] wrote: On Sat, 2007-04-07 at 17:24 +1200, Amos Jeffries wrote: Attached are two patches which constitute part of the core developments for protocol-independent handling of IP addresses in squid3. In your opinion, should these be committed to Squid 3.0? Are they likely to cause short-term stability problems? Should they be applied to Squid 3.1 instead? Thank you, Alex. Yes. No. both?. I would like to see them in 3.0. The new object I am submitting is isolated 'infrastructure' which does not affect the rest of squid in any way. It is itself the stable base needed for future work. Interesting. If IPAddress is not used by Squid before and after your patch, is there a reason to commit it now? Originally, I thought that you are modifying common code and want to commit ASAP to minimize future conflicts. Now I understand that IpAddress addition does not alter anything in Squid core (and you are not asking to commit the rest of your changes, which do). On one hand, I am tempted to vote immediate inclusion of IPAddress simply to satisfy a valuable developer. On the other hand, I do not understand why you want that file to be in HEAD if nothing is using it. Could you please clarify why you want to see IPAddress in HEAD? Currently the branch is looking at nearly 5500 lines of code changed. With nearly 3000 lines removed from the core of squid so far. In complete agreement with Henrik and Adrians views that stability in 3.0 should not be risked. I am nevertheless trying to drop that huge difference in 'small' discrete isolated chunks. I have managed to find 3 areas that will do a large amount of reduction without changing or touching the core code in any way. RFCs were step 1. IPAddress is step 2. A new rfc3596 (DNS) library based non the RFCs is a third, but that still needs major testing and is weeks away I think. When the 'non-changes' are out of the branch and in HEAD waiting to be used. The actual core changes can push ahead cleanly for IPv6 in 3.1, both inside my branch and in any others who want to jump for the new ability while HEAD is closed to them. Amos
Re: Squid 3.0 bugs that need you.
Alex Rousskov wrote: Hi there, I went through the remaining Squid3 non-enhancement bug reports targeted for 3.0. For most bugs, I was able to close or comment in the bugzilla. The following bugs are special because the questions did not seem appropriate for bugzilla. Please review. http://www.squid-cache.org/bugs/show_bug.cgi?id=1345 Bug 1345 - Problem with tar-file Squid3 needs a volunteer to remove cppunit from the distribution and, probably, from CVS. Anybody is willing to do that? This is a perfect work item for folks who hate C++ development as it should not require much C++ programming... yay! this has been causing me some nightmares just thinking about it. I'll get onto a purge this weekend unless there is anyone else who wants to do it or veto it. Speak up within 8 hours or I'll create a 'nocpp' branch and get to work. http://www.squid-cache.org/bugs/show_bug.cgi?id=1475 Bug 1475 - File descriptor limit does work. I am not sure what the final resolution for this bug is. Henrik? Adrian? ESI bugs targeted for 3.0 http://www.squid-cache.org/bugs/show_bug.cgi?id=951 http://www.squid-cache.org/bugs/show_bug.cgi?id=1088 If nobody is willing to work on ESI now, I would like to bump any 3.0 target milestone for ESI to 3.1 because ESI is an optional feature. Any objections? Also, there are quite a few bugs targeted for 3.1 and bugs without a specific target. I reviewed the ones without a target for possible 3.0 inclusion and updated some of the comments. If there is a Squid3 bug that is not targeted for 3.0 release, but you think it should, please flag it as such. Thank you, Alex.
Re: Squid 3.0 bugs that need you.
Alex Rousskov wrote: Hi there, I went through the remaining Squid3 non-enhancement bug reports targeted for 3.0. For most bugs, I was able to close or comment in the bugzilla. The following bugs are special because the questions did not seem appropriate for bugzilla. Please review. http://www.squid-cache.org/bugs/show_bug.cgi?id=1345 Bug 1345 - Problem with tar-file Squid3 needs a volunteer to remove cppunit from the distribution and, probably, from CVS. Anybody is willing to do that? This is a perfect work item for folks who hate C++ development as it should not require much C++ programming... Done, patch submitted to bugzilla. Branch nocpp under squid3 HEAD has been left open for now. Amos
Re: Squid 3.0 bugs that need you.
Alex Rousskov wrote: Hi there, I went through the remaining Squid3 non-enhancement bug reports targeted for 3.0. For most bugs, I was able to close or comment in the bugzilla. The following bugs are special because the questions did not seem appropriate for bugzilla. Please review. http://www.squid-cache.org/bugs/show_bug.cgi?id=1345 Bug 1345 - Problem with tar-file Squid3 needs a volunteer to remove cppunit from the distribution and, probably, from CVS. Anybody is willing to do that? This is a perfect work item for folks who hate C++ development as it should not require much C++ programming... http://www.squid-cache.org/bugs/show_bug.cgi?id=1475 Bug 1475 - File descriptor limit does work. I am not sure what the final resolution for this bug is. Henrik? Adrian? ESI bugs targeted for 3.0 http://www.squid-cache.org/bugs/show_bug.cgi?id=951 http://www.squid-cache.org/bugs/show_bug.cgi?id=1088 If nobody is willing to work on ESI now, I would like to bump any 3.0 target milestone for ESI to 3.1 because ESI is an optional feature. Any objections? Also, there are quite a few bugs targeted for 3.1 and bugs without a specific target. I reviewed the ones without a target for possible 3.0 inclusion and updated some of the comments. If there is a Squid3 bug that is not targeted for 3.0 release, but you think it should, please flag it as such. Thank you, Alex. Here is the patch for that additional ICAP problem that came to light testing cppunit removal. Amos --- Makefile.am 2007-04-14 03:26:33.0 +1200 +++ Makefile.am-2 2007-04-14 02:41:52.0 +1200 @@ -1249,6 +1249,7 @@ ACLStringData.cc \ ACLRegexData.cc \ ACLUserData.cc \ + AsyncCall.cc \ authenticate.cc \ BodyPipe.cc \ cache_manager.cc \ @@ -1413,6 +1414,7 @@ ACLStringData.cc \ ACLRegexData.cc \ ACLUserData.cc \ + AsyncCall.cc \ authenticate.cc \ BodyPipe.cc \ cache_manager.cc \ @@ -1563,6 +1565,7 @@ ACLStringData.cc \ ACLRegexData.cc \ ACLUserData.cc \ + AsyncCall.cc \ authenticate.cc \ BodyPipe.cc \ cache_manager.cc \ @@ -1737,6 +1740,7 @@ ACLStringData.cc \ ACLRegexData.cc \ ACLUserData.cc \ + AsyncCall.cc \ authenticate.cc \ BodyPipe.cc \ cache_cf.cc \ @@ -1893,6 +1897,7 @@ ACLStringData.cc \ ACLRegexData.cc \ ACLUserData.cc \ + AsyncCall.cc \ authenticate.cc \ BodyPipe.cc \ cache_manager.cc \ @@ -2228,6 +2233,7 @@ ACLStringData.cc \ ACLRegexData.cc \ ACLUserData.cc \ + AsyncCall.cc \ authenticate.cc \ BodyPipe.cc \ cache_manager.cc \
Re: Style of commit messages
Adrian Chadd wrote: On Mon, Apr 16, 2007, Henrik Nordstrom wrote: Please follow the following style in your commit messages First if the primary author of the patch isn't you then have a line indicating the autor of the change Author: Name email Next a short summary of the change. Approximately one line. Then a blank line Followed by a full description of the change. The first two lines (Author:, and the summary) is picked up by the changeset tools. Could you put this into the default CVS commit templates? Adrian While you are adjusting cvs is there a good reason for the squid-cvs email being in default diff format? I think we would all find it a little more useful reading and using updates if that email could be a unified diff. Amos
cppunit removal
AUTHOR: Ams Jeffries [EMAIL PROTECTED]
The squid 3.0 will not be packaged with cppunit 1.10.0.
The developers have decided that the packaging of cppunit within the
squid sources has caused more trouble than was solved. Ease of install
independently of squid combined with the apparently widespread use of
cppunit as a standard tool we feel there is no need for it to be included.
For the initial release to date there will be no replacement testing
package and no automatic install for cppunit unless one is added by OS
distribution maintainers.
Squid releases already makes an attempt to detect external installs of
cppunit and use those by preference over the packaged version. This
existing detection has been left in place and will enable testing on any
system with cppunit. Maintainers and source testers will need to have
cppunit installed on their systems or not use make check.
Patch file generated Tue Apr 17 00:18:17 NZST 2007 from
CVS branch nocpp
CVS base branch HEAD
CVS repository: [EMAIL PROTECTED]:/cvsroot/squid
CVS module: squid3
cvs -q rdiff -u -kk -r Z-nocpp_merge_HEAD -r nocpp squid3
Index: squid3/bootstrap.sh
diff -u squid3/bootstrap.sh:1.20 squid3/bootstrap.sh:1.20.6.2
--- squid3/bootstrap.sh:1.20Mon Mar 20 19:15:31 2006
+++ squid3/bootstrap.sh Fri Apr 13 03:57:49 2007
@@ -66,8 +66,7 @@
for dir in \
\
- lib/libTrie \
- lib/cppunit-1.10.0
+ lib/libTrie
do
if [ -z $dir ] || [ -d $dir ]; then
if (
Index: squid3/configure.in
diff -u squid3/configure.in:1.130 squid3/configure.in:1.129.2.2
--- squid3/configure.in:1.130 Sun Apr 15 08:00:25 2007
+++ squid3/configure.in Mon Apr 16 00:11:34 2007
@@ -1789,17 +1789,17 @@
SQUID_CPPUNIT_LA=''
SQUID_CPPUNIT_INC=`$CPPUNITCONFIG --cflags`
else
- SQUID_CPPUNIT_DIR='cppunit-1.10.0'
- SQUID_CPPUNIT_LA='$(top_builddir)/lib/cppunit-1.10.0/src/cppunit/libcppunit.la'
- SQUID_CPPUNIT_LIBS='$(SQUID_CPPUNIT_LA)'
- SQUID_CPPUNIT_INC='-I$(top_builddir)/lib/cppunit-1.10.0/include -I $(top_srcdir)/lib/cppunit-1.10.0/include'
+ echo cppunit does not apear to be installed. squid does not require this, but code testing with 'make check' will fail.
+ SQUID_CPPUNIT_DIR=''
+ SQUID_CPPUNIT_LA=''
+ SQUID_CPPUNIT_LIBS=''
+ SQUID_CPPUNIT_INC=''
fi
AC_ARG_WITH(cppunit-basedir,
[ --with-cppunit-basedir=/path/to/cppunit-base
Path where the cppunit headers are libraries are found
- for unit testing. (defaults to use internal copies -
- version 1.10.0)
+ for unit testing. ('make check' will fail without it)
],
[ if test -f $withval/include/cppunit/TestCase.h; then
echo Using cppunit includes from $withval
@@ -3305,6 +3305,5 @@
])
AC_CONFIG_SUBDIRS(lib/libTrie)
-AC_CONFIG_SUBDIRS(lib/cppunit-1.10.0)
AC_OUTPUT
Index: squid3/lib/Makefile.am
diff -u squid3/lib/Makefile.am:1.17 squid3/lib/Makefile.am:1.17.2.1
--- squid3/lib/Makefile.am:1.17 Sat Sep 2 06:52:12 2006
+++ squid3/lib/Makefile.am Fri Apr 13 03:41:49 2007
@@ -3,7 +3,7 @@
# $Id$
#
-DIST_SUBDIRS = libTrie cppunit-1.10.0
+DIST_SUBDIRS = libTrie
SUBDIRS= libTrie @SQUID_CPPUNIT_DIR@
install: all
Debug-Sections for Release
This is a cleanup of the published Debug Sections in Squid 3.0. Source files are only altered in the commented DEBUG: areas. All changes are to better prepare these sections for automatic processing. Some are to bring code sections marked with unknown (ie section ??) up to date by marking with the actual section that is already being used in the code. Have also updated the mk-debugs.sh patterns to include more inclusive filepath expansions actually used for locating and sorting these DEBUG: tags. Patch file generated Wed Apr 18 00:07:16 NZST 2007 from CVS branch ayjwork CVS base branch HEAD CVS repository: [EMAIL PROTECTED]:/cvsroot/squid CVS module: squid3 cvs -q rdiff -u -kk -r Z-ayjwork_merge_HEAD -r ayjwork squid3 Index: squid3/doc/debug-sections.txt diff -u squid3/doc/debug-sections.txt:1.6 squid3/doc/debug-sections.txt:1.6.22.1 --- squid3/doc/debug-sections.txt:1.6 Tue Jul 22 19:12:50 2003 +++ squid3/doc/debug-sections.txt Tue Apr 17 04:34:41 2007 @@ -1,13 +1,27 @@ + + * DEBUG: +none +none Common Removal Policy +none ETag parsing support +none FDE +none Generate squid.conf.default and cf_parser.h +none Linked list functions (deprecated) +none LRU Removal Policy +none Memory Allocation +none Refcount allocator section 0 Announcement Server -section 0 CGI Cache Manager section 0 Client Database section 0 Debug Routines section 0 DNS Resolver -section 0 WWW Client +section 0 Hash Tables +section 0 UFS Store Dump +section --External DISKD process implementation. +section 1 Main Loop section 1 Startup and Main Loop section 2 Unlink Daemon section 3 Configuration File Parsing section 4 Error Generation +section 5 Comms section 5 Socket Functions section 6 Disk I/O Routines section 7 Multicast @@ -24,40 +38,58 @@ section 18Cache Manager Statistics section 19Store Memory Primitives section 20Storage Manager +section 20Storage Manager Heap-based replacement +section 20Storage Manager Logging Functions +section 20Storage Manager MD5 Cache Keys +section 20Storage Manager Swapfile Metadata +section 20Storage Manager Swapfile Unpacker +section 20Storage Manager Swapin Functions +section 20Storage Manager Swapout Functions +section 20Store Rebuild Routines +section 20Swap Dir base object +section 21Integer functions section 21Misc Functions +section 21Time Functions section 22Refresh Calculation section 23URL Parsing -section 24WAIS Relay +section 23URL Scheme parsing section 25MIME Parsing section 26Secure Sockets Layer Proxy section 27Cache Announcer section 28Access Control section 29Authenticator +section 29Negotiate Authenticator +section 29NTLM Authenticator section 30Ident (RFC 931) section 31Hypertext Caching Protocol section 32Asynchronous Disk I/O section 33Client-side Routines section 34Dnsserver interface section 35FQDN Cache -section 36Cache Directory Cleanup section 37ICMP Routines section 38Network Measurement Database section 39Cache Array Routing Protocol -section 40User-Agent and Referer logging +section 40Referer Logging +section 40User-Agent Logging section 41Event Processing section 42ICMP Pinger program section 43AIOPS +section 43Windows AIOPS section 44Peer Selection Algorithm section 45Callback Data Registry section 46Access Log +section 47Store COSS Directory Routines section 47Store Directory Routines section 48Persistent Connections -section 49SNMP +section 49SNMP Interface +section 49SNMP support section 50Log file handling section 51Filedescriptor Functions section 52URN Parsing section 53AS Number handling +section 53Radix Tree data structure implementation section 54Interprocess Communication +section 54Windows Interprocess Communication section 55HTTP Header section 56HTTP Message Body section 57HTTP Status-line @@ -66,7 +98,7 @@ section 60Packer: A uniform interface to store-like modules section 61Redirector section 62Generic Histogram -section 63- +section 63Low Level Memory Pool Management section 64HTTP Range Header section 65HTTP Cache Control Header section 66HTTP Header Tools @@ -81,18 +113,25 @@ section 75WHOIS protocol section 76Internal Squid Object handling section 77Delay Pools -section 78DNS lookups -section 79Lowlevel store I/O -section 80WCCP -section 81Cache Store +section 78DNS lookups; interacts with lib/rfc1035.c +section 79Disk IO Routines +section 79Squid-side DISKD I/O functions. +section 79Squid-side Disk I/O functions. +section 79Storage Manager COSS Interface +section 79Storage Manager UFS Interface +section 80WCCP Support
Re: Debug-Sections for Release
Amos Jeffries wrote: This is a cleanup of the published Debug Sections in Squid 3.0. Source files are only altered in the commented DEBUG: areas. All changes are to better prepare these sections for automatic processing. Some are to bring code sections marked with unknown (ie section ??) up to date by marking with the actual section that is already being used in the code. Have also updated the mk-debugs.sh patterns to include more inclusive filepath expansions actually used for locating and sorting these DEBUG: tags. So has anyone with approval ability looked at auditing this for inclusion in HEAD yet? Amos
nocpp remainders not yet in HEAD
Henrik, The quick method of dropping files in the cppunit* directory missed a few. I just picked up on it now. Here is a current patch which takes care of the last remainders. Amos Patch file generated Wed Apr 25 20:55:01 NZST 2007 from CVS branch nocpp CVS base branch HEAD CVS repository: [EMAIL PROTECTED]:/cvsroot/squid CVS module: squid3 cvs -q rdiff -u -kk -r Z-nocpp_merge_HEAD -r nocpp squid3 Index: squid3/lib/cppunit-1.10.0/.cvsignore diff -u squid3/lib/cppunit-1.10.0/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,9 +0,0 @@ -Makefile -config.log -config.status -cppunit-config -libtool -autom4te.cache -configure.lineno -_configs.sed -Makefile.in Index: squid3/lib/cppunit-1.10.0/config/.cvsignore diff -u squid3/lib/cppunit-1.10.0/config/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/config/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/config/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/config/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,3 +0,0 @@ -config.h -stamp-h1 -Makefile.in Index: squid3/lib/cppunit-1.10.0/doc/.cvsignore diff -u squid3/lib/cppunit-1.10.0/doc/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/doc/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/doc/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/doc/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,3 +0,0 @@ -Doxyfile -Makefile -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/examples/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,2 +0,0 @@ -Makefile -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/ClockerPlugIn/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/ClockerPlugIn/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/ClockerPlugIn/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/ClockerPlugIn/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/examples/ClockerPlugIn/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,2 +0,0 @@ -Makefile -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/DumperPlugIn/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/DumperPlugIn/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/DumperPlugIn/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/DumperPlugIn/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/examples/DumperPlugIn/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,2 +0,0 @@ -Makefile -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/cppunittest/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/cppunittest/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/cppunittest/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/cppunittest/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/examples/cppunittest/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,5 +0,0 @@ -.deps -.libs -Makefile -cppunittestmain -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/hierarchy/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/hierarchy/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/hierarchy/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/hierarchy/.cvsignore:1.2 Sat Jun 10 16:36:01 2006 +++ squid3/lib/cppunit-1.10.0/examples/hierarchy/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,5 +0,0 @@ -.deps -.libs -Makefile -hierarchy -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/money/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/money/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/money/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/money/.cvsignore:1.2 Sat Jun 10 16:36:02 2006 +++ squid3/lib/cppunit-1.10.0/examples/money/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,5 +0,0 @@ -.deps -.libs -Makefile -MoneyApp -Makefile.in Index: squid3/lib/cppunit-1.10.0/examples/simple/.cvsignore diff -u squid3/lib/cppunit-1.10.0/examples/simple/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/examples/simple/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/examples/simple/.cvsignore:1.2 Sat Jun 10 16:36:02 2006 +++ squid3/lib/cppunit-1.10.0/examples/simple/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,5 +0,0 @@ -.deps -.libs -Makefile -simple -Makefile.in Index: squid3/lib/cppunit-1.10.0/include/.cvsignore diff -u squid3/lib/cppunit-1.10.0/include/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/include/.cvsignore:removed --- squid3/lib/cppunit-1.10.0/include/.cvsignore:1.2 Sat Jun 10 16:36:02 2006 +++ squid3/lib/cppunit-1.10.0/include/.cvsignore Wed Apr 25 01:55:06 2007 @@ -1,2 +0,0 @@ -Makefile -Makefile.in Index: squid3/lib/cppunit-1.10.0/include/cppunit/.cvsignore diff -u squid3/lib/cppunit-1.10.0/include/cppunit/.cvsignore:1.2 squid3/lib/cppunit-1.10.0/include/cppunit/.cvsignore:removed ---
Re: OS builds
Adrian Chadd wrote: On Wed, May 02, 2007, Robert Collins wrote: On Wed, 2007-05-02 at 14:05 +1200, [EMAIL PROTECTED] wrote: Without knowing a lot about the sourceforge compile farm it seems to me we could possible leverage that. Has anyone looked at it before? Not anymore: As of 2007-02-08, SourceForge.net Compile Farm service has been officially discontinued. https://sourceforge.net/forum/forum.php?forum_id=665363 Pick what hardware you'd like to play with and stuff can be arranegd. HP have some test machines you can get accounts on; the local computer club has access to reasonably diverse platforms and I can also add a machine or two to my server rack at home. Ah d'rats. Thanks Adrian, I was thinking ore in the line of something that could be added easily to the squid-cache.org site and running semi-automatically. Amos
Re: OS builds
Adrian Chadd wrote: On Wed, May 02, 2007, Amos Jeffries wrote: Pick what hardware you'd like to play with and stuff can be arranegd. HP have some test machines you can get accounts on; the local computer club has access to reasonably diverse platforms and I can also add a machine or two to my server rack at home. Ah d'rats. Thanks Adrian, I was thinking ore in the line of something that could be added easily to the squid-cache.org site and running semi-automatically. Well, name yer OSes/platforms. Virtualisation makes this task a bit less painful (ie, we could organise a 4gb-full desktop PC running VMWare and some PC-based OSes to start off. I've got space, power, cooling and bandwidth for it.) 'Name thy OS' would be those seen in squid-users where people are stuck with old 2.6 or 2.5 etc. The culprits seemed to be RedHat and CentOS. Amos
Re: new.squid-cache.org updated
Adrian Chadd wrote: Hi, I've updated new.squid-cache.org to be able to house non-templated content in random places. * templated content is now in files suffixed with .dyn * magic with rewrite rules to handle emulating index.dyn without forcing all pages down the PHP template assembly thingy * seems to do the right thing * new.squid-cache.org/Versions/ is now a symlink to the existing location; so we don't have to rebuild nightly snapshots in 1 location I'll commit this all to CVS soon. I'd appreciate some prodding to ensure everything (bar /Advisories) is working. I'd like to get the final kinks sorted out so we can put this site online as soon as possible. Thanks! Adrian Ah, you have changed the apache magic needed to display the site. Could you send me the new apache config needed please? For now all I can do is this patch which gets the main template back to xhtml standard compliance. Amos ? .apache ? standards.diff ? update.sh Index: cgi/header.inc === RCS file: /squid/www2/cgi/header.inc,v retrieving revision 1.6 diff -u -r1.6 header.inc --- cgi/header.inc 5 May 2007 16:19:30 - 1.6 +++ cgi/header.inc 6 May 2007 04:32:11 - @@ -25,7 +25,7 @@ lia href=/Support/ accesskey=3 title=Support/a/li lia href=/Intro/ accesskey=4 title=About/a/li lia href=/Support/contact.dyn accesskey=5 title=Contact/a/li - lia href=http://www.cafepress.com/squidproxy; accesskey=6 title= target=_blankShop/a/li + lia href=http://www.cafepress.com/squidproxy; accesskey=6 title= onclick=javascript: window.open(this.href); return false;Shop/a/li /ul /div /div
Re: new.squid-cache.org updated
Adrian Chadd wrote: On Sun, May 06, 2007, Amos Jeffries wrote: Ah, you have changed the apache magic needed to display the site. Could you send me the new apache config needed please? For now all I can do is this patch which gets the main template back to xhtml standard compliance. Check out http://www.creative.net.au/, the latest newspost has the magic required to have this work in apache-1. Adrian - lia href=http://www.cafepress.com/squidproxy; accesskey=6 title= target=_blankShop/a/li + lia href=http://www.cafepress.com/squidproxy; accesskey=6 title= onclick=javascript: window.open(this.href); return false;Shop/a/li Hm, when did the standard require Javascript to open the link in a new window? Adrian The xhtml standard technically does not permit specific any for of new window openings though the scripting standards do. 'target' has been invalid since html 4.0 strict and the first versions of xhtml. It was only really a 4.0 frameset transition item at that. Though the most common browsers still let it work in non-compliance mode. Any DOM script is usable, but I picked javascript as the most widely supported by browsers. Amos
Re: cvs commit: www2/content/Download binaries.dyn
Amos Jeffries wrote: amosjeffries2007/05/13 00:50:45 MDT Modified files: content/Download binaries.dyn Log: Updated release versions and dates. Corrected obvious bad dates for some releases based on info in squid-dev. Revision ChangesPath 1.2 +29 -33www2/content/Download/binaries.dyn gah. that should have been part of the standards commit. the actual dates one is comming next.
Re: cvs commit: squid3/src/ICAP ICAPConfig.cc ICAPConfig.h ....
Alex Rousskov wrote:
On Sat, 2007-05-19 at 02:38 +1200, Amos Jeffries wrote:
Oh dear I was sure y test build had --enable-icap.
Seems not though, very sorry.
Blame evil #ifdefs!
I was going to, then I looked at the code more.
I can't see _any_ in that file file.
I'm using the same g++ 4.1.2 on ubuntu.
I just purged all the build created files, bootstrapped, configured with
--enable-icap and rebuilt. (Am I missing an compile option for Xaction?)
It still has not compiled the parts of code that these errors occur in
and I cannot see any OS-specific directives in those files.
I have even added a predirective #error cvs conflict to stop the
compiles if it runs on that code ... and nothing appears.
Tried make check. make all, make install. nothing.
Index: ICAP/ICAPXaction.cc
===
RCS file: /squid/squid3/src/ICAP/ICAPXaction.cc,v
retrieving revision 1.16
diff -u -r1.16 ICAPXaction.cc
--- ICAP/ICAPXaction.cc 18 May 2007 06:41:30 - 1.16
+++ ICAP/ICAPXaction.cc 18 May 2007 08:35:46 -
snip
@@ -118,14 +118,10 @@
disableRetries(); // we only retry pconn failures
- ICAPXaction.cc
if (connection 0) {
connection = comm_open(SOCK_STREAM, 0, getOutgoingAddr(NULL), 0,
COMM_NONBLOCKING, s.uri.c_str());
-===
-connection = comm_open(SOCK_STREAM, 0, getOutgoingAddr(NULL), 0,
-COMM_NONBLOCKING, s.uri.buf());
- 1.15
+}
This change is more of a problem. To me it looks like the latter bit
that you have removed is part of a previous conflict that got overlooked.
ALEX: am I right in thinking the part _after_ the is the new piece
that should be kept after converting to use c_str()?
Yes, I think you are right.
Thank you,
Alex.
Okay, I am preparing an update for HEAD with the code corrected. But
would still like to find out what the evil #ifefs or their cousins are
doing for future reference.
Amos
Re: squid3_logdaemon work (round 1) complete, again!
Adrian Chadd wrote: On Fri, May 18, 2007, Adrian Chadd wrote: I've mostly copied over the older buffering thats in squid-2.6 and squid-3 logfile.c(c). Please review and provide feedback. I'm not going to try and improve the logfile writing in LogFileBlocking; I just want it to mimic what the current codebase does. Improvements can come later. I've been running it for half a day under a ~ 250 req/sec load. It seems to be doing fine. I'd like to commit this to squid-3 in the next couple of days. Please let me know if there are any objections. Adrian Now that you can I think you should take the opportunity to make it string-safe. Amos
Re: cvs commit: squid3/lib MemPool.cc
Alex Rousskov wrote:
rousskov2007/05/22 10:40:06 MDT
Modified files:
lib MemPool.cc
Log:
Bug #1966 fix: Use rounded String MemPool sizes in the hard-coded pool
config to avoid warnings that the configured pool size does not match the
actual size.
Revision ChangesPath
1.7 +7 -2 squid3/lib/MemPool.cc
Did you run make check before committing any of the days changes?
I'm getting 7 of 14 core unit tests fail. Two on MemPool.cc:240, the
rest on silent segfault.
Assertion failed: (aLabel != NULL aSize) at MemPool.cc:240
FAIL: tests/testString
/bin/bash: line 4: 23339 Aborted (core dumped) ${dir}$tst
FAIL: tests/testCacheManager
Assertion failed: (aLabel != NULL aSize) at MemPool.cc:240
FAIL: tests/testDiskIO
/bin/bash: line 4: 23351 Aborted (core dumped) ${dir}$tst
FAIL: tests/testEvent
/bin/bash: line 4: 23358 Aborted (core dumped) ${dir}$tst
FAIL: tests/testEventLoop
/bin/bash: line 4: 23371 Aborted (core dumped) ${dir}$tst
FAIL: tests/testHttpRequest
Amos
Re: SqString
Alex Rousskov wrote:
On Fri, 2007-05-25 at 20:39 +1200, Amos Jeffries wrote:
I have just been experimenting with a few options short of a full backout.
My initial idea of dropping the constructor drags the changes into areas
the initial patch didn't touch. No go there.
Yes, of course.
I have had some success dropping the str* overloading entirely.
Since the old code did not have those, I suspect the new code should
have complete success without them.
Ah, but it did. it just used a mix of some #define (2.x,3.0) and class
methods (3.0) and added upper case letters in the name.
Replacing it all with a single casting operator which explicitly forces
the magic casts to reverse from the expensive copy 'up' toward string
into a cheap cast 'down' to a c_str() call.
I am not sure why we need that operator now. On the IRC, you said that
it it seems to be the only working alternative to complete removal of
the change but I do not understand why.
Ok let me make this clearer. the #defines and methods above did
_explicit_ calls to c_str().
At this stage we actually have a three-way choice:
- backout
- add in c_str() every place its needed.
- add forced down-casting.
since the down-caster is simply an inline of c_str() the last two
options are the same, one with more work, one with less. Same actions.
With the downcasting its only 'implicit' in the nature of we cannot say
'its used there, and there, and there' without a full code audit.
Being part of string we can be certain that it will be be used only
where the old methods/defines were valid.
Moreover, if we must have implicit casts for the new API to work,
perhaps we should back out the entire API change for now because we are
not good at predicting the side-effects of such implicit casts.
It's one of the options.
Keep in mind this c_str() call being explicitly done all over squid at
present anyway. The only known risk is a buffer over- or under-run which
would occur anyway if the alternative explicit c_str() call was coded.
Here is my problem: The original API change changed some strcmp-like
calls. Some of the changes were incorrect. It is difficult for me to say
whether removing strcmp overloading and adding implicit char* casts will
expose all incorrect changes that we have not found yet. Unfortunately,
I doubt it.
What I would like to see is a diff against the pre-new-API code showing
that no calls got rewritten. Renaming a method is fine, but most other
changes should be postponed. The new patch you posted does not show me
how the main code is affected by the two changes combined, which is the
only thing that matters. Comparing with the current, broken code is not
very informative...
After you mentioned this on IRC I went back to my copy of the original
patch.
Attached is a pseudo-patch (has stuff elided so I don't expect it will
map cleanly) showing all the str* changes and re-arranging you are
worried about. All I have elided is the effected debug calls and
straight name mods; buf() to c_str() etc.
I have also left in the hash function-ptrs where I explicitly set the
namespace on strcmp so as not to clash their old values with anything new.
Amos
Patch file generated Fri May 18 18:34:42 NZST 2007 from
CVS branch ayjwork
CVS base branch HEAD
CVS repository: [EMAIL PROTECTED]:/cvsroot/squid
CVS module: squid3
cvs -q rdiff -u -kk -r Z-ayjwork_merge_HEAD -r ayjwork squid3
Index: squid3/src/ACLHTTPHeaderData.cc
diff -u squid3/src/ACLHTTPHeaderData.cc:1.3 squid3/src/ACLHTTPHeaderData.cc:1.2.14.3
--- squid3/src/ACLHTTPHeaderData.cc:1.3 Sat Apr 28 15:51:47 2007
+++ squid3/src/ACLHTTPHeaderData.cc Wed May 2 05:53:05 2007
@@ -87,14 +87,14 @@
char* t = strtokFile();
assert (t != NULL);
hdrName = t;
-hdrId = httpHeaderIdByNameDef(hdrName.buf(), strlen(hdrName.buf()));
+hdrId = httpHeaderIdByNameDef(hdrName.c_str(), hdrName.size());
regex_rule-parse();
}
Index: squid3/src/AuthUser.cc
diff -u squid3/src/AuthUser.cc:1.7 squid3/src/AuthUser.cc:1.3.4.3
--- squid3/src/AuthUser.cc:1.7 Wed May 9 01:51:04 2007
+++ squid3/src/AuthUser.cc Thu May 17 21:44:45 2007
@@ -143,7 +143,7 @@
if (!proxy_auth_username_cache) {
/* First time around, 7921 should be big enough */
proxy_auth_username_cache =
-hash_create((HASHCMP *) strcmp, 7921, hash_string);
+hash_create((HASHCMP *) std::strcmp, 7921, hash_string);
assert(proxy_auth_username_cache);
eventAdd(User Cache Maintenance, cacheCleanup, NULL, Config.authenticateGCInterval, 1);
}
Index: squid3/src/DelayTagged.cc
diff -u squid3/src/DelayTagged.cc:1.4 squid3/src/DelayTagged.cc:1.3.24.3
--- squid3/src/DelayTagged.cc:1.4 Sat Apr 28 15:51:48 2007
+++ squid3/src/DelayTagged.cc Mon May 7 03:32:21 2007
@@ -77,7 +77,7 @@
DelayTaggedCmp(DelayTaggedBucket::Pointer const left, DelayTaggedBucket::Pointer const right)
{
/* for rate limiting, case insensitive */
-return left
Re: SqString
Alex Rousskov wrote:
On Sat, 2007-05-26 at 04:08 +1200, Amos Jeffries wrote:
Alex Rousskov wrote:
On Fri, 2007-05-25 at 20:39 +1200, Amos Jeffries wrote:
I have just been experimenting with a few options short of a full backout.
My initial idea of dropping the constructor drags the changes into areas
the initial patch didn't touch. No go there.
Yes, of course.
I have had some success dropping the str* overloading entirely.
Since the old code did not have those, I suspect the new code should
have complete success without them.
Ah, but it did. it just used a mix of some #define (2.x,3.0) and class
methods (3.0) and added upper case letters in the name.
Class methods (versus overloaded globals) make a big difference when it
comes to silent type conversion (or lack of it). That is, in part, the
reason why the replacements in the new API where not functionally
equivalent to the original code.
True, But please understand, my reason for picking the overload path in
the first place was the amount of times that path was taken in the
original code.
Ok let me make this clearer. the #defines and methods above did
_explicit_ calls to c_str().
And you want to remove those methods because ...? Can you just rename
them to match the STL naming scheme?
See for yourself:
int
String::caseCmp (char const *aString) const
{
return strcasecmp(buf(), aString);
}
int
String::caseCmp (char const *aString, size_t count) const
{
return strncasecmp(buf(), aString, count);
}
Now spot the benefit in that duplicated code...
Secondly, they were a custom API to the old String:: that was used in
only a few places with the non-wrapped version still used over most of
the code.
And how is the above different from:
strncasecmp(str.buf(), aStr, count);
OR
strncasecmp((char*)str, aStr, count);
[ with (char*)str == str.buf() ].
Does the problem boil down to not
having an STL equivalent of some custom String methods?
No as the above shows they are _excatly_ equivalent.
All the problems I have seen are due to the magic of C++ choosing the
expensive method of conversion. If there are other bugs or posts than
1967 and 1970 that might effect my view on this please point them out.
What I would like to see is a diff against the pre-new-API code showing
that no calls got rewritten. Renaming a method is fine, but most other
changes should be postponed. The new patch you posted does not show me
how the main code is affected by the two changes combined, which is the
only thing that matters. Comparing with the current, broken code is not
very informative...
After you mentioned this on IRC I went back to my copy of the original
patch.
Attached is a pseudo-patch (has stuff elided so I don't expect it will
map cleanly) showing all the str* changes and re-arranging you are
worried about. All I have elided is the effected debug calls and
straight name mods; buf() to c_str() etc.
I have also left in the hash function-ptrs where I explicitly set the
namespace on strcmp so as not to clash their old values with anything new.
I am uncomfortable with the changes in the latest patch because they
change the working code. I did not do full analysis, but here are a few
specific places that worry me:
1) String::buf() did not return NULL for an empty but initialized
String. SqString::empty() does:
WTF? here is the original code:
char const *
String::buf() const
{
return buf_;
}
WHERE: buf_ is either NULL, or a char*. NO initialisation at all going
on there.
WHERE empty() returns (size()==0 || buf_ == NULL);
TRUE - IFF buf_ == char*
FALSE - IFF buf_ ==
I the case(s) you point out below the initial operation of buf()
provided a buffer-overrun in the edge case where buf_ != NULL, it
attempted to use it in a comparison or function call. **Even if it was
full of non-null garbage and the len_ was officially 0.***
-if (target sct-target.buf() !strcmp (target,
+if (target !sct-target.empty() !strcmp(target,
and
-else if (!target !sct-target.buf())
+else if (!target sct-target.empty())
and possibly elsewhere.
2) String::cmp does not just call strncmp so I cannot be sure this and
other similar changes are simple renaming:
No, cmp was mapped to compare, which performs the internal operation of
strncmp (doing its own internal check on size()).
Consider the two following code paths:
(cmp / compare)
- test len_ == 0- return 0 == param[0]
- test this* for being null/empty - return 0 == param[0]
- test para for being null or empty - return 0 == buf_[0]
- retrn output of strncmp(buf_, param, len_)
(strncmp)
- test strlen(lhs) == 0 - return 0 == rhs[0]
- test strlen(rhs) == 0 - return 0 == lhs[0]
- skip down for strlen places
OR rhs[p] != lhs[p]
- return lhs[p] - rhs[p];
So you can see the cmp(str, len) calls were
Re: squid-3 + SqString
Adrian Chadd wrote: Hi guys, Could we please make a decision about SqString? I'm leaning towards backing it out for now, releasing Squid-3, and then doing a phased introduction post squid-3 - starting with just accessor method changes.. Adrian I was waiting on Alex (or anyones) response to my last email, before going either way. partial backout: proposal (no responses) full backout: 3 for, 2 wavering with doubts, 1 abstention. Amos
Re: squid-3 + SqString
Henrik Nordstrom wrote: mån 2007-05-28 klockan 23:17 +0800 skrev Adrian Chadd: Personally, I think we've all learnt a bit from this experience, and there's a clearer path forward for tidying up the string (ab)uses going on in the codebase. I'm all for calling this a lesson learnt, backing it out now, then converting a few things to match the std::string interface - specifically, the accessor methods for the c buffer and the length. +1. Until SQUID_3_0 is branched from HEAD. Regards Henrik rolled back. Amos
Status of squid3-ipv6
I have had my build here running for several days now with no branch-specific bugs. Just a few from 3.0-HEAD and some of those are fixed by patches now included in the branch. As such, I am moving the branch status from Alpha-testing to Beta testing where its likely to remain until its merged to Squid 3.1. Amos Jeffries
Re: Introduction
Mark Nottingham wrote: Hi, I've been bugging Henrik and Adrian enough that it'd be useful to be on this list. I'm interested in helping with documentation, asking stupid questions and filing bugs. Cheers, Hi Mark, still interested? If so, what sort of documentation were you imagining? I have been doing some work in a 'docs' branch experimenting with doxygen to try and document the code and integrate it with the current incomplete 2.x (sgml/latex) user manual by Duane on the side. Most of it is going through and writing code comments in a specific format (easy) to describe the operations and effects of each function and class (not so easy). If you are interested in helping there I'll commit what I have so far back to CVS and make it a formal branch. pPS. Anyone else who wants to help in their spare time give a shout out too. Amos
Callbacks help please
I need some help getting my head around the callback system peoples. The situation: with IPv6 I have it looking up records and failover to lookup the A records at the DNS level. However IFF the DNS server responds to the indicating a CNAME. THe case is quite common that the CNAME points to an A not an . Thus the remote server does not add anything in the additional section. These additional records are what squid used to depend on for use when a CNAME arrived. I'm trying to add CNAME capability so the absence of additional A or will failover to looking them up. (This is presently showing me the same 'can't connect to X website' we get so many complaints about, maybe the same problem.) There the problem occurs: When ipcacheParse locates a CNAME, I tried calling ipcache_gethostbyname(...) to generate a loopback for the lookup. How do I suspend the current callback *leaving it as if never processed* for future re-handling? I have everything else going fine. It works after the initial page load once the 'bad IP' has expired from cache. Its just getting this to work on first load that the hiccup. Amos
Re: Callbacks help please
Amos Jeffries wrote: I need some help getting my head around the callback system peoples. The situation: with IPv6 I have it looking up records and failover to lookup the A records at the DNS level. However IFF the DNS server responds to the indicating a CNAME. THe case is quite common that the CNAME points to an A not an . Thus the remote server does not add anything in the additional section. These additional records are what squid used to depend on for use when a CNAME arrived. I'm trying to add CNAME capability so the absence of additional A or will failover to looking them up. (This is presently showing me the same 'can't connect to X website' we get so many complaints about, maybe the same problem.) There the problem occurs: When ipcacheParse locates a CNAME, I tried calling ipcache_gethostbyname(...) to generate a loopback for the lookup. How do I suspend the current callback *leaving it as if never processed* for future re-handling? I have everything else going fine. It works after the initial page load once the 'bad IP' has expired from cache. Its just getting this to work on first load that the hiccup. Amos Never mind. I think I have mucked up a suitable alternative. Amos
Re: XML Parser error
Wynand Swanepoel wrote: I am running Squid 2.6Stable13 on FreeBSD 4.9 as a transparent proxy using wccp. I get the following error when I do a search on the search engine www.aardvark.co.za: Warning: Sablotron error on line 1: XML parser error 3: no element found in /var/www/html/hyperseek_2003/cgi-bin/search/data/custom/google_adverts.inc on line 74 Warning: Sablotron error on line 1: XML parser error 3: no element found in /var/www/html/hyperseek_2003/cgi-bin/search/data/custom/search_plugin.inc on line 101 Any idea where I should look to solve it? One the web server providing www.aardvark.co.za? .inc is a commonly used extension for PHP files, also the page is full of non-HTML code that exposes company internal directory structures and software, just the kind of website I'd expect to see broken easily. Look to the webmaster for that kind of thing. Amos
Squid and SIGPIPE
I have been overlooking these for a while on the assumption that it was gdb catching something when it should have left it. I have just started wondering if this was in fact bad unexpected behaviour for the squid signals. Should I be looking into it? I run all my test builds under gdb with -ND to catch any big problems visibly instead of silent child restarts. It has been giving me Program received signal SIGPIPE, Broken pipe. 0xe410 in __kernel_vsyscall () #0 0xe410 in __kernel_vsyscall () #1 0xb7ce60b3 in __write_nocancel () from /lib/tls/i686/cmov/libc.so.6 #2 0x081128e2 in commHandleWrite (fd=14, data=0x8326884) at comm.cc:1963 #3 0x080876d5 in comm_select (msec=10) at comm_epoll.cc:309 #4 0x0811623e in CommSelectEngine::checkEvents (this=0xbfbb92fc, timeout=10) at comm.cc:2614 #5 0x0809ba81 in EventLoop::checkEngine (this=0xbfbb92bc, engine=0xbfbb92fc, primary=true) at EventLoop.cc:50 #6 0x0809bc1a in EventLoop::runOnce (this=0xbfbb92bc) at EventLoop.cc:124 #7 0x0809bd4a in EventLoop::run (this=0xbfbb92bc) at EventLoop.cc:100 #8 0x080d62ee in main (argc=4, argv=0xbfbb93b4) at main.cc:1310 or similar under some easily repeatable conditions.
Re: Squid and SIGPIPE
Adrian Chadd wrote: On Thu, Jul 05, 2007, Amos Jeffries wrote: I have been overlooking these for a while on the assumption that it was gdb catching something when it should have left it. I have just started wondering if this was in fact bad unexpected behaviour for the squid signals. Should I be looking into it? Nope! put this in ~/.gdbinit: handle all nostop noprint handle SIGSEGV stop handle SIGPIPE stop Thanks for that Adrian. I have tested it in my config now. handle SIGPIPE stop did the opposite of what is wanted, nostop print lets me know when they occur, but leaves squid running. I assume it was a typos or version defaults difference? Amos
default user
Hey guys, I find myself in need of a ./configure option to set the cache_effective_user other than 'nobody' on a per-dist basis, leaving the squid.conf setup as-is. Can someone please explain to me how the @HTTP_PORT@ etc get replaced in squid.conf / cf.data.pre I have tried modifying src/makefile.am and ./configure.in to no visible effect. Amos
Re: default user
Duane Wessels wrote:
On Tue, 10 Jul 2007, Amos Jeffries wrote:
Hey guys,
I find myself in need of a ./configure option to set the
cache_effective_user other than 'nobody' on a per-dist basis, leaving
the squid.conf setup as-is.
Can someone please explain to me how the @HTTP_PORT@ etc get
replaced in squid.conf / cf.data.pre
I have tried modifying src/makefile.am and ./configure.in to no
visible effect.
Calling AC_DEFINE (etc) causes the variables to be set for autoconf.
After ./configure runs, you should see them in config.status. eg:
s,@CACHE_HTTP_PORT@,3128,;t t
config.status does the substitutions from Makefile.in to Makefile.
for example:
$ sh config.status
...
config.status: creating src/Makefile
So in src/Makefile.in you have:
DEFAULT_HTTP_PORT = @CACHE_HTTP_PORT@
While in src/Makefile you have:
DEFAULT_HTTP_PORT = 3128
Our src/Makefile (but you must edit src/Makefile.am of course) has
a long-ish sed commands to do very similar substitutions on the
cf.data.pre file:
cf.data: cf.data.pre Makefile
sed \
[EMAIL PROTECTED]@%$(DEFAULT_HTTP_PORT)%g;\
[EMAIL PROTECTED]@%$(DEFAULT_ICP_PORT)%g;\
...
Thanks Duane,
What you have just written matched perfectly with my earlier reading
straight from the code. So I still can't see whats going wrong here.
So if you want to add your own macro goop you have to (at least):
- modify configure.in (if you want it to be set via ./configure)
yes:::
AC_ARG_WITH(cache-user,
[ --with-cache-user=USER System user account for squid permissions.
default: nobody],
[ if tets -z $withval; then
AC_DEFINE(cfg_CACHE_EFFECTIVE_USER, $withval, [change User for
squid to $withval])
else
AC_DEFINE(cfg_CACHE_EFFECTIVE_USER, nobody, [default User for squid])
fi
])
AC_SUBST(cfg_CACHE_EFFECTIVE_USER)
- modify src/Makefile.am
yes:::
[EMAIL PROTECTED]@%$(cfg_CACHE_EFFECTIVE_USER)%g;\
- run bootstrap.sh (Makefile.am - Makefile.in)
yes. it gives me ...
./src/Makefile.in: (looks OK)
cfg_CACHE_EFFECTIVE_USER = @cfg_CACHE_EFFECTIVE_USER@
...
[EMAIL PROTECTED]@%$(cfg_DEFAULT_CACHE_EFFECTIVE_USER)%g;\
./configure: (looks OK)
# Check whether --with-cache-user was given.
if test ${with_cache_user+set} = set; then
withval=$with_cache_user; if test -z $withval; then
cat confdefs.h \_ACEOF
#define cfg_CACHE_EFFECTIVE_USER $withval
_ACEOF
else
cat confdefs.h \_ACEOF
#define cfg_CACHE_EFFECTIVE_USER nobody
_ACEOF
fi
fi
- modify the sed command in the cf.data target of src/Makefile
?? was done in Makefile.am, bootstrap did it for src/Makefile automatically.
- modify cf.data.pre to use the new macro
yes:::
DEFAULT: @DEFAULT_CACHE_EFFECTIVE_USER@
- run ./configure and test it..
yes, gives me ...
./src/Makefile:
cfg_CACHE_EFFECTIVE_USER =
...
[EMAIL PROTECTED]@%$(cfg_CACHE_EFFECTIVE_USER)%g;\
And thats where I see the problem. Running make now will replace with
instead of nobody (default option wanted for this test)
Amos
Re: exposing ACL denies in cache.log?
Adrian Chadd wrote: Is there something nice in the logfile format for exposing the custom error page in access.log. I'd like to expose the custom errors chosen through the use of external_acl's so reporting scripts can extract info out of them later. Any ideas? src/access_log.cc:438 logformat_token_table has some possibilities. I would think something to do with the REPLY_LINE or REPLY_HEADER might fit seeing as the custom error is an internal reply. None appear to fit closely so it looks like its going to be up to you. Amos
dnsserver
Hmm, just handling these dnsserver typos I had. Can anyone give one good reason for the dnsserver external DNS helper now that squid has non-blocking event-driven internal DNS? Amos
Re: squid3-largeobj squid3/src HttpHdrRange.cc...
Tsantilas Christos wrote: Henrik Nordstrom wrote: On tis, 2007-07-31 at 21:09 +, chtsanti wrote: Please avoid casting unless needed. The compiler automatically promotes to larger types when needed, and will tell you if you try to do the reverse.. So stop casting things to (int64_t). OK. Sometimes I am using this typecasts just to note that here this operations must be done in 64bits. But yes this is why the comments exists ... Moreover casting maybe it is dangerous in the cases where the signess of an integer changes The exception is the upshifts if the value shifted may be large. But I think these should be changed to store bytes to begin with avoiding the problem. It's really more of a theoretical configuration limitation than a real limitation. It's very unlikely anyone would want to configure readahead_gap or quick_abort_min/max as large as 2 GB or more.. So make Config.readAheadGap and Config.quickAbort.min/max b_size_t instead of kb_size_t, and stop upshifting it to compare... Maybe even should be a b_int64_t. Probably we should even kill the kb_size_t type entirely, converting them all to b_int64_t. I did not change it now because, if I am not wrong, the kb_size_t type has the meaning that the users enters the configuration parameter in Kbytes (eg quick_abort_min 128 is 128Kbytes). But if it is a required I will do it Regards, Christos How about I first dig-up/rewrite some old code I wrote years ago that parses a value nX where n is some integer and X is one of the B/KB/MB/GB/...etc base strings? Then we can go about making all these annoying magic squid.conf values more human readable where they are supposed to be bandwidth amounts or storage sizes. And have the parser determine the minimum values base multiplier. Interest? Amos
Re: squid3-largeobj squid3/src HttpHdrRange.cc...
Amos Jeffries wrote: Alex Rousskov wrote: On Wed, 2007-08-01 at 20:18 +1200, Amos Jeffries wrote: How about I first dig-up/rewrite some old code I wrote years ago that parses a value nX where n is some integer and X is one of the B/KB/MB/GB/...etc base strings? That would be nice. I think Squid3 already has something similar for some time values, but it is not required. Same for some size values, as Henrik pointed out. Just digging through the code. Yes Henrik is right the byte-units parsing is already there. Although having said that, the current parser requires whitespace between the value and the units. I'm not certain that is a good thing. Amos
Re: squid3-largeobj squid3/src HttpHdrRange.cc...
Alex Rousskov wrote: On Wed, 2007-08-01 at 20:18 +1200, Amos Jeffries wrote: How about I first dig-up/rewrite some old code I wrote years ago that parses a value nX where n is some integer and X is one of the B/KB/MB/GB/...etc base strings? That would be nice. I think Squid3 already has something similar for some time values, but it is not required. Same for some size values, as Henrik pointed out. Just digging through the code. Yes Henrik is right the byte-units parsing is already there. Amos
mirror page.
Duane, When I fix the HTML compliance on those pages the background color you picked for the region headers starts to actually show up. Did you actually intend the bright/light blue? Amos
Attn: Peter J. Cherny
Peter, I have received your testing report on the IPv6 branch, but my replies to you seem to keep bouncing. I hope this gets to you one way or another. If you have an alternate contact I can send to please let me know, otherwise we should continue the debug process via squid-dev mailing list. Thanks for your testing. Amos Jeffries Squid Development Team - IPv6
WebDAV tester wanted.
One of the IPv6 squid testers has reported strange errors with a WebDAV enabled squid3-ipv6 build. Unfortunately he had no time available to track these down, and I don’t have WebDAV capability setup for use or testing. I am seeking someone who does have the time and setup to test WebDAV in squid under an IPv6 setup. I am willing to act as a free consultant in the IPv6 side of the setup in exchange for this testing if needed. Amos Jeffries Squid Development Team - IPv6
bug 2000 - patch
Here is the patch to obsolete StoreEntryStream from squid.
If any of you want to check it before I push it to HEAD next weekend.
It blocks out the StoreEntryStream class code and test cases, and
replaces all use of the stream with calls to storeAppendPrintf().
I'm not sure if it needs a Changelog entry since this is code new to 3.0
anyway.
Amos
Patch file generated Sun Aug 5 01:03:02 NZST 2007 from
CVS branch bug2000
CVS base branch HEAD
CVS repository: [EMAIL PROTECTED]:/cvsroot/squid
CVS module: squid3
cvs -q rdiff -u -kk -r Z-bug2000_merge_HEAD -r bug2000 squid3
Index: squid3/src/Makefile.am
diff -u squid3/src/Makefile.am:1.122 squid3/src/Makefile.am:1.122.10.1
--- squid3/src/Makefile.am:1.122 Tue Jun 19 15:51:04 2007
+++ squid3/src/Makefile.am Sat Aug 4 04:14:42 2007
@@ -2121,7 +2121,16 @@
# string needs mem.cc.
tests_testString_SOURCES= \
+ Mem.h \
mem.cc \
+ mem_node.h \
+ mem_node.cc \
+ stmem.cc \
+ MemObject.h \
+ tests/stub_MemObject.cc \
+ tests/stub_comm.cc \
+ tests/stub_store.cc \
+ cbdata.cc \
String.cc \
tests/testMain.cc \
tests/testString.cc \
Index: squid3/src/Mem.h
diff -u squid3/src/Mem.h:1.5 squid3/src/Mem.h:1.5.28.1
--- squid3/src/Mem.h:1.5 Sun May 28 17:50:18 2006
+++ squid3/src/Mem.h Sat Aug 4 04:14:42 2007
@@ -40,7 +40,7 @@
class CacheManager;
-#include iosfwd
+class StoreEntry;
class Mem
{
@@ -50,8 +50,8 @@
static void RegisterWithCacheManager(CacheManager manager);
static void Stats(StoreEntry *);
static void CleanIdlePools(void *unused);
-static void Report(std::ostream );
-static void PoolReport(const MemPoolStats * mp_st, const MemPoolMeter * AllMeter, std::ostream );
+static void Report(StoreEntry *sentry);
+static void PoolReport(const MemPoolStats * mp_st, const MemPoolMeter * AllMeter, StoreEntry *);
};
#endif /* SQUID_MEM */
Index: squid3/src/StoreEntryStream.h
diff -u squid3/src/StoreEntryStream.h:1.2 squid3/src/StoreEntryStream.h:1.2.36.2
--- squid3/src/StoreEntryStream.h:1.2 Fri May 5 18:50:22 2006
+++ squid3/src/StoreEntryStream.h Sat Aug 4 06:02:42 2007
@@ -30,15 +30,16 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111, USA.
*
*/
-
#ifndef SQUID_STORE_ENTRY_STREAM_H
#define SQUID_STORE_ENTRY_STREAM_H
+#if 0 /* obsoleting StoreEntryStream */
+
#include Store.h
#include ostream
-/*
+/**
* This class provides a streambuf interface for writing
* to StoreEntries. Typical use is via a StoreEntryStream
* rather than direct manipulation
@@ -51,9 +52,7 @@
StoreEntryStreamBuf(StoreEntry *anEntry) : anEntry(anEntry)
{
-anEntry-lock()
-
-;
+anEntry-lock();
anEntry-buffer();
}
@@ -63,7 +62,8 @@
}
protected:
-/* flush the current buffer and the character that is overflowing
+/**
+ * flush the current buffer and the character that is overflowing
* to the store entry.
*/
virtual char overflow(char aChar = traits_type::eof())
@@ -127,4 +127,5 @@
StoreEntryStreamBuf * rdbuf() const { return const_castStoreEntryStreamBuf *(_buffer); }
};
+#endif
#endif /* SQUID_STORE_ENTRY_STREAM_H */
Index: squid3/src/mem.cc
diff -u squid3/src/mem.cc:1.38 squid3/src/mem.cc:1.38.12.1
--- squid3/src/mem.cc:1.38 Tue May 22 09:52:04 2007
+++ squid3/src/mem.cc Sat Aug 4 04:14:42 2007
@@ -34,23 +34,17 @@
*/
#include squid.h
-
-#include iomanip
-#include ostream
-
#include event.h
#include CacheManager.h
#include Mem.h
#include memMeter.h
#include Store.h
-#include StoreEntryStream.h
#include MemBuf.h
#include SquidTime.h
/* module globals */
/* local prototypes */
-static void memStringStats(std::ostream );
/* module locals */
static MemAllocator *MemPools[MEM_MAX];
@@ -94,66 +88,58 @@
/* local routines */
static void
-memStringStats(std::ostream stream)
+memStringStats(StoreEntry *sentry)
{
int i;
int pooled_count = 0;
size_t pooled_volume = 0;
/* heading */
-stream String Pool\t Impact\t\t\n \t (%strings)\t (%volume)\n;
+storeAppendPrintf(sentry,String Pool\t Impact\t\t\n \t (%%strings)\t (%%volume)\n);
/* table body */
for (i = 0; i mem_str_pool_count; i++) {
const MemAllocator *pool = StrPools[i].pool;
const int plevel = pool-getMeter().inuse.level;
-stream std::setw(20) std::left pool-objectType();
-stream std::right \t xpercentInt(plevel, StrCountMeter.level);
-stream \t xpercentInt(plevel * pool-objectSize(), StrVolumeMeter.level) \n;
+storeAppendPrintf(sentry, %20s\t %10.0d\t %10.0d\n,
+ pool-objectType() ,
+ xpercentInt(plevel, StrCountMeter.level) ,
+ xpercentInt(plevel * pool-objectSize(), StrVolumeMeter.level) );
pooled_count += plevel;
pooled_volume += plevel * pool-objectSize();
}
/* malloc strings */
-stream std::setw(20) std::left Other Strings;
-
Re: cvs commit: www2/content/Download mirrors.dyn
Duane Wessels wrote: wessels 2007/08/04 15:03:00 MDT Modified files: content/Download mirrors.dyn Log: Fix Amos' bug Revision ChangesPath 1.12 +1 -1 www2/content/Download/mirrors.dyn Don't know what the bug was (background-color?) but I'm not seeing any DB records on that page anymore. Amos
Re: bug 2000 - patch
Robert Collins wrote: On Sun, 2007-08-05 at 19:39 +1200, Amos Jeffries wrote: Here is the patch to obsolete StoreEntryStream from squid. If any of you want to check it before I push it to HEAD next weekend. It blocks out the StoreEntryStream class code and test cases, and replaces all use of the stream with calls to storeAppendPrintf(). I'm not sure if it needs a Changelog entry since this is code new to 3.0 anyway. So why is this being removed? Building on recent testing versions of Debian with upcoming g++ 4.1.3 and 4.2.1 it fails the unit-tests. std::setw() on a string/char requires something that is not implemented in the StoreEntrystream* classes. My limited-knowledge tests haven't located exactly what it needs to be implemented now for it to work. The alternative to restore the builds, is to pull the MemObject stats display (only place the stream is ever actually used) back inline with the rest of squid to use storeEntryAppendPrintf(). And drop the stream until someone who knows what needs to be fixed can do so. Amos
Re: bug 2000 - patch
Robert Collins wrote: On Sun, 2007-08-05 at 22:03 +1200, Amos Jeffries wrote: Building on recent testing versions of Debian with upcoming g++ 4.1.3 and 4.2.1 it fails the unit-tests. std::setw() on a string/char requires something that is not implemented in the StoreEntrystream* classes. My limited-knowledge tests haven't located exactly what it needs to be implemented now for it to work. I can try to have a look at it next weekend. Theres some family stuff on, so no guarantees. Okay, if you want a go at it I'll wait a big longer. The alternative to restore the builds, is to pull the MemObject stats display (only place the stream is ever actually used) back inline with the rest of squid to use storeEntryAppendPrintf(). And drop the stream until someone who knows what needs to be fixed can do so. Well, I'd really rather we don't do that, because I was trying to get rid of the inability to do stream operations by introducing that. I presume bug 2000 has the compile error? yes. Have you looked at the g++ changelog to see if anything stands out? yes, there is no mention in either the Debian changelog or the gcc one of anything stream related back to before the versions that are known to have no probems.
Re: docs squid3/src/ICAP ICAPInitiate.h,1.2,1.2.10.1
Alex Rousskov wrote: On Tue, 2007-07-31 at 04:29 +, Amos Jeffries wrote: Update of cvs.devel.squid-cache.org:/cvsroot/squid/squid3/src/ICAP Modified Files: Tag: docs ICAPInitiate.h Log Message: Make doxygen happier with the source. Index: ICAPInitiate.h === RCS file: /cvsroot/squid/squid3/src/ICAP/ICAPInitiate.h,v retrieving revision 1.2 retrieving revision 1.2.10.1 diff -C2 -d -r1.2 -r1.2.10.1 *** ICAPInitiate.h 8 May 2007 16:51:27 - 1.2 --- ICAPInitiate.h 31 Jul 2007 04:29:06 - 1.2.10.1 *** *** 87,91 // communication with the initiator virtual void noteInitiatorAborted() = 0; ! AsyncCallWrapper(93,3, ICAPInitiate, noteInitiatorAborted) protected: --- 87,91 // communication with the initiator virtual void noteInitiatorAborted() = 0; ! AsyncCallWrapper(93,3, ICAPInitiate, noteInitiatorAborted) /**DOCS_NOSEMI*/ protected: Are there any alternative options to make doxygen happier without modifying the sources? Perhaps we can add a simple preprocessing script that would add the above crap before starting doxygen? As a followup on this Similar problem with: MEMPROXY_CLASS_INLINE(*) searching through the code for it showed a very inconsistent use of ';' after that one, some code has it others don't. Seeing as there is code with a semi in place that hasn't died on any compilers. I'm wondering if we should make that one at least semi'd universally through the code. Amos
Re: squid3-largeobj squid3/src HttpHdrRange.cc...
Tsantilas Christos wrote: At the end I just put quick_abort_min and quick_abort_max to its original behavior. They are storing values in kilobytes and if the units are missing kilobytes are the default. It is very easy to change it and also is easy to make parser to not require spaces between value and units. But a problem I am seeing is that at the begging we just want to simplify the code but now we are talking about correcting others thinks which are not critical or important. Any way, it is not difficult to change the above but the original goal was just to fix the bugs of squid3... I dont know... Regards, Christos Quite true, Christos. Thanks for the kick. This apparently has been the problem with 3.0 from early on. Far too many 'improvements' popping up. For myself I'm discussing and mostly just keeping an email folder of small ideas for future. When I get my chance at 3.1 a lot will begin to happen. I'm okay with putting the idea on backburner. Right now you are the only one to have put hand to code on this. We are only waiting on two blocker bugs now before 3.0 or PRE7. How soon? and which? I'm just so impatient to know :-) Amos Henrik Nordstrom wrote: On fre, 2007-08-03 at 00:35 +1200, Amos Jeffries wrote: Although having said that, the current parser requires whitespace between the value and the units. I'm not certain that is a good thing. Been like that for ever. Or at least as long as Squid has been parsing units.. i.e. Squid-1.1 or something like that. (1997 timeframe). I don't have a problem having the parser changed so that it also accepts specifications without the space however. Regards Henrik
Re: Squid3: schedule and naming
Hi Amos, Amos Jeffries wrote: I just spent the afternoon going over Christos full patch for squid3-largeobj and only found a few very minor things. Though I still don't know squid well enough to tell if there is anything missed out. Did you use it to download large files? Did you do some tests using it (just browsing ot something else)? I believe that now it is in good state... I only read the patch file. snips Not sure whether these are needed, but its not clear either way without a compile test. peer_digest.cc chunk @787 - (int) Needed here, else the compiler will produce a warning. store_dir.cc chunk @330 - (uint64_t ) Where is it? Ah, sorry. I must have copied the filename wrong. Patch file says its in src/store_digest.cc now line 247-254. Adding its the only change made, so would be nice to not do. Amos
Re: docs squid3/src AuthUser.h,1.5.2.1,1.5.2.2
I've reduced the doxygen errors down now to what seems like just a bunch of class/structs and typedefs which define different names, but are used interchangably. I've altered this one so the two method declarations actually use the same name for their param type. Is there a reason for so many this type of typedef being kept in Squid3 (Squid2 I understand)? If not, I'll mark all the ones I find as deprecated. Update of cvs.devel.squid-cache.org:/cvsroot/squid/squid3/src Modified Files: Tag: docs AuthUser.h Log Message: Give method declaration the same types as its *.cc counterpart. Index: AuthUser.h === RCS file: /cvsroot/squid/squid3/src/AuthUser.h,v retrieving revision 1.5.2.1 retrieving revision 1.5.2.2 diff -C2 -d -r1.5.2.1 -r1.5.2.2 *** AuthUser.h6 Aug 2007 05:18:23 - 1.5.2.1 --- AuthUser.h10 Aug 2007 03:45:46 - 1.5.2.2 *** *** 74,78 static void CachedACLsReset(); ! void absorb(auth_user_t *from); virtual ~AuthUser (); _SQUID_INLINE_ char const *username() const; --- 74,78 static void CachedACLsReset(); ! void absorb(AuthUser *from); virtual ~AuthUser (); _SQUID_INLINE_ char const *username() const;
Re: squid3-ipv6 squid3/src HttpHeader.cc,1.18.4.8,1.18.4.9 cache_manager.cc,1.7.8.3,1.7.8.4 client_db.cc,1.5.4.14,1.5.4.15 event.cc,1.5.8.5,1.5.8.6
Any objections to these going into 3-HEAD?
Baring the seconds precision drop to 3dp (aka ms), its all whitespace.
Modified Files:
Tag: squid3-ipv6
HttpHeader.cc cache_manager.cc client_db.cc event.cc
Log Message:
Some Formatting tweaks to squidclient output. Much more readable.
Index: event.cc
===
RCS file: /cvsroot/squid/squid3/src/event.cc,v
retrieving revision 1.5.8.5
retrieving revision 1.5.8.6
diff -C2 -d -r1.5.8.5 -r1.5.8.6
*** event.cc31 Jul 2007 06:40:27 - 1.5.8.5
--- event.cc10 Aug 2007 10:41:09 - 1.5.8.6
***
*** 269,273
storeAppendPrintf(sentry, Last event to run: %s\n\n,
last_event_ran);
! storeAppendPrintf(sentry, %s\t%s\t%s\t%s\n,
Operation,
Next Execution,
--- 269,273
storeAppendPrintf(sentry, Last event to run: %s\n\n,
last_event_ran);
! storeAppendPrintf(sentry, %-25s\t%-15s\t%s\t%s\n,
Operation,
Next Execution,
***
*** 276,280
while (e != NULL) {
! storeAppendPrintf(sentry, %s\t%f seconds\t%d\t%s\n,
e-name, e-when ? e-when - current_dtime : 0,
e-weight,
(e-arg e-cbdata) ? cbdataReferenceValid(e-arg) ? yes : no :
N/A);
--- 276,280
while (e != NULL) {
! storeAppendPrintf(sentry, %-25s\t%0.3f sec\t%5d\t %s\n,
e-name, e-when ? e-when - current_dtime : 0,
e-weight,
(e-arg e-cbdata) ? cbdataReferenceValid(e-arg) ? yes : no :
N/A);
Index: HttpHeader.cc
===
RCS file: /cvsroot/squid/squid3/src/HttpHeader.cc,v
retrieving revision 1.18.4.8
retrieving revision 1.18.4.9
diff -C2 -d -r1.18.4.8 -r1.18.4.9
*** HttpHeader.cc 30 May 2007 05:06:06 - 1.18.4.8
--- HttpHeader.cc 10 Aug 2007 10:41:09 - 1.18.4.9
***
*** 1607,1616
storeAppendPrintf(e, \nHttp Fields Stats (replies and requests)\n);
! storeAppendPrintf(e, %2s\t %-20s\t %5s\t %6s\t %6s\n,
id, name, #alive, %err, %repeat);
for (ht = (http_hdr_type)0; ht HDR_ENUM_END; ++ht) {
HttpHeaderFieldInfo *f = Headers + ht;
! storeAppendPrintf(e, %2d\t %-20s\t %5d\t %6.3f\t %6.3f\n,
f-id, f-name.buf(), f-stat.aliveCount,
xpercent(f-stat.errCount, f-stat.parsCount),
--- 1607,1616
storeAppendPrintf(e, \nHttp Fields Stats (replies and requests)\n);
! storeAppendPrintf(e, %2s\t %-25s\t %5s\t %6s\t %6s\n,
id, name, #alive, %err, %repeat);
for (ht = (http_hdr_type)0; ht HDR_ENUM_END; ++ht) {
HttpHeaderFieldInfo *f = Headers + ht;
! storeAppendPrintf(e, %2d\t %-25s\t %5d\t %6.3f\t %6.3f\n,
f-id, f-name.buf(), f-stat.aliveCount,
xpercent(f-stat.errCount, f-stat.parsCount),
Index: client_db.cc
===
RCS file: /cvsroot/squid/squid3/src/client_db.cc,v
retrieving revision 1.5.4.14
retrieving revision 1.5.4.15
diff -C2 -d -r1.5.4.14 -r1.5.4.15
*** client_db.cc3 Jun 2007 12:58:32 - 1.5.4.14
--- client_db.cc10 Aug 2007 10:41:09 - 1.5.4.15
***
*** 249,256
while ((c = (ClientInfo *) hash_next(client_table))) {
storeAppendPrintf(sentry, Address: %s\n, hashKeyStr(c-hash));
! storeAppendPrintf(sentry, Name: %s\n, fqdnFromAddr(c-addr));
storeAppendPrintf(sentry, Currently established connections: %d\n,
c-n_established);
! storeAppendPrintf(sentry, ICP Requests %d\n,
c-Icp.n_requests);
--- 249,256
while ((c = (ClientInfo *) hash_next(client_table))) {
storeAppendPrintf(sentry, Address: %s\n, hashKeyStr(c-hash));
! storeAppendPrintf(sentry, Name:%s\n, fqdnFromAddr(c-addr));
storeAppendPrintf(sentry, Currently established connections: %d\n,
c-n_established);
! storeAppendPrintf(sentry, ICP Requests %d\n,
c-Icp.n_requests);
***
*** 294,300
storeAppendPrintf(sentry, TOTALS\n);
! storeAppendPrintf(sentry, ICP : %d Queries, %d Hits (%3d%%)\n,
icp_total, icp_hits, percent(icp_hits, icp_total));
! storeAppendPrintf(sentry, HTTP: %d Requests, %d Hits (%3d%%)\n,
http_total, http_hits, percent(http_hits, http_total));
}
--- 294,300
storeAppendPrintf(sentry, TOTALS\n);
! storeAppendPrintf(sentry, ICP : %7d Queries, %7d Hits (%3d%%)\n,
icp_total, icp_hits,
Re: Squid 3 build error using Visual Studio
Serassio Guido wrote: Hi, Trying to build the current Squid 3 source using Visual Studio 2005 on Windows, I get the following error. Any C++ suggestion ? hehe, StoreEntryStream, again. Alex just fixed the last bug in there for me, he may have better grasp of it at the moment. Here is someone else with the same problem and an answer (maybe): http://www.codecomments.com/archive292-2005-2-396222.html Compiling... mem.cc c:\work\nt-3.0\src\StoreEntryStream.h(119) : error C2512: 'std::basic_ostream_Elem,_Traits' : no appropriate default constructor available with [ _Elem=char, _Traits=std::char_traitschar ] Regards Guido - Guido Serassio Acme Consulting S.r.l. - Microsoft Certified Partner Via Lucia Savarino, 1 10098 - Rivoli (TO) - ITALY Tel. : +39.011.9530135 Fax. : +39.011.9781115 Email: [EMAIL PROTECTED] WWW: http://www.acmeconsulting.it/
Re: cvs commit: squid3/src BodyPipe.cc BodyPipe.h
rousskov2007/08/13 10:48:20 MDT snip The new code also keeps track of the number of outstanding events and skips that number if the consumer leaves. TODO: when AscyncCall support is improved, should we just schedule calls directly to consumer? It could be a much cleaner solution than counting pending calls and skipping them when needed. You mean like client-streams was meant to be (I think, after only a few days reading the code and a bias from work on other real-time applications). Where only the tail gets event calls and read events pull as much buffer data through the chain as possible? Amos
StoreIOBuffer field types
As part of the checks to find possible causes of my servers falling over
I went to see if coverity had any more issues located.
One of them was unsafe use of a variable typed 'size_t' setting up
StoreIOBUffer local variable
The constructor using it is:
StoreIOBuffer(size_t aLength, int64_t anOffset, char *someData) :
length (aLength), offset (anOffset), data (someData)
{
flags.error = 0;
}
where length/aLength are the size_t in question.
I had thought it the length of the buffer and offset in the buffer. But
seeing the offset is now int64, is it the lengths of the file being
buffered? In which case it should be a 64-bit too I think.
Or was I right about being buffer length? in which case it should be an
unsigned something? which prevents this warning showing.
Amos
Re: cvs commit: squid3/src ftp.cc
Alex Rousskov wrote: On Mon, 2007-08-13 at 12:01 +0200, Guido Serassio wrote: Now fixed. If you have time, it may be better to replace a virtual FtpStateData::haveControlChannel(char*) into a static FtpStateData::HaveControlChannel(FtpStateData*, char*) and let it check the ftpState pointer itself. This will avoid more code repetition and may even work faster. Whats with the uppercasing? the method is not a type. Is this a new convention not mentioned in the wiki/devel-faq? The FTP code is all a it messy still with its partial conversion and overlaps with other protocols code. When the whole area is cleaned up several of those functions should be inline. I was wondering about an *this check inside (does away with the extraneous parameter), but I'm not certain enough to code it for 3.0. I would also suggest to use a different method name since the code assumes it is only called to detect attempts to call a method when a control channel is open. Also, doneWithServer() actually checks for the presence of both data and control channels. Ah, quite right. doneWithServer() is only one of the cases that method should handle. I'll be adding the other soon. Amos
Re: cvs commit: squid3/src ftp.cc
On Tue, 2007-08-14 at 17:58 +0200, Henrik Nordstrom wrote: On tis, 2007-08-14 at 23:01 +1200, Amos Jeffries wrote: Alex Rousskov wrote: On Mon, 2007-08-13 at 12:01 +0200, Guido Serassio wrote: Now fixed. If you have time, it may be better to replace a virtual FtpStateData::haveControlChannel(char*) into a static FtpStateData::HaveControlChannel(FtpStateData*, char*) and let it check the ftpState pointer itself. This will avoid more code repetition and may even work faster. Whats with the uppercasing? the method is not a type. Is this a new convention not mentioned in the wiki/devel-faq? Hmm.. it should be in the style guide somewhere. http://www.squid-cache.org/Devel/squid-3-style.txt What I based the following on. Alex posted an upper-cased internal-but-public method (NOT data member). object local methods and variables start with a lowercase, global variables and static class members with an upper case.
Re: IPv6 support and Windows
Hi Amos, Not good news here ... :-( We have a problem: You have used getaddrinfo() and freeaddrinfo() functions, but on Windows they are available only starting from Windows XP. This means that your code, even if compiled without IPv6 support, cannot run on Windows 2000. I'm also not sure if getaddrinfo() and freeaddrinfo() functions are available on all systems, someone know something about ? We have already tested Linux 2.6+ and FreeBSD 4.5+ (4.3 _says_ it has them) Not sure about Solaris, OS2, or Darwin yet. I have found two interesting substitute here: AYJ: pulled these out of order due to my response sequence http://doxygen.postgresql.org/getaddrinfo_8c-source.html I'm not sure if the licensing of the implementation coming from the PostgreSQL allow its usage in Squid. PostGres looks good until I audit the code. These are both killer bugs for us: Bugs: - only one addrinfo is set even though hintp is NULL or ai_socktype is 0 - AI_CANONNAME is not supported. http://www.koders.com/c/fid11C8C256BE9D1C263E0725F657383F62F24759F5.aspx?s=md5 The very first line says it's a (partial) implementation which makes me think there are edge cases missing, etc. Though a quick check of the code I couldn't see any obvious problems or notes about killer bugs. I'm sure there are more re-writes around, and sure enough Google codesearch find me a clean fetchmail implementation of JUST getaddrinfo(), freeaddrinfo() and gai_strerror(). ftp://ftp.sunfreeware.com/pub/freeware/SOURCES/fetchmail-6.3.6.tar.gzcs_f=fetchmail-6.3.6/libesmtp/getaddrinfo.c#a0 That is probably better for us being more complete in the IPv4-area. The crunch now will be whether getnameinfo() is also that unsupported in NT/2k ? AND its already under GNU license to FSF. Amos
Re: cvs commit: squid3/src BodyPipe.cc BodyPipe.h
On Tue, 2007-08-14 at 09:54 +1200, Amos Jeffries wrote: rousskov2007/08/13 10:48:20 MDT snip The new code also keeps track of the number of outstanding events and skips that number if the consumer leaves. TODO: when AscyncCall support is improved, should we just schedule calls directly to consumer? It could be a much cleaner solution than counting pending calls and skipping them when needed. You mean like client-streams was meant to be (I think, after only a few days reading the code and a bias from work on other real-time applications). Where only the tail gets event calls and read events pull as much buffer data through the chain as possible? No, nothing that complex! Chains are evil :-). BodyPipe is not a chain. It is a dumb connection/transport between two independent but cooperating jobs: the body consumer and the body producer. Currently, BodyPipe asynchronously calls itself so that it can synchronously call the consumer or the producer with the right parameters when new data arrives, etc. Aha! gotcha. I hope this is all commented in the code. I haven't got the docs branch around to ICAP yet. This creates a problem if the consumer is not there when (or changes after) the asynchronous call is scheduled: The new consumer may get only some of the scheduled calls, and things will not work well. When asynchronous calls are no longer hacked on top of a single-parameter Event, the above indirection will not longer be needed. BodyPipe will be able to asynchronously call the consumer or the producer with the right parameters. At that time, the committed fix, and 80% of current call wrapping functions will no longer be needed. Alex. Thanks Amos
Re: cvs commit: squid3/src ftp.cc
On tis, 2007-08-14 at 23:01 +1200, Amos Jeffries wrote: Alex Rousskov wrote: On Mon, 2007-08-13 at 12:01 +0200, Guido Serassio wrote: Now fixed. If you have time, it may be better to replace a virtual FtpStateData::haveControlChannel(char*) into a static FtpStateData::HaveControlChannel(FtpStateData*, char*) and let it check the ftpState pointer itself. This will avoid more code repetition and may even work faster. Whats with the uppercasing? the method is not a type. Is this a new convention not mentioned in the wiki/devel-faq? Hmm.. it should be in the style guide somewhere. object local methods and variables start with a lowercase, global variables and static class members with an upper case. The FTP code is all a it messy still with its partial conversion and overlaps with other protocols code. Can you be specific on what you think need further conversion or overlaps? Ah, sorry, I said that without taking a good overview look at it (yay doxygen). At the line-level it seems to have a lot of global stuff. I must have gotten FTP confused with the HttpRequest usages. The main gist of my thinking already exists in the form of ServerStateData for FTP at least. Alex made some comments earlier about use of temporary buffers in several of the protocol automata. And here is a short list of currently global functions which I think should be provided either by the main body of squid (non-FTP specific) or part of the classes private methods. is_month() escapeAC() ftpOpenListenSocket() ftpFail() ftpUrlWith2f() - should be part of URL class when its ready ftpTrySlashHack() (maybe others, I haven't looked closely at each one yet) maybe as ftpListParts open class: ftpListPartsFree() ftpListPartsParse() Ah, quite right. doneWithServer() is only one of the cases that method should handle. I'll be adding the other soon. And quite likely the whole function is not needed any more after the state fixes, but it's a reasonable safeguard so I don't object it's addition. truely. Amos
Re: Squid 3 build error using Visual Studio
On tis, 2007-08-14 at 14:10 +0300, Tsantilas Christos wrote:
The major problem you may have is that c++ compilers some times return
error messages which looks completely unrelated with the real problem.
Or quite related but where it's impossible to grok what is related or
how...
http://www.henriknordstrom.net/code/loreley_warning.html
(I had similar problem today morning trying to understand what the hell
means the undefined reference to vtable for .)
My experience: Missing object file implementing the class, only having
header references. But if I hadn't known already that the vtable is what
the object internal table of virtual methods is called I would not have
understood that error much at all..
But I think as someones learns the compilers and tools which his is
using these problems disappeared
And then the tools gets upgraded and you have to learn new errors..
The problem with C++ compared to C is complexity. C++ is a fairly
complex language with many levels of abstraction and a lot of hidden
magics which needs to be understood apart from the syntax itself. This
makes a very steep learningcurve when trying to understand existing
code.
Tools like doxygen helps a lot to decipher unknown code. Without them
reading and understanding complex C++ code is a nightmare.
But this is also where the powers of C++ lies. The abstraction levels do
allows for clean code to be done easier than in for example C.
The Squid-3 code is currently a bit messy with 4-5 differen generations
of coding style. But gradually things do get cleaned up. And I am also
looking forward to the docs project getting merged which is a
significant step along that path imho.
:-) Thanks. Even if it was your idea in the first place.
Guido (or anyone);
if you want to use the docs project to understand any code its public at
http://squid.treenet.co.nz/Doc/Code/
Two notes though for now:
- there is presently some breakage caused by those semi-colon-free macros:
void orphan() {}
MACRO(x)
void fooBarInvisible() { orphan(); }
- MACRO will screw up so docs will not include fooBarInvisible() and
details for orphan() will say its never referenced.
So for the present, unless a function has a docs comment or is linked from
the module page directly it needs checking for use manually.
Overall though its good now for a quick check of inheritance details and
what belongs where.
Amos
Re: IPv6 support and Windows
Hi Amos, Not good news here ... :-( We have a problem: You have used getaddrinfo() and freeaddrinfo() functions, but on Windows they are available only starting from Windows XP. This means that your code, even if compiled without IPv6 support, cannot run on Windows 2000. I'm also not sure if getaddrinfo() and freeaddrinfo() functions are available on all systems, someone know something about ? We have already tested Linux 2.6+ and FreeBSD 4.5+ (4.3 _says_ it has them) Not sure about Solaris, OS2, or Darwin yet. I have just imported and properly wrapped the libesmtp getaddrinfo and supporting implementation from fetchmail 6.3.8. I have done some simple configure and compile tests here. Its time to pass it back to you Guido for the win32 test. I'm kind of expecting another set of errors off these. It's just a matter of which ones. Amos
Re: squid-3 ?
How long before the next pre or release candidate of squid-3 ? Now that both of the blocker bugs are closed. I submitted the last of the compile errors I can find today, it now builds for both make check and make install. Baring any of you others finding another I think we have reached step 2 of the release process again today. The fortnight clock should be counting down to a 1 Sept release. Amos
Re: squid-3 ?
On Wed, 2007-08-15 at 09:58 +0200, Guido Serassio wrote: Currently Squid is broken on all platforms without strtoll(), HP Tru64 is one. I'm working on it, and I'm expecting to fix the problem and run some tests on other platforms (Irix, NetBSD and OpenBSD) within the incoming weekend. Sounds good, thank you. I think that we must wait for two weeks to see if the latest large-objects changes are reasonably stable. I think we should release an RC1 snapshot now. I was a bit undecided yesterday. I kind of want to wait a short while to see if large-obj is stable enough, but not enough to speak up. But with strtoll as Guido said, I think its clearly too early for an RC. PRE7 at the end of this week I would agree with if there are no other majors discovered before then. As I see it a PRE is considered 'mostly usable'. But for a full RC I think we should be certain it actually is good quality (for some new definition of good ;-), I suppose). Amos
Re: Code documentation trends
On Wed, 2007-08-15 at 02:32 +, Amos Jeffries wrote:
snip
*** MemPool.h23 Jun 2007 01:43:43 - 1.10.10.1
--- MemPool.h15 Aug 2007 02:32:57 - 1.10.10.2
***
*** 120,124
};
! /* Support late binding of pool type for allocator agnostic classes */
class MemAllocatorProxy
{
--- 120,126
};
! /**
! * Support late binding of pool type for allocator agnostic classes
! */
Did you add the two extra lines to make doxygen happy? Is there a less
invasive way to accomplish the same? (Please do not interpret these
questions as criticism, I am just trying to understand the motivation
and doxygen limitations).
No, a block doc-comment only requires the /** at the beginning ,
everything else is arrangement and visual effect.
The extra lines like that are usually because I expect a full description
to go in there. Will be happening to all API stuff in .h, much less so on
Internals.
That one is also going to need a few doxygen tags like \ingroup and a
less-cryptic description. I didn't quite get time last night to go through
the Mem* component pieces and link them into a module.
Time for part 2 of the auto-docs lessons:
What I have settled on a week ago is a 'looks nice and can be read easily'
style, which requires people only to know that doc comments start with /**
and have * at the start of each line inside the comment.
Lines that start with \foo are special to the auto-docs and only to be
altered if understood (doxygen help is online for reference).
Do make any special formatting such as lists, paragraphing, large titles,
example code snippets require the doxygen commands. Formatting by
whitespace indentation (ie- ) will no longer work.
What I'm hoping for is to have a paragraph for each class about where its
intended to be used etc.
Objects where I only have enough details to say this is part of X module,
are going in as /// ... for now (which I would like NOT to have in the
final code pushed to HEAD, its quite ugly to see /// \FooBarInternals,
but it works).
Part 3 (module/component names and organisation, will have to come later
when I have a more certain list).
class MemAllocatorProxy
{
***
*** 138,143
mutable MemAllocator *theAllocator;
};
/* help for classes */
! /* Put this in the class */
#define MEMPROXY_CLASS(CLASS) \
/* TODO change syntax to allow moving into .cci files */ \
--- 140,154
mutable MemAllocator *theAllocator;
};
+
/* help for classes */
! /**
! * Put this in the class
! \code
!class foo
!{
! MEMPROXY_CLASS(foo);
!}
! \endcode
! */
#define MEMPROXY_CLASS(CLASS) \
/* TODO change syntax to allow moving into .cci files */ \
Does this change work around some other doxygen limitation? Or are you
trying to provide a better illustration of what Put this in the class
means?
I was just copying the 'how to use' example as-is from the old SGML file.
\code and \endcode just tell doxygen its a piece of example code to be
displayed. Theres lots of little bits still need doing for the Mem
component.
You can see what it looks like at:
http://squid.treenet.co.nz/Doc/Code/MemPool_8h.dyn#2eab9a6fa490ed8426a1800e0ec4
I think maybe it's the text-description not making it clear its a
code-example. That description needs fixing.
None of these changes are a big deal, of course. I am just worried about
the general trend of hiding code behind overly verbose documentation...
Does doxygen support some kind of \include mechanism where verbose
documentation can be moved away from the code while still being linked
to specific code pieces? Detailed documentation is great, but if it
makes the actual code difficult to see, it is doing more harm than good,
IMO.
Yes, but that defeats the entire purpose of this project. Having the
documentation at the point of code where it will get noticed for change by
any average-joe editors altering that code. I don't expect it to be huge.
I've had a hard time converting the SGML because it references only 2.5
examples and function names, describing how they behave in 2.5, and had to
check each function operation and change some of it in migration.
***
*** 146,150
static inline MemAllocatorProxy Pool()
! /* put this in the class .h, or .cci as appropriate */
#define MEMPROXY_CLASS_INLINE(CLASS) \
MemAllocatorProxy CLASS::Pool() \
--- 157,171
static inline MemAllocatorProxy Pool()
! /**
! * Put this in the class .h, or .cci as appropriate
! \code
!class foo
!{
! MEMPROXY_CLASS(foo);
!}
!
!MEMPROXY_CLASS_INLINE(foo);
! \endcode
! */
#define MEMPROXY_CLASS_INLINE(CLASS) \
MemAllocatorProxy CLASS::Pool() \
Like this new documentation bug, for example. :-)
You see the examples aren't always right.
Thank you,
Alex.
Welcome.
Amos
Re: Documenting non-public members
I noticed this in a recent squid3/doc/Programming-Guide commit message:
\section Coding Code Conventions
...
Now that documentation is generated automatically from the
sources
some common comment conventions need to be adopted.
\subsectionAPI vs Internal Component Commenting
...
First among these is a definition seperation between
component API
and Internal operations. API functions and objects should
always be
commented and in the *.h file for the component. Internal
logics and
objscte should be commented in the *.cc file wherever they are
defined.
I am not sure this is a good guideline for C++ because class data
members should be briefly documented where they are declared (which is
in the .h file), even if they are not a part of the API.
This is perhapse due more discussion amongst us all.
My view so far is:
Most of the squid code is commented for programmers in the .cc where the
code is written. Leaving that as-is where possible, this is actually more
convenient for most functions where editors can change code and description
in one place.
BUT, API stuff MUST be written in the .h since there is no guarantee of
the user(s) having time to find and read the .cc (or it even being
available).
If we all agree that .h is the place for class internal docs, I'll accept
it. But do consider the convenience factor in mind for all future
maintainers when deciding.
There is simply
no other good place to document them because there is no corresponding
definition.
There is the place where the {...} code itself lives. Which is the written
intention there. I picked this because that is the current documenting
style for squid.
The edge case here is virtual functions defined as virtual foo Bar(){} in
the .h file itself.
Those needs a brief comment saying class X defined as empty or do-nothing.
(doxygen will, as currently set, pull the parent class docs for a method
into the childs. Thus the parent has full description, child has mod
notes)
A short description of methods should also be appropriate (and is often
very helpful) in .h file, even if they are not a part of public API.
True, we can do this regardless of where the detail description is. ANY
auto-docs comment in .h or .cc(i) just ahead of a object-name are
catenated for the final product.
$0.02,
and $0.01 bank fees ;-)
Amos
Re: IPv6 support and Windows
Hi Amos, At 07.25 15/08/2007, Amos Jeffries wrote: Hi Amos, Not good news here ... :-( We have a problem: You have used getaddrinfo() and freeaddrinfo() functions, but on Windows they are available only starting from Windows XP. This means that your code, even if compiled without IPv6 support, cannot run on Windows 2000. I'm also not sure if getaddrinfo() and freeaddrinfo() functions are available on all systems, someone know something about ? I have just imported and properly wrapped the libesmtp getaddrinfo and supporting implementation from fetchmail 6.3.8. I have done some simple configure and compile tests here. Its time to pass it back to you Guido for the win32 test. I think that will be better for us to use a wrapper for the functions, something like squid-getaddrinfo(), this will allow a simpler usage on Windows (see my previous mail). True. x seems to be the preferred prefix. I'll also see about wrap/include of the dependancy functions individually. ANYONE: Is anyone able to confirm for me whether I'm supposed to be using AC_SEARCH_LIBS() or AC_FUNC_CHECK() to do that properly? I'm kind of expecting another set of errors off these. It's just a matter of which ones. For now look for an undefined IPv6 into ACLIP.cc (line 296). Fixed. See tomorrow, I'm going to a barbecue with friends, today in Italy is a Holiday. :-) happy holiday :-) Amos
Re: Documenting non-public members
(pulling this back into squid-dev. private response was a slip of the
paste key.)
On Thu, 2007-08-16 at 10:40 +1200, Amos Jeffries wrote:
Most of the squid code is commented for programmers in the .cc where the
code is written. Leaving that as-is where possible, this is actual more
convenient for most functions as editors can change code and description
in one place.
BUT, API stuff MUST be written in the .h since there is no guarantee of
the user(s) having time to find and read the .cc (or it even being
available).
I am not sure how you distinguish code users from developers, but I
Ah, I mean user as someone writing code (A) somewhere than needs to use
an object (B)s' interface to make it do stuff. Rather than writing new
code inside object B itself.
agree that APIs (whatever that is) should be documented in (or its
documentation should be referenced from) .h files.
There is simply
no other good place to document them because there is no corresponding
definition.
There is the place where the {...} code itself lives. Which is the
written
intention there. I picked this because that is the current documenting
style for squid.
Class data members do not have a single place where they live inside the
code. IMO, virtually every data member should be briefly documented in
the .h file. If one cannot express what the data member means in one
short line, it is a good indication of a poor design.
Similar things can be said about groups of data and function members:
There is no place to document groupings in .cc file because member
definitions are far from each other.
The edge case here is virtual functions defined as virtual foo Bar(){}
in
the .h file itself.
Those needs a brief comment saying class X defined as empty or
do-nothing.
Actually, placing a .h comment that bar() implementation does nothing is
a dangerous practice! The child should not care or rely upon specific
parent's implementation of a virtual method because that implementation
may change. What does matter for the child is whether the parent method
should be called by the child. And, sometimes, whether it should be
called before or after the child code.
A short description of methods should also be appropriate (and is
often
very helpful) in .h file, even if they are not a part of public API.
True, we can do this regardless of where the detail description is. ANY
auto-docs comment in .h or .cc(i) just ahead of a object-name are
catenated for the final product.
It is much easier for me to find short member descriptions/briefs in
a .h file than in a .cc file, but if somebody is browsing doxygen pages,
there should indeed be no difference.
My rule of thumb is to place API descriptions and what is it? briefs
in .h files and how/why/etc implementation notes in .cc files.
... Um... Thats the way I had it. looking ahead to see ones' own back ... /
Amos
Re: squid-3 ?
Robert Collins wrote: On Thu, 2007-08-16 at 11:59 +0300, Tsantilas Christos wrote: Hi all, Alex Rousskov wrote: The primary idea behind RC1 is to bring in users who are ignoring PRE releases because there were so many PREs. We need more testers than a handful of folks running PREs on busy sites. RC1 release to attract testers is a good idea. The only comment I have is that it is still summer and some of squid code submitters are still in vacations so 1 or 2 developers get more load . Maybe September is a better month... Whats this summer you speak of? Go that man! Stops listening to radio flood warnings in this weekends destination and looks out window at gale force winds. Amos
Re: squid3-ipv6 squid3 configure.in,1.63.2.36,1.63.2.37
Is anyone able to help me with this configure problem? The goal is to have it test for a functions existence and if present define HAVE_function and add lib/function.$obj to the compile options. As far as I understand autoconf; I thought these checks as written below would do exactly that but its failing for me. dnl Search for OS support of IP Next Generation functions AC_CHECK_FUNC(getaddrinfo,[ AC_MSG_RESULT([yes]) ],[ AC_LIBOBJ(getaddrinfo) ]) + AC_SUBST(HAVE_GETADDRINFO) AC_CHECK_FUNC(getnameinfo,[ AC_MSG_RESULT([yes]) ],[ AC_LIBOBJ(getnameinfo) ]) + AC_SUBST(HAVE_GETNAMEINFO) Any ideas/help? Amos
Re: squid3-ipv6 squid3 configure.in,1.63.2.36,1.63.2.37
Never mind. 5 mins more thought found a working solution. Amos
Re: squid3-ipv6 squid3 configure.in,1.63.2.36,1.63.2.37
On tor, 2007-08-16 at 23:51 +1200, Amos Jeffries wrote: Is anyone able to help me with this configure problem? The goal is to have it test for a functions existence and if present define HAVE_function and add lib/function.$obj to the compile options. As far as I understand autoconf; I thought these checks as written below would do exactly that but its failing for me. dnl Search for OS support of IP Next Generation functions AC_CHECK_FUNC(getaddrinfo,[ AC_MSG_RESULT([yes]) ],[ AC_LIBOBJ(getaddrinfo) ]) + AC_SUBST(HAVE_GETADDRINFO) AC_CHECK_FUNC(getnameinfo,[ AC_MSG_RESULT([yes]) ],[ AC_LIBOBJ(getnameinfo) ]) + AC_SUBST(HAVE_GETNAMEINFO) Any ideas/help? Add it to the AC_REPLACE_FUNCS list already existing in configure.in for this purpose? Thanks Henrik. I couldn't find the AC_REPLACE_FUNCS outside a special library test (there is a long list for header file checks). But thats even better that the other one I found. Amos
Re: IPv6 support and Windows
Hi Amos, At 07.25 15/08/2007, Amos Jeffries wrote: Hi Amos, Not good news here ... :-( We have a problem: You have used getaddrinfo() and freeaddrinfo() functions, but on Windows they are available only starting from Windows XP. This means that your code, even if compiled without IPv6 support, cannot run on Windows 2000. I'm also not sure if getaddrinfo() and freeaddrinfo() functions are available on all systems, someone know something about ? We have already tested Linux 2.6+ and FreeBSD 4.5+ (4.3 _says_ it has them) Not sure about Solaris, OS2, or Darwin yet. I have just imported and properly wrapped the libesmtp getaddrinfo and supporting implementation from fetchmail 6.3.8. I have done some simple configure and compile tests here. Its time to pass it back to you Guido for the win32 test. I think that will be better for us to use a wrapper for the functions, something like squid-getaddrinfo(), this will allow a simpler usage on Windows (see my previous mail). I'm kind of expecting another set of errors off these. It's just a matter of which ones. snip err fixed earlier. I think that's everything now. CVS is compiling again here. Should be ready again for your next test. See tomorrow, I'm going to a barbecue with friends, today in Italy is a Holiday. :-) My turn for a few days holiday now. :-) This one is fully offline :-( Amos
Re: Code documentation trends
On Thu, 2007-08-16 at 10:21 +1200, Amos Jeffries wrote: What I'm hoping for is to have a paragraph for each class about where its intended to be used etc. That would be terrific! I was trying to provide that for the classes I was adding. Theres lots of little bits still need doing for the Mem component. You can see what it looks like at: http://squid.treenet.co.nz/Doc/Code/MemPool_8h.dyn#2eab9a6fa490ed8426a1800e0ec4 I think maybe it's the text-description not making it clear its a code-example. That description needs fixing. Yeah, I can hardly grok the current doxygen rendering. I just got around to the initial marking stage for MemPools. (most things listed as API are internals but defined in .h) There is a much better rendering of those macros now: http://squid.treenet.co.nz/Doc/Code/group__MemPoolsAPI.dyn Said that, I care more about how .h looks, and the change makes it look worse, IMO. I'm not sure its any worse than the macro definition itself. .h is still bad though, any Ideas? Amos
Re: squid3-ipv6 squid3 configure.in,1.63.2.36,1.63.2.37
Henrik Nordstrom wrote: On fre, 2007-08-17 at 14:23 +1200, Amos Jeffries wrote: Thanks Henrik. I couldn't find the AC_REPLACE_FUNCS outside a special library test (there is a long list for header file checks). ??? my configure.in reads AC_SUBST(LIBREGEX) AC_REPLACE_FUNCS(\ drand48 \ tempnam \ strerror \ initgroups ) dnl Not cached since people are likely to tune this AC_MSG_CHECKING(Default FD_SETSIZE value) Oh, I (miss-)interpreted that as part of the libregex check above it since it had no start comment. Fixing. Amos
Re: configuration guide link
Now that there's automatically generated guides from the source available (and they're not easy to find!), should the configuration guide link on http://www.squid-cache.org/ and elsewhere point to them? The visolve stuff is great, but it's not authoritative. Just a thought, I good one though. I have just split the Configuration Guide link in the menubar into three. One for each of the official latest, and one for the old visolve page. The visolve page is a year outdated now. Amos
Re: configuration guide link
On Wed, Aug 22, 2007, Amos Jeffries wrote: I good one though. I have just split the Configuration Guide link in the menubar into three. One for each of the official latest, and one for the old visolve page. The visolve page is a year outdated now. Hm, lets make the page look pretty now ? :) I have been giving that some thought and there are a few major changes I'd put in place before any CSS prettiness 1) add a SEE_ALSO: item to the cf.data.pre format - many of the options apply to more than just one 'area' ie icp_port fits into networking options and ICAP options areas 2) make a nice consistent way of titling the areas 3) re-order all the areas based on dependancies and common-usage - ie *_access depend on acl X I know thats a lot of change, but to be pretty it's needed. Amos
Re: configuration guide link
Adrian Chadd wrote: On Wed, Aug 22, 2007, Amos Jeffries wrote: I have been giving that some thought and there are a few major changes I'd put in place before any CSS prettiness 1) add a SEE_ALSO: item to the cf.data.pre format - many of the options apply to more than just one 'area' ie icp_port fits into networking options and ICAP options areas 2) make a nice consistent way of titling the areas 3) re-order all the areas based on dependancies and common-usage - ie *_access depend on acl X I know thats a lot of change, but to be pretty it's needed. Yup, but is there any particular reason why pretty-ifying the pages has to block on tidying up/improving cf.data.pre? Nothing blocking, just a wishlist really. Amos
Re: configuration guide link
Adrian Chadd wrote: On Wed, Aug 22, 2007, Amos Jeffries wrote: I have been giving that some thought and there are a few major changes I'd put in place before any CSS prettiness 1) add a SEE_ALSO: item to the cf.data.pre format - many of the options apply to more than just one 'area' ie icp_port fits into networking options and ICAP options areas 2) make a nice consistent way of titling the areas 3) re-order all the areas based on dependancies and common-usage - ie *_access depend on acl X I know thats a lot of change, but to be pretty it's needed. Yup, but is there any particular reason why pretty-ifying the pages has to block on tidying up/improving cf.data.pre? Oh, (click!) were you asking me to do the pretty? Amos
Re: configuration guide link
On Wed, Aug 22, 2007, Amos Jeffries wrote: Nothing blocking, just a wishlist really. Squid's wishlist is long; lets try to trim off whatever we can. :P I agree they are good ideas - and I'd really like to see those section markers be explicit section markers rather than comments which we pre mark - but I think those pages are butt ugly and I didn't have any intention of putting them anywhere very public until they were made much prettier. Maybe I'll just have to pretty them up myself and include it in my squid appliance.. I just did it. The templates are now labeled with CSS hooks, and a new cfgman.css is added to modify some default.css settings so they don't look bad. Anything else will depend on getting rid of the pre Amos
Re: configuration guide link
On Thu, Aug 23, 2007, Amos Jeffries wrote: I just did it. The templates are now labeled with CSS hooks, and a new cfgman.css is added to modify some default.css settings so they don't look bad. Anything else will depend on getting rid of the pre Which won't be that hard - create a section marker which generates the same output used to delineate sections in the cf file today. (Ie, uppercase and then a line of dashes.) That kills the most horrible use of pre in cfgman. The other use is comments but there's too much assumed layout there for it to be a non-trivial task. I'll wait for your CSS modifications to show up on the live site before I fiddle with the entry layouts. Just while I was doing it I had a thought. Correct me if I'm wrong on this but it seems the aliases for any option are actually all the old deprecated names for it. If thats right, the Also known As: section should become Replaces: as we really want the deprecated names to die off. Amos
default cache user
A while ago I patched my branch to allow configure-time setting (--with-cache-user=USER) of the squid run-time default user. I'm thinking this is worth adding to 3.0 before release. - it has been well tested now by me (~1 month use in production). - it reduces the patch level needed by OS maintainers - it only touches the squid.conf parser by making the default an autoconf macro instead of hard-coded. Any objections? Amos
Re: default cache user
On Fri, Aug 24, 2007, Amos Jeffries wrote: A while ago I patched my branch to allow configure-time setting (--with-cache-user=USER) of the squid run-time default user. I'm thinking this is worth adding to 3.0 before release. - it has been well tested now by me (~1 month use in production). - it reduces the patch level needed by OS maintainers - it only touches the squid.conf parser by making the default an autoconf macro instead of hard-coded. Not from me. What about calling it default-cache-user and I wondered about that. I don't exactly have a preference, just picked a shorter one. Though I suppose --with-default-user is better. default-cache-group ? group is not involved/configurable that way. Squid pulls default group from host passwd file seperately. cf.data.pre explains how. Amos
dlmalloc
Henrik, There are still traces of dlmalloc left in lib/Makefile.am (maybe elsewhere). Are they meant to be left or removed? Amos
epoll
After todays changes I am now getting (with --enable-epoll) checking for epoll_ctl in -lepoll... no checking for epoll_ctl... yes checking if epoll works... yes Error - no epoll support found Try running 'sh ./scripts/get_epoll-lib.sh' then run configure again The script mentions does not seem to exist. Amos
Re: epoll
On ons, 2007-08-29 at 00:34 +1200, Amos Jeffries wrote: After todays changes I am now getting (with --enable-epoll) checking for epoll_ctl in -lepoll... no checking for epoll_ctl... yes checking if epoll works... yes Error - no epoll support found Try running 'sh ./scripts/get_epoll-lib.sh' then run configure again Ah, sorry. Small typo there. Note: You do not need to specify --enable-epoll. It's automatically selected if found working which it is on your system.. The script mentions does not seem to exist. Right. Question is how many will need it.. I think not, and have now killed the section which refers to the script. Builds using the (these days) undocumented --enable-epoll option without properly working epoll will now either fail or result in a non-working binary, just as is the case when enabling kqueue on a system not supporting kqueue.. Thank you. I thought that would be the case. I was just trying to use/test a build as much like the OS its going into as the official package maintainer (Luigi) sets on build. On the side of this. I read through the PRE7 release notes and noticed that while the squid.conf changes are covered there is no comparable list of ./configure changes. Amos
Re: errorpages squid/errors/English/templates ERR_ACCESS_DENIED, 1.1.2.1,1.1.2.2 ERR_CANNOT_FORWARD,1.1.2.1,1.1.2.2 ERR_DNS_FAIL, 1.1.2.1,1.1.2.2
On Mon, Sep 03, 2007, Amos Jeffries wrote: Adrian, If your .pl script wraps each 'section' with: div id=$section_name.../div you will not need to place p/p in the meta data so often. Well, the trouble is that some of the messages actually have multiple paragraphs in them; I'm not sure what to do there just yet. Those are the few that might still need p inside the div, I also noticed a few with pre. p or anything can be nested inside the div easily and div#$section can be used in css as a nice handy anchor for customizations. Also, adding an optional @external-stylesheet@ header (*after* the inline css defaults) will let some admin use local css to re-style their own $section's too. Thats the eventual idea.. Cool :-) Amos
