Re: [PATCH] port of http_access2 from 2.6
On 01/20/2010 04:02 PM, Amos Jeffries wrote: On Wed, 20 Jan 2010 09:58:01 -0700, Alex Rousskov rouss...@measurement-factory.com wrote: Should the documentation mention that redirects are performed before the second access check? I think so. That being one of the two differences between this and http_access. Or did you mean the cf.data.pre changes were not descriptive/clear enough? It is the latter. I think the option documentation blob in cf.data.pre should mention the redirection in addition to adaptation. I am not a big fan of fooN names. Is using something more specific like post_adaptation_access be better long-term? Does adapted_http_access seem clearer? Being that it's a variant of http_access, always done and only useful when checking for adapted things. adapted_http_access sounds good to me. Thank you, Alex.
Squid 2.8 Roadmap
Now that Adrian has moved his work to Lusca, it looks like the Squid 2.8 roadmap http://wiki.squid-cache.org/RoadMap/Squid2 isn't reflecting reality (please correct me if I'm wrong, adri!). Although I don't want to accelerate 2.x work, nor get in the way of 3.x work, I think it would be good to solidify a lot of the improvements in 2.HEAD into a proper release, so that people don't have to run with a lot of patches from HEAD. In particular, I'd like to see the following patches in 2.8 (or 2.7STABLE, but AIUI that's not appropriate, in most cases). * Logging rewritten URLs - bug 2406 * Make PEER_TCP_MAGIC_COUNT configurable - bug 2377 * hier_code ACL - bug 2390 * HTCP / extension method patches - by benno, including 1235[3-5], 12358, 12364, 1236[7,8], 12427, 1245[5,6] patches * 64bit crash with PURGE and HTCP - bug 2799 * Add old entry back to async object - bug 2832 * CLR segfault - bug 2788 * Direct peer monitoring - bug 2643 * Adjustable latency stats - bug 2345 * Adjustable collapsed forwarding timeouts - bug 2504 * Idempotent start - bug 2599 * Configurable forward max tries - bug 2632 * Request body buffering - bug 2420 * HTCP logging - bug 2627 * ignore must-revalidate - bug 2645 * Aggressive caching - bug 2631 * Don't make fatal() dump core - bug 2673 * Make storeurl_rewriter work with Vary - bug 2678 * Make miss_access a slow lookup - bug 2688 I'm happy to help with documenting these, etc. as much as required, although I'm not really up to full release management. Any guidance, etc. would be helpful. WRT the roadmap, is the best thing to do to remove the current information and start collecting a list of applicable bugs? Or can we just give them a Milestone of 2.8 in bugzilla? Cheers, -- Mark Nottingham m...@yahoo-inc.com
Re: [PATCH] Limit log field width
On 01/21/2010 05:19 AM, Amos Jeffries wrote: Alex Rousskov wrote: Hello, The attached patch was motivated by user complaints that “vi”, “cat”, and even some more sophisticated log analysis tools have trouble handling long access.log lines. Those long log lines result from long URLs that do occur in the wild. Squid places a 8192 character limit on the URL length but that limit exceeds (a) some of the tool limits and (b) Squid's access.log buffer limit (if some other fields are logged). My solution was to honor the .precision setting in logformat field specifications. You can use it with %ru or any other text field. For example, the format code below limits logged URI size to the first 1000 characters. logformat xsquid ... %rm %.1000ru %un ... Squid access log line buffer cannot exceed 8192 characters. If you want to preserve fields logged after the URL, your logged URL width limit should be smaller than 8192 to leave space for other fields on the log line. There is no width limit by default. Here is a possible commit message: - Support maximum field width for string access.log fields. Some standard command-line and some log processing tools have trouble handling URLs or other logged fields exceeding 8KB in length. Moreover, Squid violates its own log line format and truncates the entire log line if, for example, the URL is 8KB long. By supporting .precision format argument, we allow the administrator to specify logged URL size and avoid these problems. Limiting logged field width has no effect on traffic on the wire. Um, not quite true now that we log over UDP. anyways... Also not quite true if you look at the log via ssh :-) TODO: The name comes from the printf(3) precision format part. It may be a good idea to rename our precision into max_width or similar, especially if we do not support floating point precision logging. [width[.maximum]] gets my vote. seems clear and concise. The maximum width field does not require the width field. Should I change to [width][.maximum] or [width_min].[width_max]? These are just labels in the .pre -- there is no effect on the code. TODO: Old code used chars to store user-configured field width and precision. That does not work for URLs, headers, and other entries longer than 256 characters. This patch changes the storage type to int. The code should probably be polished further to remove unsigned-signed conversions. - Please review. +1. If the TODO can be removed. You are talking about the unsigned-signed TODO? I am happy to make those changes, but they will make the patch much bigger, there is higher risk of compilation warnings, and they are not really about the feature in question. Should I still do it? Perhaps as a separate commit? Thank you, Alex.
Re: Squid 2.8 Roadmap
Mark Nottingham wrote: Now that Adrian has moved his work to Lusca, it looks like the Squid 2.8 roadmap http://wiki.squid-cache.org/RoadMap/Squid2 isn't reflecting reality (please correct me if I'm wrong, adri!). Although I don't want to accelerate 2.x work, nor get in the way of 3.x work, I think it would be good to solidify a lot of the improvements in 2.HEAD into a proper release, so that people don't have to run with a lot of patches from HEAD. In particular, I'd like to see the following patches in 2.8 (or 2.7STABLE, but AIUI that's not appropriate, in most cases). * Logging rewritten URLs - bug 2406 * Make PEER_TCP_MAGIC_COUNT configurable - bug 2377 * hier_code ACL - bug 2390 * HTCP / extension method patches - by benno, including 1235[3-5], 12358, 12364, 1236[7,8], 12427, 1245[5,6] patches * 64bit crash with PURGE and HTCP - bug 2799 * Add old entry back to async object - bug 2832 * CLR segfault - bug 2788 * Direct peer monitoring - bug 2643 * Adjustable latency stats - bug 2345 * Adjustable collapsed forwarding timeouts - bug 2504 * Idempotent start - bug 2599 * Configurable forward max tries - bug 2632 * Request body buffering - bug 2420 * HTCP logging - bug 2627 * ignore must-revalidate - bug 2645 * Aggressive caching - bug 2631 * Don't make fatal() dump core - bug 2673 * Make storeurl_rewriter work with Vary - bug 2678 * Make miss_access a slow lookup - bug 2688 I'm happy to help with documenting these, etc. as much as required, although I'm not really up to full release management. Any guidance, etc. would be helpful. WRT the roadmap, is the best thing to do to remove the current information and start collecting a list of applicable bugs? Or can we just give them a Milestone of 2.8 in bugzilla? Cheers, I know I don't have a lot of say in this, but here is my 2c anyway... If Henrik and you agree that 2.HEAD is stable enough for poduction use I wont objects. Even while reaching a point that 2.8 might happen saddens me, I can see that it might be needed. I'm happy with simply renaming 2.HEAD - 2.8 formally. But not really with branching a new release. Opening HEAD again for a possible 2.9 is IMO a bad idea. Making 2.8 formally the terminal 2.x release while allowing the possibility that its feature set is not as stone-fixed as earlier 2.x. Amos -- Please be using Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21 Current Beta Squid 3.1.0.15
Re: [PATCH] Limit log field width
Alex Rousskov wrote: On 01/21/2010 05:19 AM, Amos Jeffries wrote: Alex Rousskov wrote: Hello, snip TODO: The name comes from the printf(3) precision format part. It may be a good idea to rename our precision into max_width or similar, especially if we do not support floating point precision logging. [width[.maximum]] gets my vote. seems clear and concise. The maximum width field does not require the width field. Should I change to [width][.maximum] or [width_min].[width_max]? These are just labels in the .pre -- there is no effect on the code. Yes definitely needs to be clarified there. The BNF fooled me even with your example below it. I'll agree with your latter naming with that independence. TODO: Old code used chars to store user-configured field width and precision. That does not work for URLs, headers, and other entries longer than 256 characters. This patch changes the storage type to int. The code should probably be polished further to remove unsigned-signed conversions. - Please review. +1. If the TODO can be removed. You are talking about the unsigned-signed TODO? I am happy to make those changes, but they will make the patch much bigger, there is higher risk of compilation warnings, and they are not really about the feature in question. Should I still do it? Perhaps as a separate commit? Up to you. Amos -- Please be using Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21 Current Beta Squid 3.1.0.15
Re: Squid 2.8 Roadmap
I'm not against that, as long as 2.8 isn't rigidly feature-locked. It would be a bit weird, IMO, but I can live with it. On 23/01/2010, at 11:02 AM, Amos Jeffries wrote: Mark Nottingham wrote: Now that Adrian has moved his work to Lusca, it looks like the Squid 2.8 roadmap http://wiki.squid-cache.org/RoadMap/Squid2 isn't reflecting reality (please correct me if I'm wrong, adri!). Although I don't want to accelerate 2.x work, nor get in the way of 3.x work, I think it would be good to solidify a lot of the improvements in 2.HEAD into a proper release, so that people don't have to run with a lot of patches from HEAD. In particular, I'd like to see the following patches in 2.8 (or 2.7STABLE, but AIUI that's not appropriate, in most cases). * Logging rewritten URLs - bug 2406 * Make PEER_TCP_MAGIC_COUNT configurable - bug 2377 * hier_code ACL - bug 2390 * HTCP / extension method patches - by benno, including 1235[3-5], 12358, 12364, 1236[7,8], 12427, 1245[5,6] patches * 64bit crash with PURGE and HTCP - bug 2799 * Add old entry back to async object - bug 2832 * CLR segfault - bug 2788 * Direct peer monitoring - bug 2643 * Adjustable latency stats - bug 2345 * Adjustable collapsed forwarding timeouts - bug 2504 * Idempotent start - bug 2599 * Configurable forward max tries - bug 2632 * Request body buffering - bug 2420 * HTCP logging - bug 2627 * ignore must-revalidate - bug 2645 * Aggressive caching - bug 2631 * Don't make fatal() dump core - bug 2673 * Make storeurl_rewriter work with Vary - bug 2678 * Make miss_access a slow lookup - bug 2688 I'm happy to help with documenting these, etc. as much as required, although I'm not really up to full release management. Any guidance, etc. would be helpful. WRT the roadmap, is the best thing to do to remove the current information and start collecting a list of applicable bugs? Or can we just give them a Milestone of 2.8 in bugzilla? Cheers, I know I don't have a lot of say in this, but here is my 2c anyway... If Henrik and you agree that 2.HEAD is stable enough for poduction use I wont objects. Even while reaching a point that 2.8 might happen saddens me, I can see that it might be needed. I'm happy with simply renaming 2.HEAD - 2.8 formally. But not really with branching a new release. Opening HEAD again for a possible 2.9 is IMO a bad idea. Making 2.8 formally the terminal 2.x release while allowing the possibility that its feature set is not as stone-fixed as earlier 2.x. Amos -- Please be using Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21 Current Beta Squid 3.1.0.15 -- Mark Nottingham m...@yahoo-inc.com
Re: Squid 2.8 Roadmap
For that matter, we could get all of this into 2.7, if we relax it... On 23/01/2010, at 11:02 AM, Amos Jeffries wrote: Mark Nottingham wrote: Now that Adrian has moved his work to Lusca, it looks like the Squid 2.8 roadmap http://wiki.squid-cache.org/RoadMap/Squid2 isn't reflecting reality (please correct me if I'm wrong, adri!). Although I don't want to accelerate 2.x work, nor get in the way of 3.x work, I think it would be good to solidify a lot of the improvements in 2.HEAD into a proper release, so that people don't have to run with a lot of patches from HEAD. In particular, I'd like to see the following patches in 2.8 (or 2.7STABLE, but AIUI that's not appropriate, in most cases). * Logging rewritten URLs - bug 2406 * Make PEER_TCP_MAGIC_COUNT configurable - bug 2377 * hier_code ACL - bug 2390 * HTCP / extension method patches - by benno, including 1235[3-5], 12358, 12364, 1236[7,8], 12427, 1245[5,6] patches * 64bit crash with PURGE and HTCP - bug 2799 * Add old entry back to async object - bug 2832 * CLR segfault - bug 2788 * Direct peer monitoring - bug 2643 * Adjustable latency stats - bug 2345 * Adjustable collapsed forwarding timeouts - bug 2504 * Idempotent start - bug 2599 * Configurable forward max tries - bug 2632 * Request body buffering - bug 2420 * HTCP logging - bug 2627 * ignore must-revalidate - bug 2645 * Aggressive caching - bug 2631 * Don't make fatal() dump core - bug 2673 * Make storeurl_rewriter work with Vary - bug 2678 * Make miss_access a slow lookup - bug 2688 I'm happy to help with documenting these, etc. as much as required, although I'm not really up to full release management. Any guidance, etc. would be helpful. WRT the roadmap, is the best thing to do to remove the current information and start collecting a list of applicable bugs? Or can we just give them a Milestone of 2.8 in bugzilla? Cheers, I know I don't have a lot of say in this, but here is my 2c anyway... If Henrik and you agree that 2.HEAD is stable enough for poduction use I wont objects. Even while reaching a point that 2.8 might happen saddens me, I can see that it might be needed. I'm happy with simply renaming 2.HEAD - 2.8 formally. But not really with branching a new release. Opening HEAD again for a possible 2.9 is IMO a bad idea. Making 2.8 formally the terminal 2.x release while allowing the possibility that its feature set is not as stone-fixed as earlier 2.x. Amos -- Please be using Current Stable Squid 2.7.STABLE7 or 3.0.STABLE21 Current Beta Squid 3.1.0.15 -- Mark Nottingham m...@yahoo-inc.com
