Re: 3.1 steps forward
On Thu, Feb 14, 2008, Amos Jeffries wrote: I'm just checking 3.1 progress maps now that the current big patches are in. Adrian: http://wiki.squid-cache.org/Features/LogDaemon Can we add a timeline on this now please? If it requires a push THN will provide a little funding to get it for our use in 3.1. Its done. Its in a branch in sourceforge somewhere. Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: 3.1 steps forward
Adrian Chadd wrote: On Thu, Feb 14, 2008, Amos Jeffries wrote: I'm just checking 3.1 progress maps now that the current big patches are in. Adrian: http://wiki.squid-cache.org/Features/LogDaemon Can we add a timeline on this now please? If it requires a push THN will provide a little funding to get it for our use in 3.1. Its done. Its in a branch in sourceforge somewhere. Sorry if I was obtuse. I meant that you are the expert on this one. Are you able to re-locate that mysterious branch and estimate when you will have time to sync and push it to HEAD? Thanks Amos -- Please use Squid 2.6STABLE17+ or 3.0STABLE1+ There are serious security advisories out on all earlier releases.
Re: 3.1 steps forward
On Fri, Feb 15, 2008, Amos Jeffries wrote: Sorry if I was obtuse. I meant that you are the expert on this one. Are you able to re-locate that mysterious branch and estimate when you will have time to sync and push it to HEAD? squid3_logdaemon have you fixed the AddrInfo leak? :) Adrian
Re: squid3 future directory structure
On Thu, 2008-02-14 at 11:45 +1300, Amos Jeffries wrote: http://wiki.squid-cache.org/Features/SourceLayout Added ICMP to your list of components. It's a clear-cut one now. Thank you. Can you lead this activity? We need somebody who will drive the discussion and oversee actual changes... 1) Should we use squid/src/squid/ root for most sources to include header files as squid/group/file.h? This may be required for installed headers and 3rd party code using those headers. I'm undecided on this one. Lack of info to base the decision on. It will really depend on the descision of whether or not to publish the headers for external use. I am inclined towards the publishing, but not towards the squid/src/squid hack we need to do. Yeah. I posted a separate message on eCAP that has an alternative to exposing Squid headers and libraries. Perhaps that would work better. 2) Where to put OS-compatabilities wrappers that are currently located in squid/lib and squid/include? I like them where they are. It's the extras that have been added there (guilty!) and the mixed styles of compatibility coding that are confusing things. I do not like the generic says-nothing names lib and include. I also do not like that the two directories pollute the top-level directory. Can we, perhaps, agree on something like squid/compat/include squid/compat/lib I'm sorry Duane, but the use you seem to like of adding Squid_ brings to mind more of 'completely re-written for squid, with different behaviour or usage' function. The problem with xFoo is that other libraries we link with might have their own xFoo. A C++ solution for this is namespaces, I guess, but it would require more work to implement that approach in a clean way. 3) Where to put 3rd party libraries that are currently located in squid/lib and squid/include? ./src/dependancies/ ?? ./dependancies/ ?? Definately split from the compat files though. How about squid/import/ It is more specific, shorter, and easier to spell :-). 6) Should directory names use just_small, CamelCase, or CAPS letters? I Think CamelCase like we do for files and classes, with acroynms being upper case is working. When we can make the converted and re-used legacy files from lower-only to CamelCase it will sit better than now. Any chance to convince you and others that HTTPReply is not as readable as HttpReply? If we start using namespaces for protocols, then this becomes a non-issue because HTTP::Reply reads just fine. Perhaps this should be a separate question. Should we start using namespaces for protocols and other important groups? I think we should. That said, this may impact the /usr/local/include descision. I think there is some tradition of lower-case with underscores on most OS. I am not sure that tradition applies to C++ stuff or modern code in general. Thank you, Alex.
eCAP: expose Squid or link with eCAP lib?
Hello, The Squid eCAP branch already has limited support for loadable modules. I have also written some eCAP-specific code, but I am not satisfied with its design yet. This may be the last opportunity to change things without rewriting a lot of code so I would love to get your feedback and ideas. An eCAP loadable module needs to plug into Squid message processing pipeline, similar to how ICAP servers do that now, but without ICAP/TCP overheads. I see two design choices for giving the module efficient access to Squid: 1) Expose Squid internals: Publish/install Squid headers and libraries to give direct access to Squid resources. This approach will most likely require installing pretty much all headers because the module may need to use many Squid services (e.g., DNS lookups) and because of the dependencies between Squid headers. 2) Link with an eCAP library: Implement a Squid-independent eCAP library that Squid and modules will build with to get connected to each other. This way, Squid does not have to publish any of its headers (the library does). This approach may simplify Squid header management and even allow integration with other proxies, but it is more work because it is a stand-alone library and because Squid would have to translate between internal Squid types and eCAP library types. I like the straightforwardness of (1) but it is crude and messy. I like the elegance of (2) but it is more work. Currently, I favor (2) but I want to hear what others think. Which path would you choose? Does anybody have better ideas on how to support eCAP modules that do ICAP-like adaptations inside Squid? Thank you, Alex.
Squid Hardware
well today my colleges are discuss about the caching their very long conversation their is 2 participants I am behalf of squid and others are from BLUECOAT proxy sg hardware . well I am asking for squid developers there is any solution on hardware based embedded with squid today the marketing are being more consider to buy hardware not appoint any employee to maintain cache services what the compare with other vendors .. we have a client ready to buy solution but who is the fastest one ? -- View this message in context: http://www.nabble.com/Squid-Hardware-tp15489384p15489384.html Sent from the Squid - Development mailing list archive at Nabble.com.
Re: eCAP: expose Squid or link with eCAP lib?
On Fri, 2008-02-15 at 09:07 +1100, Robert Collins wrote: On Thu, 2008-02-14 at 09:09 -0700, Alex Rousskov wrote: An eCAP loadable module needs to plug into Squid message processing pipeline, similar to how ICAP servers do that now, but without ICAP/TCP overheads. I see two design choices for giving the module efficient access to Squid: 1) Expose Squid internals: Publish/install Squid headers and libraries to give direct access to Squid resources. This approach will most likely require installing pretty much all headers because the module may need to use many Squid services (e.g., DNS lookups) and because of the dependencies between Squid headers. I prefer this technically because it means that work done to make eCAP clean will naturally clean up squid too. I agree that a more thorough cleanup would be required to install headers. On the other hand, most of that cleanup effort is unrelated to Squid code quality as such (see the squid/src/squid/ issue for an example). The effort would be directed at satisfying external requirements placed on installable libraries and not on Squid code quality. In other words, this may add work that does not really benefit Squid directly. The amount and complexity of that extra work may even derail the cleanup! With option (2), that should not happen. 2) Link with an eCAP library: Implement a Squid-independent eCAP library that Squid and modules will build with to get connected to each other. This way, Squid does not have to publish any of its headers (the library does). This approach may simplify Squid header management and even allow integration with other proxies, but it is more work because it is a stand-alone library and because Squid would have to translate between internal Squid types and eCAP library types. Its more work both at code and at runtime. The only thing it really allows that 1) doesn't is non-GPL eCAP modules. Whether (1) requires GPL is debatable (and we should not debate that legal question here). (2) Is a cleaner solution from code quality point and module management point of view because you are forced to carefully encapsulate eCAP module capabilities; access to Squid internals is performed via well-known channels. With (1), anything within Squid is up for grabs as there are no API restrictions. Thank you, Alex.
Re: eCAP: expose Squid or link with eCAP lib?
Hello, The Squid eCAP branch already has limited support for loadable modules. I have also written some eCAP-specific code, but I am not satisfied with its design yet. This may be the last opportunity to change things without rewriting a lot of code so I would love to get your feedback and ideas. An eCAP loadable module needs to plug into Squid message processing pipeline, similar to how ICAP servers do that now, but without ICAP/TCP overheads. I see two design choices for giving the module efficient access to Squid: 1) Expose Squid internals: Publish/install Squid headers and libraries to give direct access to Squid resources. This approach will most likely require installing pretty much all headers because the module may need to use many Squid services (e.g., DNS lookups) and because of the dependencies between Squid headers. 2) Link with an eCAP library: Implement a Squid-independent eCAP library that Squid and modules will build with to get connected to each other. This way, Squid does not have to publish any of its headers (the library does). This approach may simplify Squid header management and even allow integration with other proxies, but it is more work because it is a stand-alone library and because Squid would have to translate between internal Squid types and eCAP library types. I like the straightforwardness of (1) but it is crude and messy. I like the elegance of (2) but it is more work. Currently, I favor (2) but I want to hear what others think. Which path would you choose? I'd choose (3). Some of the basic types have clean headers already. Is it possible to incrementally grow eCAP access to the internals of squid? Starting with, ie just a buffer of page text, and work the API up? I mean, we publish the available cleaned squid .h for its use starting at lowest levels needed to support eCAP and working the way up to full control paths. So the eCAP library never defines its own seperate types publicly, but uses the squid ones as they get cleaned up. I imagine the basic first-signup users of eCAP will be wanting to perform content-filters on pages as they arrive. That only needs a read-only buffer of the page data and a hook to allow/deny the transaction. Amos Does anybody have better ideas on how to support eCAP modules that do ICAP-like adaptations inside Squid? Thank you, Alex.
Re: squid3 future directory structure
On Thu, 2008-02-14 at 11:45 +1300, Amos Jeffries wrote: http://wiki.squid-cache.org/Features/SourceLayout Added ICMP to your list of components. It's a clear-cut one now. Thank you. Can you lead this activity? We need somebody who will drive the discussion and oversee actual changes... 1) Should we use squid/src/squid/ root for most sources to include header files as squid/group/file.h? This may be required for installed headers and 3rd party code using those headers. I'm undecided on this one. Lack of info to base the decision on. It will really depend on the descision of whether or not to publish the headers for external use. I am inclined towards the publishing, but not towards the squid/src/squid hack we need to do. Yeah. I posted a separate message on eCAP that has an alternative to exposing Squid headers and libraries. Perhaps that would work better. 2) Where to put OS-compatabilities wrappers that are currently located in squid/lib and squid/include? I like them where they are. It's the extras that have been added there (guilty!) and the mixed styles of compatibility coding that are confusing things. I do not like the generic says-nothing names lib and include. I also do not like that the two directories pollute the top-level directory. Can we, perhaps, agree on something like squid/compat/include squid/compat/lib I'd agree with that. I'm sorry Duane, but the use you seem to like of adding Squid_ brings to mind more of 'completely re-written for squid, with different behaviour or usage' function. The problem with xFoo is that other libraries we link with might have their own xFoo. A C++ solution for this is namespaces, I guess, but it would require more work to implement that approach in a clean way. 3) Where to put 3rd party libraries that are currently located in squid/lib and squid/include? ./src/dependancies/ ?? ./dependancies/ ?? Definately split from the compat files though. How about squid/import/ It is more specific, shorter, and easier to spell :-). Yes, that works. 6) Should directory names use just_small, CamelCase, or CAPS letters? I Think CamelCase like we do for files and classes, with acroynms being upper case is working. When we can make the converted and re-used legacy files from lower-only to CamelCase it will sit better than now. Any chance to convince you and others that HTTPReply is not as readable as HttpReply? Some, but not a lot :-) I'm constantly encountering syntax and typo errors because HTTP is not always spelt upper-case in class names. If we start using namespaces for protocols, then this becomes a non-issue because HTTP::Reply reads just fine. Perhaps this should be a separate question. Should we start using namespaces for protocols and other important groups? I think we should. Yes on both counts. its a big change, but big changes and goals are what we are about these days. Working up to it slowly would be fine, with a better definition of where and what before any code goes in, but I think it should be one of the end-goals. That said, this may impact the /usr/local/include descision. I think there is some tradition of lower-case with underscores on most OS. I am not sure that tradition applies to C++ stuff or modern code in general. Meybe yes, meybe no. I haven't seen enough C++ code in /usr/* to judge it properly by. Amos
Re: eCAP: expose Squid or link with eCAP lib?
On Fri, 2008-02-15 at 12:15 +1300, Amos Jeffries wrote: I imagine the basic first-signup users of eCAP will be wanting to perform content-filters on pages as they arrive. That only needs a read-only buffer of the page data and a hook to allow/deny the transaction. FWIW, some of the users waiting for eCAP have full-functioning Traffic Spicer modules that do a lot more than allow/deny based on read-only page buffers. I agree that it is possible to gradually cleanup and publish polished Squid headers, but the first publication would still require a major cleanup to remove the current inter-header dependencies! And when you are done with that cleanup, you might as well expose pretty much everything :-). To do gradual exposure without the major initial cleanup, you need that eCAP library that provides its own clean types and hides Squid guts from the modules. That's pretty much the same as option #2, as nobody is going to start with a full-blown eCAP library anyway. I think the core question here is whether it is a good idea to expose [polished, cleaned, improved, etc.] Squid guts? Is it better to provide a well-defined eCAP-focused interface instead? I think an eCAP interface is a better idea even if we have polished guts. Besides other speculative code quality reasons already discussed here, I add this one: Guts change. Squid3 guts will change a lot. eCAP modules should not stop Squid from changing, and should not suffer too much when Squid guts change. I should have thought of that reason earlier, but that's why it helps to discuss these things :-). Thank you, Alex.
Re: eCAP: expose Squid or link with eCAP lib?
On Fri, 2008-02-15 at 12:15 +1300, Amos Jeffries wrote: I imagine the basic first-signup users of eCAP will be wanting to perform content-filters on pages as they arrive. That only needs a read-only buffer of the page data and a hook to allow/deny the transaction. FWIW, some of the users waiting for eCAP have full-functioning Traffic Spicer modules that do a lot more than allow/deny based on read-only page buffers. I agree that it is possible to gradually cleanup and publish polished Squid headers, but the first publication would still require a major cleanup to remove the current inter-header dependencies! And when you are done with that cleanup, you might as well expose pretty much everything :-). To do gradual exposure without the major initial cleanup, you need that eCAP library that provides its own clean types and hides Squid guts from the modules. That's pretty much the same as option #2, as nobody is going to start with a full-blown eCAP library anyway. I think the core question here is whether it is a good idea to expose [polished, cleaned, improved, etc.] Squid guts? Is it better to provide a well-defined eCAP-focused interface instead? I think an eCAP interface is a better idea even if we have polished guts. Besides other speculative code quality reasons already discussed here, I add this one: Guts change. Squid3 guts will change a lot. eCAP modules should not stop Squid from changing, and should not suffer too much when Squid guts change. I should have thought of that reason earlier, but that's why it helps to discuss these things :-). True on both counts. So what shall this interface look like? I was imagining an interface that pulled/set some squid data structures for client use. Even if it was gated through an API layer. Amos
Re: Squid Hardware
On Thu, Feb 14, 2008, pokeman wrote: well today my colleges are discuss about the caching their very long conversation their is 2 participants I am behalf of squid ??? and others are from BLUECOAT proxy sg hardware . well I am asking for squid developers there is any solution on hardware based embedded with squid today the marketing are being more consider to buy hardware not appoint any employee to maintain cache services what the compare with other vendors .. we have a client ready to buy solution but who is the fastest one ? There are plenty of Squid based appliances out there; there's a list of companies using Squid linked off http://www.squid-cache.org/ . I'm in the process of (slowly!) building a commercial proxy appliance based on Squid and FreeBSD which will inherit all the bells and whistles that Squid will offer. It'll include extras like Youtube/Windows Updates caching rules, some anti phishing and anti spyware filtering, and whatever else clients are interested in paying for. Send me a private email if you're interested. Adrian
Re: eCAP: expose Squid or link with eCAP lib?
On Fri, Feb 15, 2008, Amos Jeffries wrote: I'd choose (3). Some of the basic types have clean headers already. Is it possible to incrementally grow eCAP access to the internals of squid? Starting with, ie just a buffer of page text, and work the API up? I mean, we publish the available cleaned squid .h for its use starting at lowest levels needed to support eCAP and working the way up to full control paths. So the eCAP library never defines its own seperate types publicly, but uses the squid ones as they get cleaned up. I imagine the basic first-signup users of eCAP will be wanting to perform content-filters on pages as they arrive. That only needs a read-only buffer of the page data and a hook to allow/deny the transaction. If you don't tidy up stuff early on then you end up introducing more dependencies which will make restructuring more of a pain. Adrian -- - Xenion - http://www.xenion.com.au/ - VPS Hosting - Commercial Squid Support - - $25/pm entry-level VPSes w/ capped bandwidth charges available in WA -
Re: squid3 future directory structure
On Fri, 2008-02-15 at 12:24 +1300, Amos Jeffries wrote: 6) Should directory names use just_small, CamelCase, or CAPS letters? I Think CamelCase like we do for files and classes, with acroynms being upper case is working. When we can make the converted and re-used legacy files from lower-only to CamelCase it will sit better than now. Any chance to convince you and others that HTTPReply is not as readable as HttpReply? Some, but not a lot :-) I'm constantly encountering syntax and typo errors because HTTP is not always spelt upper-case in class names. I'd prefer lowercase always for directories. filenames and CaSe are always headaches on windows :) -Rob -- GPG key available at: http://www.robertcollins.net/keys.txt. signature.asc Description: This is a digitally signed message part
