serf was Re: mod_cache wishlist

2005-08-24 Thread Justin Erenkrantz
--On August 24, 2005 4:21:38 PM +0100 Nick Kew <[EMAIL PROTECTED]> wrote: So, is it time we introduced a general-purpose apr_http_client? I'd be prepared to offer my code as a startingpoint, but I'd rather not take the driving seat for further development and documentation. We already went dow

Re: mod_cache wishlist

2005-08-24 Thread Brian Akins
Paul Querna wrote: FWIW, I will be looking at adding support for EPoll and/or KQueue to the curl_multi_* interface sometime soonish for 'work. on the curl-dev list, I suggested just using libevent (http://www.monkey.org/~provos/libevent/), because it already encapsulates all that. -- Bri

Re: mod_cache wishlist

2005-08-24 Thread Paul Querna
Brian Akins wrote: > Nick Kew wrote: > >> Alternatively, maybe someone could post an executive summary of the >> problems and benefits of standardising on libcurl? > > > I'm pretty familiar with libcurl. Great library. Here are some issues > I have had: > > - asynchronous uses select only. F

Re: mod_cache wishlist

2005-08-24 Thread Parin Shah
On 8/24/05, Colm MacCarthaigh <[EMAIL PROTECTED]> wrote: > On Wed, Aug 24, 2005 at 09:18:54AM -0500, Parin Shah wrote: > > > > I have fixed that memory leak problem. also added script to include > > > > libcurl whenever this module is included. > > > > > > I hope that it doesn't mean that libcurl i

Re: mod_cache wishlist

2005-08-24 Thread Brian Akins
Nick Kew wrote: Alternatively, maybe someone could post an executive summary of the problems and benefits of standardising on libcurl? I'm pretty familiar with libcurl. Great library. Here are some issues I have had: - asynchronous uses select only. - random crashes with openssl, normal b

Re: mod_cache wishlist

2005-08-24 Thread Nick Kew
Colm MacCarthaigh wrote: On Wed, Aug 24, 2005 at 09:18:54AM -0500, Parin Shah wrote: I have fixed that memory leak problem. also added script to include libcurl whenever this module is included. I hope that it doesn't mean that libcurl is going to be a permanent solution, when subrequests (wi

Re: mod_cache wishlist

2005-08-24 Thread Colm MacCarthaigh
On Wed, Aug 24, 2005 at 09:18:54AM -0500, Parin Shah wrote: > > > I have fixed that memory leak problem. also added script to include > > > libcurl whenever this module is included. > > > > I hope that it doesn't mean that libcurl is going to be a permanent > > solution, when subrequests (with min

Re: mod_cache wishlist

2005-08-24 Thread Parin Shah
> > I have fixed that memory leak problem. also added script to include > > libcurl whenever this module is included. > > I hope that it doesn't mean that libcurl is going to be a permanent > solution, when subrequests (with minor changes) could serve the same > purpose. Certainly not, We would h

Re: mod_cache wishlist

2005-08-23 Thread Eli Marmor
Parin Shah wrote: > I have fixed that memory leak problem. also added script to include > libcurl whenever this module is included. I hope that it doesn't mean that libcurl is going to be a permanent solution, when subrequests (with minor changes) could serve the same purpose. BTW: if subrequest

Re: mod_cache wishlist

2005-08-23 Thread Parin Shah
Hi, I have fixed that memory leak problem. also added script to include libcurl whenever this module is included. http://utdallas.edu/~parinshah/mod-c-requester.0.3.tar.gz Thanks, Parin. On 8/23/05, Parin Shah <[EMAIL PROTECTED]> wrote: > > > ohh, I thought I was taking care of it. I mean, code

Re: mod_cache wishlist

2005-08-23 Thread Parin Shah
> > ohh, I thought I was taking care of it. I mean, code frees the memory > > when no longer needed except during the shutdown of server. anyway I > > will go through the code again to check that. Also feel free to point > > out the code which is causing memory leak problem. > > I'll look through

Re: mod_cache wishlist

2005-08-23 Thread Parin Shah
> > ohh, I thought I was taking care of it. I mean, code frees the memory > > when no longer needed except during the shutdown of server. anyway I > > will go through the code again to check that. Also feel free to point > > out the code which is causing memory leak problem. > > I'll look through

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Joshua Slive wrote: It looks like you want an extreme level of flexibility for making caching decisions based on characteristics of the request. Why not piggy-back on mod_rewrite, which already has an absurdly complex matching capability. As in RewriteCond {REMOTE_ADDR} =127.0.0.1 RewriteRu

Re: mod_cache wishlist

2005-08-23 Thread Joshua Slive
Brian Akins wrote: Some pseudo match configs and code: It looks like you want an extreme level of flexibility for making caching decisions based on characteristics of the request. Why not piggy-back on mod_rewrite, which already has an absurdly complex matching capability. As in Rewrite

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Some pseudo match configs and code: Just examples, maybe not useful or doable. #only cache things in /stuff when request comes from localhost CacheEnable disk client=127.0.0.1 path=/stuff #disable cacheing from special host CacheDisable client=10.1.1.1.10 #don't cache any ssl stuff CacheDis

Re: mod_cache wishlist

2005-08-23 Thread Bill Stoddard
Brian Akins wrote: Bill Stoddard wrote: I've not looked at your code so I can't make specific recommendations. Just remember, code allocated with any of the apr_pool functions is freed only when that pool is reclaimed (end of a request for a request pool, shutdown of the server for pconf, etc

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Bill Stoddard wrote: I've not looked at your code so I can't make specific recommendations. Just remember, code allocated with any of the apr_pool functions is freed only when that pool is reclaimed (end of a request for a request pool, shutdown of the server for pconf, etc.). mod_mem_cache us

Re: mod_cache wishlist

2005-08-23 Thread Bill Stoddard
Parin Shah wrote: Cool. Very good start. Leaks memory like a sieve, but good start. ohh, I thought I was taking care of it. I mean, code frees the memory when no longer needed except during the shutdown of server. anyway I will go through the code again to check that. Also feel free to point

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Colm MacCarthaigh wrote: per-dir does not help in quick_handler. No, but it is useful at the save stage. True. This would probably be fine. I would like to see more flexible url matching, ala squid. Perhaps a way so that modules can register their own matching functions. Example. Ca

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Parin Shah wrote: ohh, I thought I was taking care of it. I mean, code frees the memory when no longer needed except during the shutdown of server. anyway I will go through the code again to check that. Also feel free to point out the code which is causing memory leak problem. I'll look throug

Re: mod_cache wishlist

2005-08-23 Thread Parin Shah
> > Cool. Very good start. Leaks memory like a sieve, but good start. > ohh, I thought I was taking care of it. I mean, code frees the memory when no longer needed except during the shutdown of server. anyway I will go through the code again to check that. Also feel free to point out the code wh

Re: mod_cache wishlist

2005-08-23 Thread Colm MacCarthaigh
On Tue, Aug 23, 2005 at 10:18:37AM -0400, Brian Akins wrote: > >I've been looking at this, and it's possibly the Syntax that put me off, > >but it looks painful on the admin, and probably on the server too. > >There's nothing in those examples that can't be achieved by making the > >non-CacheEnable

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Parin Shah wrote: I am already working on it. I have also posted initial version of this module. http://utdallas.edu/~parinshah/mod-c-requester.0.2.tar.gz -Parin. Cool. Very good start. Leaks memory like a sieve, but good start. It would be cool if we could find a way to use Apache's su

Re: mod_cache wishlist

2005-08-23 Thread Brian Akins
Colm MacCarthaigh wrote: To a large extent mod_cache_requester (which from inspection seems to be much further along than I thought) will solve this problem :) True. I still think we need deterministic temp files so that several threads are not simultaneously trying to cache the same object

Re: mod_cache wishlist

2005-08-23 Thread Parin Shah
> > > > Content definitely should not be served from the cache after it has > > expired imo. However I think an approach like; > > > > if((now + interval) > expired) { > > if(!stat(tmpfile)) { > > update_cache_from_backend(); > > } > > } > > > > ie "revalidate the ca

Re: mod_cache wishlist

2005-08-23 Thread Colm MacCarthaigh
On Tue, Aug 23, 2005 at 08:42:48AM -0400, Brian Akins wrote: > Deterministic temp files to avoid "thundering herd": > > http://marc.theaimsgroup.com/?l=apache-httpd-dev&m=112430743432417&w=2 > > Especially Colm's comments: > > Content definitely should not be served from the cache after it has >

mod_cache wishlist

2005-08-23 Thread Brian Akins
With the talk of "direction" of Apache, I though I, as an end user and developer, would offer my "wish list" for mod_cache. We have been using squid for various things, but are now mostly using Apache plus our own custom cache module. Our module has grown to support most of the "cool" feature