Caching incomplete responses was Re: re-do of proxy request body handling - ready for review

2005-02-03 Thread Justin Erenkrantz
On Thu, Feb 03, 2005 at 09:06:04AM +0200, Graham Leggett wrote: Justin Erenkrantz wrote: I don't see any way to implement that cleanly and without lots of undue complexity. Many dragons lay in that direction. When I put together the initial framework of mod_cache, solving this problem

Re: re-do of proxy request body handling - ready for review

2005-02-03 Thread Jeff Trawick
On Wed, 02 Feb 2005 14:39:37 -0800, Justin Erenkrantz [EMAIL PROTECTED] wrote: Anyhow, only one really minor nit: in spool_reqbody_cl, I think it'd be good to add a comment that we're intentionally leaving the first 16k (or so) of content in our memory buffer. That took me a little bit to

Re: re-do of proxy request body handling - ready for review

2005-02-03 Thread Jim Jagielski
On Feb 2, 2005, at 8:40 AM, Jeff Trawick wrote: Please review the proxy-reqbody branch for proposed improvements to 2.1-dev. There is a 2.0.x equivalent of the patch at http://httpd.apache.org/~trawick/20proxyreqbody.txt. 2.1 version reviewed and tested. Wow. V. nice! +1 --

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Greg Ames
Jeff Trawick wrote: Please review the proxy-reqbody branch for proposed improvements to 2.1-dev. There is a 2.0.x equivalent of the patch at http://httpd.apache.org/~trawick/20proxyreqbody.txt. +1 (reviewed, not tested) certainly an improvement over what we have today. The brains (decisions

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Ronald Park
I was recently considering a similar patch for mod_proxy along the lines of spool_reqbody_cl() method but it would go one step further: spawning off a thread to asynchronously read the request into a temp file (or files) while the initial thread would continue to stream the io_bufsize chunks down

Multi-threaded proxy? was Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Paul Querna
Ronald Park wrote: I was recently considering a similar patch for mod_proxy along the lines of spool_reqbody_cl() method but it would go one step further: spawning off a thread to asynchronously read the request into a temp file (or files) while the initial thread would continue to stream the

Re: Multi-threaded proxy? was Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Ronald Park
Hmm, don't know enough about Event MPM to comment on that part of the message, but with regards to the performance for small requests, in my original 'design' for this, I figured it would do one synchronous read first, pass that through the filter chain and, if '(!seen_eos)', then it would pay the

Re: Multi-threaded proxy? was Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Mladen Turk
Paul Querna wrote: One thought I have been tossing around for a long time is tying the proxy code into the Event MPM. Instead of a thread blocking while it waits for a backend reply, the backend server's FD would be added to the Event Thread, and then when the reply is ready, any available

Re: Multi-threaded proxy? was Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Ronald Park
Imagine, just as a wild theoretical scenario (:D), that you have the following setup: Apache - (proxy) - Squid - (cache miss) - Apache - (docroot) Where the back-end Apache serves up large files (in the 2G range) (and, yes, there are far more files that can be effectively cached). Now imagine

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Jeff Trawick
On Wed, 02 Feb 2005 12:32:39 -0500, Ronald Park [EMAIL PROTECTED] wrote: I was recently considering a similar patch for mod_proxy along the lines of spool_reqbody_cl() method but it would go one step further: spawning off a thread to asynchronously read the request into a temp file (or files)

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Ronald Park
True, the client and the proxying server are tied up intrinsically. I used the wrong wording to name who would benefit. I meant that the server providing the proxied content (the 'other' server; the one not directly talking to the client) could be done and on it's way doing other work while the

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Graham Leggett
Ronald Park wrote: I was recently considering a similar patch for mod_proxy along the lines of spool_reqbody_cl() method but it would go one step further: spawning off a thread to asynchronously read the request into a temp file (or files) while the initial thread would continue to stream the

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Graham Leggett
Ronald Park wrote: In this picture: C -- A -- P C, the client, makes a request to A which proxies it to P. A P can do their exchange independant of C A. If A-P can be done fast, but C-A is slow, then P's wasting resource, no? If you configured mod_cache, my understanding is that P would

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Justin Erenkrantz
--On Wednesday, February 2, 2005 8:49 PM +0200 Graham Leggett [EMAIL PROTECTED] wrote: Mod_cache already supports the concept of spooling files to disk (or memory, or shared memory), and can be taught how to serve an incompletely downloaded file to other clients (apparently it cannot at the

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Graham Leggett
Justin Erenkrantz wrote: I don't understand the purpose of serving incomplete files from a cache. Can you please elaborate on what you think mod_cache should do? -- justin Since the early days of mod_proxy, there has been a race condition in the caching code which has been carried over to

RE: re-do of proxy request body handling - ready for review

2005-02-02 Thread Sander Striker
-Original Message- From: Graham Leggett [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 02, 2005 10:39 PM To: dev@httpd.apache.org Subject: Re: re-do of proxy request body handling - ready for review Justin Erenkrantz wrote: I don't understand the purpose of serving

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Justin Erenkrantz
--On Wednesday, February 2, 2005 11:38 PM +0200 Graham Leggett [EMAIL PROTECTED] wrote: If mod_cache was taught to serve a being-cached URL directly from the cache (shadowing the real download), there would be no need for parallel connections to the backend server while the file is being cached,

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Justin Erenkrantz
--On Wednesday, February 2, 2005 8:40 AM -0500 Jeff Trawick [EMAIL PROTECTED] wrote: Please review the proxy-reqbody branch for proposed improvements to 2.1-dev. There is a 2.0.x equivalent of the patch at http://httpd.apache.org/~trawick/20proxyreqbody.txt. To help people review Jeff's

Re: re-do of proxy request body handling - ready for review

2005-02-02 Thread Graham Leggett
Justin Erenkrantz wrote: I don't see any way to implement that cleanly and without lots of undue complexity. Many dragons lay in that direction. When I put together the initial framework of mod_cache, solving this problem was one of my goals. How do we know when another worker has already