Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-11-01 Thread Ruediger Pluem
On 10/27/2006 03:28 PM, [EMAIL PROTECTED] wrote: Author: minfrin Date: Fri Oct 27 06:28:56 2006 New Revision: 468373 URL: http://svn.apache.org/viewvc?view=revrev=468373 Log: mod_cache: Pass the output filter stack through the store_body() hook, giving each cache backend the ability to

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-11-01 Thread Ruediger Pluem
On 11/01/2006 02:30 PM, Graham Leggett wrote: On Wed, November 1, 2006 2:51 pm, Ruediger Pluem wrote: This breaks compilation of mod_disk_cache on systems that have APR_HAS_SENDFILE set. This was fixed in a commit shortly after. I have svn up to the latest revision and it is not

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h

2006-11-01 Thread Graham Leggett
On Wed, November 1, 2006 5:48 pm, Ruediger Pluem wrote: I have svn up to the latest revision and it is not fixed. If you still think that it is fixed please point me to the revision that should have fixed it and I will have a look if something is wrong on my side. If you do an svn log on

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h

2006-11-01 Thread Ruediger Pluem
On 11/01/2006 06:02 PM, Graham Leggett wrote: On Wed, November 1, 2006 5:48 pm, Ruediger Pluem wrote: I have svn up to the latest revision and it is not fixed. If you still think that it is fixed please point me to the revision that should have fixed it and I will have a look if something

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-30 Thread Issac Goldstand
Justin Erenkrantz wrote: I will also note that the mod_cache provider system has explicit versioning, so any modifications to the providers should be represented with a new version number. (i.e. providers for version 0 should work while offering new features in version 1-class providers.) We

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-30 Thread Justin Erenkrantz
On 10/30/06, Issac Goldstand [EMAIL PROTECTED] wrote: Can you clarify the above a bit? I don't understand what you're referring to. Looking at the 2.2.3 tag, what versioning is currently in place? Look at disk_cache_register_hook. /* cache initializer */ ap_register_provider(p,

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-30 Thread Issac Goldstand
Justin Erenkrantz wrote: On 10/30/06, Issac Goldstand [EMAIL PROTECTED] wrote: Can you clarify the above a bit? I don't understand what you're referring to. Looking at the 2.2.3 tag, what versioning is currently in place? Look at disk_cache_register_hook. /* cache initializer */

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-30 Thread Graham Leggett
On Mon, October 30, 2006 7:58 am, Justin Erenkrantz wrote: We have a real world case where the cache is expected to process a many MB or many GB file completely, before sending that same response to the network. This is too slow, and takes up too much RAM, resulting in a broken response to

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-30 Thread Nick Kew
On Mon, 30 Oct 2006 01:53:18 +0200 Graham Leggett [EMAIL PROTECTED] wrote: The current expectation that it be possible to separate completely the storing of the cached response and the delivery of the content is broken. Why is that? (references to previous discussion will do, if applicable)

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-30 Thread Graham Leggett
On Mon, October 30, 2006 12:57 pm, Nick Kew wrote: The current expectation that it be possible to separate completely the storing of the cached response and the delivery of the content is broken. Why is that? Because: - the cache_body() hook is expected to swallow an entire brigade

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-30 Thread Nick Kew
On Mon, 30 Oct 2006 14:03:03 +0200 (SAST) Graham Leggett [EMAIL PROTECTED] wrote: On Mon, October 30, 2006 12:57 pm, Nick Kew wrote: The current expectation that it be possible to separate completely the storing of the cached response and the delivery of the content is broken. Why is

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h

2006-10-30 Thread Graham Leggett
On Mon, October 30, 2006 2:44 pm, Nick Kew wrote: Hang on! Where's the file coming from? If it's local and static, what is mod_cache supposed to gain you? And if not, what put it in a (single) file bucket before it reached mod_cache? In the case of the person who reported this issue, the

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-30 Thread Justin Erenkrantz
On 10/30/06, Issac Goldstand [EMAIL PROTECTED] wrote: Looking at provider.c, a couple of questions spring to mind: 1) Why isn't this part of apr-util? (it seems similar to apr_optional.h - just intended for vtables rather than functions, and with this version info) apr_optional is an ugly hack

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-30 Thread Issac Goldstand
Justin Erenkrantz wrote: On 10/30/06, Issac Goldstand [EMAIL PROTECTED] wrote: Looking at provider.c, a couple of questions spring to mind: 1) Why isn't this part of apr-util? (it seems similar to apr_optional.h - just intended for vtables rather than functions, and with this version info)

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-29 Thread Justin Erenkrantz
On Fri, Oct 27, 2006 at 01:28:57PM -, [EMAIL PROTECTED] wrote: Author: minfrin Date: Fri Oct 27 06:28:56 2006 New Revision: 468373 URL: http://svn.apache.org/viewvc?view=revrev=468373 Log: mod_cache: Pass the output filter stack through the store_body() hook, giving each cache backend

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-29 Thread Graham Leggett
Justin Erenkrantz wrote: -1. This breaks the abstraction between the cache providers and the filter streams. The cache providers should not be in the business of delivering content down to the next filter - that is the job of mod_cache. Following this route is completely anti-thetical to the

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-29 Thread Justin Erenkrantz
On 10/29/06, Graham Leggett [EMAIL PROTECTED] wrote: The current expectation that it be possible to separate completely the storing of the cached response and the delivery of the content is broken. We have a real world case where the cache is expected to process a many MB or many GB file

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-27 Thread Davi Arnaut
[EMAIL PROTECTED] wrote: Author: minfrin Date: Fri Oct 27 06:28:56 2006 New Revision: 468373 URL: http://svn.apache.org/viewvc?view=revrev=468373 Log: mod_cache: Pass the output filter stack through the store_body() hook, giving each cache backend the ability to make a better decision as

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-27 Thread Graham Leggett
On Fri, October 27, 2006 4:38 pm, Davi Arnaut wrote: Where is pdconf ? Check out all those APR_HAS_SENDFILE. Aaargh... will fix. Those if( and if ( are going on my nerves! :) Damn, missed some. Swapping the metadata check would make the code much more readable: if (is_metadata)

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mod_mem_cache.c

2006-10-27 Thread Joe Orton
On Fri, Oct 27, 2006 at 11:38:02AM -0300, Davi Arnaut wrote: +/* Is our network connection still alive? + * If not, we must continue caching the file, so keep looping. + * We will return the error at the end when caching is done. +

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-27 Thread Niklas Edmundsson
On Fri, 27 Oct 2006, Graham Leggett wrote: On Fri, October 27, 2006 4:38 pm, Davi Arnaut wrote: Where is pdconf ? Check out all those APR_HAS_SENDFILE. Aaargh... will fix. The purpose of that code was originally to make EnableSendfile Off in the config file work. APR_HAS_SENDFILE only

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h

2006-10-27 Thread Graham Leggett
On Fri, October 27, 2006 6:05 pm, Niklas Edmundsson wrote: The purpose of that code was originally to make EnableSendfile Off in the config file work. APR_HAS_SENDFILE only tells you that APR has sendfile. It still does that as far as I can see: #if APR_HAS_SENDFILE core_dir_config

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h

2006-10-27 Thread Paul Querna
Graham Leggett wrote: On Fri, October 27, 2006 6:05 pm, Niklas Edmundsson wrote: The purpose of that code was originally to make EnableSendfile Off in the config file work. APR_HAS_SENDFILE only tells you that APR has sendfile. It still does that as far as I can see: #if APR_HAS_SENDFILE

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-27 Thread Graham Leggett
On Fri, October 27, 2006 6:00 pm, Joe Orton wrote: Err. We had the data in memory, we are going to read it back from disk again just in order to not block ? That's nonsense. Agreed. Please explain. This is a disk cache. Why would you write expensive bucket data to cache, and then expensive

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h

2006-10-27 Thread Graham Leggett
On Fri, October 27, 2006 6:14 pm, Paul Querna wrote: Honestly, I am -1 on any module needing knowledge of the core module setting. We should be making the abstraction better, not pulling things from the core config. This is true, but right now we don't have this level of abstraction, and I'm

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-27 Thread Niklas Edmundsson
On Fri, 27 Oct 2006, Graham Leggett wrote: Err. We had the data in memory, we are going to read it back from disk again just in order to not block ? That's nonsense. Agreed. Please explain. This is a disk cache. Why would you write expensive bucket data to cache, and then expensive bucket

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-27 Thread Davi Arnaut
Niklas Edmundsson wrote: On Fri, 27 Oct 2006, Graham Leggett wrote: Err. We had the data in memory, we are going to read it back from disk again just in order to not block ? That's nonsense. Agreed. Please explain. This is a disk cache. Why would you write expensive bucket data to cache,

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mo

2006-10-27 Thread Greg Marr
At 01:41 PM 10/27/2006, Davi Arnaut wrote: Niklas Edmundsson wrote: And when you have a file backend, you want to hit your disk cache and not the backend when delivering data to a client. People might think that this doesn't matter, but for large files, especially larger than RAM in your

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mo

2006-10-27 Thread Davi Arnaut
Greg Marr wrote: At 01:41 PM 10/27/2006, Davi Arnaut wrote: Niklas Edmundsson wrote: And when you have a file backend, you want to hit your disk cache and not the backend when delivering data to a client. People might think that this doesn't matter, but for large files, especially larger

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mo

2006-10-27 Thread Davi Arnaut
Davi Arnaut wrote: Greg Marr wrote: At 01:41 PM 10/27/2006, Davi Arnaut wrote: Niklas Edmundsson wrote: And when you have a file backend, you want to hit your disk cache and not the backend when delivering data to a client. People might think that this doesn't matter, but for large

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/

2006-10-27 Thread Graham Leggett
Davi Arnaut wrote: But that's a corner case. There is no reason in doing this for small files (common case). For example, in a enterprise grade server memory is cheap and permanent storage is slow and expensive. If permanent storage is slow and expensive, what on earth is the admin doing

Re: svn commit: r468373 - in /httpd/httpd/trunk: CHANGES modules/cache/mod_cache.c modules/cache/mod_cache.h modules/cache/mod_disk_cache.c modules/cache/mod_disk_cache.h modules/cache/mo

2006-10-27 Thread Graham Leggett
Davi Arnaut wrote: Because the data is already in memory. Why should I write something to disk, erase it from memory, and read it again shortly ? Why should I take care of something that is the job of the OS ? Why should I trash the VM constantly ? Because reading slowly from a cache disk is