On Tue, 2 May 2006, Plüm, Rüdiger, VF EITO wrote:
Another thing: I guess on systems with no mmap support the current
implementation
of mod_disk_cache will eat up a lot of memory if you cache a large local file,
because it transforms the file bucket(s) into heap buckets in this case.
Even if mmap is present I think that mod_disk_cache causes the file buckets
to be transformed into many mmap buckets if the file is large. Thus we do not
use sendfile in the case we cache the file.
Correct. When caching a 4.3GB file on a 32bit arch it gets so bad that
mmap eats all your address space and the thing segfaults. I initally
thought it was eating memory, but that's only if you have mmap
disabled.
I the case that a brigade only contains file_buckets it might be possible to
"copy" this brigade, sent it up the chain and process the copy of the brigade
for disk storage afterwards. Of course this opens a race if the file gets
changed in between these operations.
This approach does not work with socket or pipe buckets for obvious reasons.
Even heap buckets seem to be a somewhat critical idea because of the
added memory usage.
I did the somewhat naive approach of only doing background caching
when the buckets refer to a single sequential file. It's not perfect,
but it solves the main case where you get a huge amount of data to
store ...
/Nikke - stumbled upon more than one bug when digging into
mod_disk_cache
--
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se | [EMAIL PROTECTED]
---------------------------------------------------------------------------
Anything is edible if it's chopped finely enough
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
