On Tue, May 02, 2006 at 02:21:27PM +0200, Plüm, Rüdiger, VF EITO wrote: > Another thing: I guess on systems with no mmap support the current > implementation > of mod_disk_cache will eat up a lot of memory if you cache a large local file, > because it transforms the file bucket(s) into heap buckets in this case. > Even if mmap is present I think that mod_disk_cache causes the file buckets > to be transformed into many mmap buckets if the file is large. Thus we do not > use sendfile in the case we cache the file. > I the case that a brigade only contains file_buckets it might be possible to > "copy" this brigade, sent it up the chain and process the copy of the brigade > for disk storage afterwards. Of course this opens a race if the file gets > changed in between these operations. > This approach does not work with socket or pipe buckets for obvious reasons. > Even heap buckets seem to be a somewhat critical idea because of the added > memory usage.
The way I would expect it to work would be by passing f->next in to the store_body callback, it looks doomed to eat RAM as currently designed. mod_disk_cache's store_body implementation can then do: 1. read bucket(s) from brigade, appending to some temp brigade 2. write bucket(s) in temp brigade to cache file 3. pass temp brigade on to f->next 4. clear temp brigade to ensure memory is released 5. goto 1 joe
