Joe Orton wrote:
The existing 2.x store_body interface passed a brigade to the storage
provider's store_body() callback. It is impossible for the provider to
store all of such a brigade without consuming an arbitrary amount of
RAM, since the brigade may contain morphing buckets (a CGI/PIPE bucket
is the pathological case). This is bad.
Ways to fix this that I can think of:
1) change the store_body interface to allow the storage provider direct
access to f->next, so it can flush buckets up the output filter chain
when they have been stored. As seen on trunk.
2) keep the interface as-is, but read buckets in mod_cache and partition
the brigade manually; only pass a "small" brigade with known-length
buckets to the provider. (so no morphing and no arbitrary memory
consumption)
3) change the interface: deal with the buckets entirely in mod_cache and
just pass (char *,size_t) pairs to store_body
4) change the interface: pass some abstract "flush-me" callback in,
which the provider can call to pass up then delete the bucket.
(apr_brigade_flush doesn't quite fit the bill unfortunately)
How about implementing the providers as a filter (same API as normal
output filters, but separate chain just for mod_cache)? The final
filter in the chain destroys the buckets and the first can do any
"special handling" for special bucket types. Then in the existing cache
save filter, rather than passing the brigades to store_body as-is, we
would duplicate each bucket on the brigade, pass the original up the
output filter chain and then pass the duplicate up the cache chain (the
idea being to prioritize the client waiting; if it's a FILE bucket, the
network filter will SENDFILE it quickly, whereas the cache filters may
butcher the bucket with morphing, etc).
In addition to providing a more flexible API to the providers, this can
also help with the "stacked providers" idea that was mentioned a while
back[1].
Issac
[1] http://marc.theaimsgroup.com/?l=apache-httpd-dev&m=114614668505950&w=2
