On Fri, 2004-03-12 at 04:21, Bojan Smojver wrote:
> Quoting Mark Rowe <[EMAIL PROTECTED]>:
FYI, I'll only speak for APR/Apache 2.0. 1.3 has a somewhat different
implementation.
> > Check out this further pool test,
> [...]
> > apr_pool_create_ex(&subp1, p, fun, NULL);
> > apr_pool_create_ex(&subp2, p, fun, NULL);
> > for (i = 0; i < 10000; i++) {
> > apr_pool_clear(subp1);
> > s = apr_pstrcat(subp1, s, tenbyte_string, NULL);
> > apr_pool_clear(subp2);
> > s = apr_pstrdup(subp2, s);
>
> The the above duplicates s, so now we have 220,000 raw storage requirement,
> given there are two pools in play. At least I think.
Let's disect this shall we? :)
> apr_pool_create_ex(&subp1, p, fun, NULL);
Create a subpool of p (which happens to be the request pool, but that doesn't
really matter here), sharing the allocator of p.
> apr_pool_create_ex(&subp2, p, fun, NULL);
Ditto.
> for (i = 0; i < 10000; i++) {
> apr_pool_clear(subp1);
Clear the subpool, returning all of it's memory, save 8k, to the allocator.
> s = apr_pstrcat(subp1, s, tenbyte_string, NULL);
Allocate a new string out of subp1, of length s + tenbyte_string. Note
that once len(s) passes the 8k mark, we are always having to get memory
from the allocator in this case. In the allocator we look for a block
of the suitable size (always multiples of 4k), and return that. Note
that the smaller blocks are just sitting there, waiting to be reused.
> apr_pool_clear(subp2);
Return all the memory of subp2 to the allocator, save 8k.
> s = apr_pstrdup(subp2, s);
And allocate a new string out of subp2 of length s. Note that this
will get another block from the allocator, the same size as allocated
a few lines above for subp1. These blocks are in use at the same time,
so no opportunity for reuse.
> > }
> > apr_pool_destroy(subp1);
> > apr_pool_destroy(subp2);
And destroy both subpools, returning all their memory to the allocator
for reuse.
> Isn't there some sort of algorithm in play that says:
>
> - allocate contiguousregion of memory
> - fit stuff into that region
> - if it doesn't fit, allocate twice the size (or something)
> - fit stuff into that other region if first doesn't have enough space
> - and so on
That would be the algorithm employed by apr_psprintf and friends.
> This might explain why the memory usage is soaring as you keep increasing the
> string size.
The memory usage is soaring because no memory is being reused.
> I'm not sure why memory isn't returned to the system on apr_pool_destroy().
> Maybe it simply gets pasted to the parent pool in case that one needs more
> memory in the future?
The memory is returned to the pools allocator. All pools sharing an
allocator can reuse memory from there. The allocator will return memory
it is holding, over a certain threshold, to the system. In the Apache
HTTP Server 2.0, the MaxMemFree directive can be used to influence the
threshold. The default is unlimited (except on NetWare IIRC).
HTH,
Sander
