On 8 December 2010 13:28, Jim Fulton <j...@zope.com> wrote:
>> With 20 active threads, each having rendered the Plone 4 front page,
>> this approach reduced the memory usage with 70 MB.
> Out of a total of what?
In my case out of 430 MB non-shared for the process.
> Note that if a process is CPU bound (as most dynamic Python apps
> should be), then there is little or no benefit in having multiple
> threads, due to the (damn) GIL.
The case I'm thinking of is when one thread is being used in a write
transaction, while another is doing a read.
If the database is bigger than the allowed memory usage, then I guess
threads can also ensure that requests for in-memory objects can be
served while some threads are blocked due to swapping and/or reading
pickles from disk.
> Except that you can't create wekrefs to strings or unicode.
I see. Maybe another scheme could be devised.
> Also, while interning is fine for an experiment, it's wasteful for
> strings that are rarely needed.
How so? As far as I can see, interning is still subjected to reference
counting. The only real difference is that a hash table is maintained
(fairly minimal memory use + probable computation of string hash).
> Sharing immutable data between threads is very appealing
> intellectually. I've certainly thoughtr about it a lot. In practice,
> I doubt the benefit will be worth the extra overhead (let alond the
> effort :).
I think if the case can be made for threading, then it's worth
pursuing. Alternatively, applications might put all non-trivial
strings into blobs, but I don't know if there's a non-trivial overhead
with that approach.
For more information about ZODB, see the ZODB Wiki:
ZODB-Dev mailing list - ZODB-Dev@zope.org