Re: [ZODB-Dev] Re: Connection pool makes no sense
Tim Peters wrote: [Dieter Maurer] They did not tell us about their application. But, Zope's database adapters work like this. They use the ZODB cache (and its pool) as an RDB connection pool. Thus, if the ZODB caches are not released, the RDB connections won't. I believe you, but I'm not sure what to make of it; for example, 1. The OP is the only person who has this problem? 2. Other people have this problem too, and don't know what to do about it either, but never complain? 3. Other people have had this problem, but do know what to do about it? I am one of those people of point 2 and 3. This behaviour which I recognise of zope from years back, is still plaguing me with some apps we have. They still run zope2.6 and DCO2, so maybe some of the problems are aleviated in newer zopes, but I do recognise the problem description here. /dario -- -- --- Dario Lopez-Kästen, IT Systems Services Chalmers University of Tech. Lyrics applied to programming application design: emancipate yourself from mental slavery - redemption song, b. marley ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
Tim Peters wrote at 2006-1-2 15:05 -0500: I'm still baffled by how you get into this state to begin with. Florent explained why earlier, and I didn't see a reply to this part: [Florent Guillaume] By itself Zope never uses more than one connection per thread, and the number of thread is usually small. You replied that you had hundreds rather than thousands of connections, but it's a still mystery here how you manage to have that many. For example, Zope defaults to using 4 ZServer threads, so do you specify a much larger zserver-threads value than the default? I explained how Zope can aggregate much more connections when the maximal number of worker threads exceeds the size of the connection pool. Apparently, you missed this message. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
I'm still baffled by how you get into this state to begin with. Florent explained why earlier, and I didn't see a reply to this part: [Florent Guillaume] Huh are you sure? That would mean you have thousands of threads. Or hundreds or ZEO clients. Or hundreds of ZODB mountpoints. By itself Zope never uses more than one connection per thread, and the number of thread is usually small. You replied that you had hundreds rather than thousands of connections, but it's a still mystery here how you manage to have that many. For example, Zope defaults to using 4 ZServer threads, so do you specify a much larger zserver-threads value than the default? ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
FYI, I added code to clear the cache in _ConnectionPool at the time a closed Connection is forgotten. This will be in ZODB 3.6 final, which in turn will be in Zopes 2.9 and 3.2 final. I don't have time for more here now, so if others want more it's up to them ;-) ... P.S. Call me Pitcher, i don't like 'OP' name. No offense intended! OP is a traditional abbreviation of Original Poster. Your first message used a character set that rendered your name as ??? ? ? in my email client, and OP seemed a lot clearer than ??? ? ? :-) Happy New Year to everyone who is reading this!!! Best wishes to you! I wish you to be healthy and all your family members too! Long live ZODB! :) Best wishes to you to, Pitcher! ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
... [Tim] FYI, I added code to clear the cache in _ConnectionPool at the time a closed Connection is forgotten. This will be in ZODB 3.6 final, which in turn will be in Zopes 2.9 and 3.2 final. [Pitcher] Okay. Tank you. But it's good not always... You never said anything good or bad of the idle period. The idea of having this parameter doesn't seem to be useful for you? I simply haven't thought about it, and won't have time for that in the foreseeable future either. ZODB wasn't designed with hundreds of simultaneous connections in mind, and for that reason it's not surprising if odd things happen when you push it that way. IMO it's good to have a little connection buffer right after the site activity gets down so in case of next activity jump we have ready connections (may be with cache, to keep good application perfomance). And after we have stabilized site load, connection pool may shrink to the size that would solve the incoming requests (depending of database server load and etc.) What should be the idle period value - let user to decide. And no need to the pool_size parameter, or may be the list value of the connection pool to have ready to server connections. That's why subj was about connection poll that makes no sense in that architecture. Connection management wasn't designed with this use case in mind at all. Zope typically makes very modest demands on # of connections, and non-Zope ZODB uses typically even less. In both, it's considered to be a problem in the app if the app _ever_ opens more than pool_size connections simultaneously (that's why warning and critical log messages are made when the app does), it's not expected that pool_size will be set to something much larger than the default 7, and keeping caches around is important. Didn't mean to be rude... Sorry if i sounded offensive. No offense taken! I don't mean to sound dismissive either -- you may (or may not) have good ideas here, but I simply don't have time to think about them either way. Hence: I don't have time for more here now, so if others want more it's up to them ;-) We have discussion that long that would be enough to rewrite Connection, DB, and utils classes/modules totaly... I would be good testing case. I've implemented idle logic as i could without deep knowledge of zodb. And next time we upgrade our system to a new version - will have to rewrite this logic again. This is the right list to discuss such changes. It would probably help a lot if more people who cared about these use cases volunteered to help. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
David Rushby wrote at 2005-12-30 11:14 -0800: ... Since ZODB doesn't know whether the connection it's releasing from the pool is still in use, I don't know whether resetting the connection's cache is appropriate as a general solution. But it fixes a definite problem with Zope's behavior as a ZODB client application. It is appropriate because the connection is closed (and therefore cannot be used any longer) and it cannot be opened again (as opening is not a Connection but a DB method). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
Tim Peters wrote at 2005-12-30 14:51 -0500: [Dieter Maurer] They did not tell us about their application. But, Zope's database adapters work like this. They use the ZODB cache (and its pool) as an RDB connection pool. Thus, if the ZODB caches are not released, the RDB connections won't. I believe you, but I'm not sure what to make of it; for example, 1. The OP is the only person who has this problem? 2. Other people have this problem too, and don't know what to do about it either, but never complain? I expect (but the original poster may step in) that the problem could occur in Zope only when the number of threads exceeds the pool size (or additional connections are used in an application specific way) as otherwise, there are no dropped connections. Because formerly, it made no sense to have more worker threads than that given by the pool size, this situation is likely to occur rarely. ... If not, you may have better luck on the zope-db list (which is devoted to using other databases with Zope): The problem is not with the RDB but with the ZODB connections that are magically not garbage collected. He will probably not find help on zope-db. That suggestion was based on a guess that #3 (above) is most likely. Of course I don't know, but #1 and #2 seem unlikely on the face of it. If other people using RDB don't have this problem, then zope-db is the right place to ask how they manage to avoid it. If the poster has no evidence that the ZODB connections are definitely kept but just see that the relational database connections remain open, the reason might indeed be at a completely different place: There are some reports on zope-db about DAs leaking relational database connections. The problem is not related to ZODB connection handling. Instead, the relational database connection is kept open even if the DA object was invalidated (and especially cleared). We observed such behaviour with Zope 2.7/ZODB 3.2 and ZPsycopgDA. When we used resetCaches (which should in principle release the old caches (I also added an minimizeCache because the cyclic gc does not work with Zope 2.7's ExtensionClass objects)), a new set of Postgres connections was opened without the old ones being closed. We worked around this problem by: * avoiding to use resetCaches * restarting Zope once a weak to get rid of stale Postgres connections ... OTOH, if that's not what's going on here, I'd expect to have heard about this here more than once in the last 5 years ;-) The older ZODB code (ZODB 3.2 and before) was seriously flawed with respect to cache release handling. Fortunately, it was very rare that caches need to be released. I found the flaws because we used temporary connections extensively and, of course, their caches need to go away with the temporary connection. I had a hard fight to get rid of the memory leaks induced by those flaws. Now (ZODB 3.4 and above) caches might get released more often. True, the garbage collector now has a chance to collect cycles including ExtensionClass objects, but it is very easy to defeat the GC -- an object with a __del__ method is sufficient. Perhaps because the OP is unique in allowing hundreds (or thousands -- whatever) of Connections to be active simultaneously? Don't know. He did not say that. If his hypothesis is indeed true and some connections exceeding the pool size are kept indefinitely, then already slightly exceeding the pool size may lead to an unbound number of connections provided the exceedance occurs frequently enough. I suggested before that forcing calls to gc.collect() would give more evidence. If that doesn't help, then it's most likely that the application is keeping Connections alive. Since I'm not familiar with the RDB code, I suppose it's even possible that such code uses __del__ methods, and creates cycles of its own, that prevent cyclic gc from reclaiming them. In that case, there are serious problems in that code. There is also another gc attribute holding the garbage cycles not released. The poster may examine them to check whether they contain indeed ZODB connection objects. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
Tim Peters wrote at 2005-12-29 12:59 -0500: ... [Tim Peters] It means that _ConnectionPool no longer has a reason to remember anything about that Connection. Application code can continue keeping it alive forever, though. [Denis Markov] But what about RDB-Connection what stay in cache forever? Sorry, I don't know anything about how your app uses RDB connections. ZODB isn't creating them on its own ;-) They did not tell us about their application. But, Zope's database adapters work like this. They use the ZODB cache (and its pool) as an RDB connection pool. Thus, if the ZODB caches are not released, the RDB connections won't. ... Nothing can be done to _force_ Connections to go away forever. But most of their resources could be freed when they are removed from the pool. I definitely would do this -- in the sense of defensive programming. Leaking the connections is much more critical than the all attribute growth... ... as a result we get many RDB-Connections what will never use but hang our RDB At this point I have to hope that someone else here understands what you're doing. I can recognize standard Zope behaviour -- although other applications might show similar behaviour... If not, you may have better luck on the zope-db list (which is devoted to using other databases with Zope): The problem is not with the RDB but with the ZODB connections that are magically not garbage collected. He will probably not find help on zope-db. Some hints how to analyse garbage collection problems might help him. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
[Dieter Maurer] They did not tell us about their application. But, Zope's database adapters work like this. They use the ZODB cache (and its pool) as an RDB connection pool. Thus, if the ZODB caches are not released, the RDB connections won't. I believe you, but I'm not sure what to make of it; for example, 1. The OP is the only person who has this problem? 2. Other people have this problem too, and don't know what to do about it either, but never complain? 3. Other people have had this problem, but do know what to do about it? ... If not, you may have better luck on the zope-db list (which is devoted to using other databases with Zope): The problem is not with the RDB but with the ZODB connections that are magically not garbage collected. He will probably not find help on zope-db. That suggestion was based on a guess that #3 (above) is most likely. Of course I don't know, but #1 and #2 seem unlikely on the face of it. If other people using RDB don't have this problem, then zope-db is the right place to ask how they manage to avoid it. Some hints how to analyse garbage collection problems might help him. Alas, the #1 cause for garbage collection problems is an application keeping objects alive that the author thinks, or just assumes, should be trash. IOW, there usually isn't a problem with garbage collection when one is _suspected_, because the disputed objects are in fact not trash. If that's what's going on here, mucking with ZODB might soften the symptoms but without curing the underlying application problem. OTOH, if that's not what's going on here, I'd expect to have heard about this here more than once in the last 5 years ;-) Perhaps because the OP is unique in allowing hundreds (or thousands -- whatever) of Connections to be active simultaneously? Don't know. I suggested before that forcing calls to gc.collect() would give more evidence. If that doesn't help, then it's most likely that the application is keeping Connections alive. Since I'm not familiar with the RDB code, I suppose it's even possible that such code uses __del__ methods, and creates cycles of its own, that prevent cyclic gc from reclaiming them. In that case, there are serious problems in that code. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
[Tim Peters] ... I suggested before that forcing calls to gc.collect() would give more evidence. If that doesn't help, then it's most likely that the application is keeping Connections alive. [David Rushby] I'm not the OP, but I was able to reproduce the problem he was encountering. Calls to gc.collect() *do* solve the problem, without any changes to the application. Thank you! That's progress. But mightn't clearing the released ZODB Connection's cache (http://mail.zope.org/pipermail/zodb-dev/2005-December/009688.html ) be a better solution? I didn't suggest it as a solution, but as a way to get evidence. Now I'd like to know whether adding gc.collect() solves the OP's problem, in the OP's original context, too. I expect that clearing the cache when _ConnectionPool forgets a Connection is harmless, but that will take more thought to be sure, and in any case it doesn't address that trash Connection objects, and trash caches, would still accumulate. If possible, I would rather fix the underlying problem than just relieve the symptom du jour. For example, maybe it's a bug in Python's gc after all, or maybe it would be better to break the reference cycles when _ConnectionPool forgets a Connection (so that the trash gets reclaimed right then, instead of waiting for a gc collection cycle to reach the generation in which the trash lives). ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Connection pool makes no sense
В Чтв, 29/12/2005 в 11:30 +0100, Florent Guillaume пишет: A little bit of history... We have zope as an application server for heavy loaded tech process. We have high peaks of load several times a day and my question is about how can we can avoid unused connections to remain in memory after peak is passed? Before ZODB-3.4.1 connection pool was fixed size of pool_size and that caused zope to block down while load peaks. ZODB-3.4.2 that is shipped with Zope-2.8.5 has connection pool that does not limit the opened connections, but tries to reduce the pool to the pool_size and this behavior is broken IMO. Follow my idea... After peak load I have many (thousands of connections) that have cached up different objects including RDB connections. Hundreds... my mistake. Huh are you sure? That would mean you have thousands of threads. Or hundreds or ZEO clients. Or hundreds of ZODB mountpoints. By itself Zope never uses more than one connection per thread, and the number of thread is usually small. If you see many RDB connections, then it's a RDB problem and not a ZODB problem. Something not releasing RDB connections quick enough, or leaking RDB connections. Not agree. Can you answer the question? Does self.all.remove(c) mean that we WANT to destroy connection instance? If not then where in ZODB source code i can see connection destruction? Clearing cache and calling _v_database_connection.close() method? You've just caught me on thousands but gave no comments on deletion of connection instances... but this is the clue to the topic. Florent --== *** ==-- Заместитель директора Департамента Информационных Технологий Юдыцкий Игорь Владиславович ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
[Florent Guillaume] ... The self.all.remove(c) in _ConnectionPool attempts to destroy the connection. Nope, it's simply getting rid of a weak reference that no longer serves a purpose, to avoid unbounded growth of the .all set in case of ill-behaved application code, and to speed Python's cyclic gc a little. Removing the Connection from .available removed _ConnectionPool's only strong reference to the Connection. If something else has a reference to it once it's closed, then that's a bug, and it shouldn't. Yup! ... Even hundreds of ZODB connections is absurd. I'd settle for calling it uncommon and unexpected. Again, with 4 threads you should never get more than 4 Filestorage connections plus 4 TemporaryStorage connections. Bears repeating ;-) ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
...
[Florent Guillaume]
The self.all.remove(c) in _ConnectionPool attempts to destroy the
connection. If something else has a reference to it once it's closed,
then that's a bug, and it shouldn't. It should only keep a weak
reference to it at most.
[EMAIL PROTECTED]
But it's nonsense!
Please try to remain calm here. It's not nonsense, but if you're screaming
too loudly you won't be able to hear :-)
If weakref exists then some other object has ref to the obj!
Or there are no strong reference to `obj`, but `obj` is part of cyclic
garbage so _continues to exist_ until a round of Python's cyclic garbage
collection runs.
And weakValueDictionary is cleaned up automatically when the
last strong ref disappears.
That's a necessary precondition, but isn't necessarily sufficient. When the
last strong reference to a value in a weakValueDictionary goes away, if that
value is part of cyclic garbage then the weakValueDictionary does not change
until Python's cyclic gc runs.
Destroying obj with this logic is absurd:
I covered that before, so won't repeat it. You misunderstood the intent of
this code.
...
del self.data[id(obj)] == there is no use to delete obj by
deleting weakref... we just deleting weakref from the weakValueDictionary!
Yes, it's just deleting the weakref -- and that's all it's trying to do, and
there are good reasons to delete the weakref here (but are not the reasons
you thought were at work here).
Try this: 1. add this method to Connection class definition
def __del__(self):
print 'Destruction...'
then do this:
You're _really_ going to confuse yourself now ;-) Because Connections are
always involved in reference cycles, adding a __del__ method to Connection
guarantees that Python's garbage collection will _never_ reclaim a
Connection (at least not until you explicitly break the reference cycles).
import sys
sys.path.append('/opt/Zope/lib/python')
from ZODB import Connection
c = Connection.Connection()
del(c)
c = Connection.Connection()
del(c._cache)
You're breaking a reference cycle by hand here, so that it becomes
_possible_ for gc to clean up the Connection. But the only reason that was
necessary is because you added a __del__ method to begin with.
del(c)
Destruction...
See? You can NOT delete object because _cache keeps reference to it...
and connection remains forever!!!
That's because you added a __del__ method; it's not how Connection normally
works. I'll give other code below illustrating this.
It's cache has RDB connection objects and they are not closed. Connection
becomes inaccessible and unobtainable trough the connection pool.
In your code above, `c` was never in a connection pool. You're supposed to
get a Connection by calling DB.open(), not by instantiating Connection()
yourself (and I sure hope you're not instantiating Connection() directly in
your app!).
That's what I wanted to say. It's definitely a BUG.
Sorry, there's no evidence of a ZODB bug here yet.
Consider this code instead. It opens 10 Connections in the intended way
(via DB.open()), and creates a weakref with a callback to each so that we
can tell when they're reclaimed. It then closes all the Connections, and
destroys all its strong reference to them:
import weakref
import gc
import ZODB
import ZODB.FileStorage
class Wrap:
def __init__(self, i):
self.i = i
def __call__(self, *args):
print Connection #%d went away. % self.i
N = 10
st = ZODB.FileStorage.FileStorage('blah.fs')
db = ZODB.DB(st)
cns = [db.open() for i in xrange(N)]
wrs = [weakref.ref(cn, Wrap(i)) for i, cn in enumerate(cns)]
print closing connections
for cn in cns:
cn.close()
print del'ing cns
del cns # destroy all our hard references
print invoking gc
gc.collect()
print done
This is the output:
closing connections
del'ing cns
invoking gc
Connection #0 went away.
Connection #1 went away.
Connection #2 went away.
Done
Note that nothing happens before Python's cyclic gc runs. That's because
Connections are in reference cycles, and refcounting cannot reclaim objects
in trash cycles. Because I used weakref callbacks instead of __del__
methods, cyclic gc _can_ reclaim Connections in trash cycles.
When the 10 Connections got closed, internally _ConnectionPool added them,
one at a time, to its .available queue. When #7 was closed, the pool grew
to 8 objects, so it forgot everything it knew about the first Connection
(#0) in its queue. Nothing happens then, though, because nothing _can_
happen before cyclic gc runs. When #8 was closed, #1 got removed from
.available, and when #9 was closed, #2 got removed from .available.
When gc.collect() runs, those 3 Connections (#0, #1, and #2) are all
reclaimed. The other 7 Connections (#3-#9) are still alive, sitting in the
.available queue waiting to be reused.
___
For more information about ZODB, see the ZODB Wiki:
RE: [ZODB-Dev] Re: Connection pool makes no sense
Oops! I sent this to zope-dev instead of zodb-dev by mistake. [EMAIL PROTECTED] Not agree. Can you answer the question? Does self.all.remove(c) mean that we WANT to destroy connection instance? [Tim Peters] It means that _ConnectionPool no longer has a reason to remember anything about that Connection. Application code can continue keeping it alive forever, though. [Denis Markov] But what about RDB-Connection what stay in cache forever? Sorry, I don't know anything about how your app uses RDB connections. ZODB isn't creating them on its own ;-) On the next peak load we get some next ZODB-Connections with RDB-Connection After repush() old ZODB-Connections will be killed (if pool_size) I don't like the word killed here, because it seems highly misleading. ZODB doesn't destroy any Connections or any caches. ZODB destroys all its strong references to old Connections, and that's all. Nothing can be done to _force_ Connections to go away forever. It's ZODB's job here to make sure it isn't forcing Connections (beyond the pool_size limit) to stay alive, and it's doing that job. It can't kill Connections. but RDB-Connection stay in cache forever And so on There's one cache per Connection. If and when a Connection goes away, its cache goes away too. So when you say something stays in cache forever, I don't know what you mean -- you apparently have many (hundreds? thousands?) of Connections, in which case you also have many (hundreds or thousands) of caches. I don't know how an RDB-Connection gets into even one of those caches to begin with. as a result we get many RDB-Connections what will never use but hang our RDB At this point I have to hope that someone else here understands what you're doing. If not, you may have better luck on the zope-db list (which is devoted to using other databases with Zope): http://mail.zope.org/mailman/listinfo/zope-db ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
Tim Peters wrote at 2005-12-29 11:28 -0500: ... Or there are no strong reference to `obj`, but `obj` is part of cyclic garbage so _continues to exist_ until a round of Python's cyclic garbage collection runs. And this is *VERY* likely as any persistent object in the cache has a (strong, I believe) reference to the connection which in turn references any of these objects indirectly via the cache. In my view, closed connections not put back into the pool should be explicitely cleaned e.g. their cache cleared or at least minimized. If for some reason, the garbage collector does not release the cache/cache content cycles, then the number of connections would grow unboundedly which is much worse than an unbound grow of the all attribute. Pitcher seem to observe such a situation (where for some unknown reason, the garbage collector does not collect the connection. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] Re: Connection pool makes no sense
[Tim] ... Or there are no strong reference to `obj`, but `obj` is part of cyclic garbage so _continues to exist_ until a round of Python's cyclic garbage collection runs. [Dieter Maurer] And this is *VERY* likely as any persistent object in the cache has a (strong, I believe) reference to the connection which in turn references any of these objects indirectly via the cache. I'm not sure I follow: it's not just very likely that Connections end up in cycles, it's certain that they do. The small test code I posted later should make that abundantly clear. They end up in cycles even if they're never used: call DB.open(), and the Connection it returns is already in a cycle (at least because a Connection and its cache each hold a strong reference to the other). In my view, closed connections not put back into the pool That never happens: when an open Connection is closed, it always goes back into the pool. If that would cause the configured pool_size to be exceeded, then other, older closed Connections are removed from the pool to make room. It's an abuse of the system for apps even to get into that state: that's why ZODB logs warnings if pool_size is ever exceeded, and logs at critical level if it's exceeded a lot. Connections should be viewed as a limited resource. should be explicitely cleaned e.g. their cache cleared or at least minimized. The code that removes older Connections from the pool doesn't do that now; it could, but there's no apparent reason to complicate it that I can see. If for some reason, the garbage collector does not release the cache/cache content cycles, then the number of connections would grow unboundedly which is much worse than an unbound grow of the all attribute. There's a big difference, though: application code alone _could_ provoke unbounded growth of .all without the current defensive coding -- that doesn't require hypothesizing Python gc bugs for which there's no evidence. If an application is seeing unbounded growth in the number of Connections, it's a Python gc bug, a ZODB bug, or an application bug. While cyclic gc may still seem novel to Zope2 users, it's been in Python for over five years, and bug reports against it have been very rare -- most apps stopped worrying about cycles years ago, and Zope3 has cycles just about everywhere you look. ZODB isn't a pioneer here. I ran stress tests against ZODB a year or so ago (when the new connection management code was implemented) that created millions of Connections, and saw no leaks then, regardless of whether they were or weren't explicitly closed. That isn't part of the test suite because it tied up a machine for a day ;-), but nothing material has changed since then that I know of. It's possible a new leak got introduced, but I'd need more evidence of that before spending time on it; the small test code I posted before showed that at least that much still works as designed, and that hit all the major paths thru the connection mgmt code. Pitcher seem to observe such a situation (where for some unknown reason, the garbage collector does not collect the connection. I don't believe we have any real idea what they're doing, beyond that something somewhere is sticking around longer than they would like. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
