Tim Peters wrote at 2005-6-27 15:38 -0400: > ... [Dieter] >> Probably, "ZEOClientStorage" (and the ZEO server) should use >> "SO_KEEPALIVE" to enable TCP keepalive messages. However, the default >> TCP timeouts are probably too high (2 hours) for many ZODB applications >> (like ours).
[Tim] >Section 22.214.171.124 of RFC 1122 doesn't read like it thinks keep-alives are >either portable or appropriate for this. In particular, > > It is extremely important to remember that ACK segments that > contain no data are not reliably transmitted by TCP. > Consequently, if a keep-alive mechanism is implemented it > MUST NOT interpret failure to respond to any specific probe > as a dead connection. [Dieter] Then hopefully, TCP knows that and observes it in the implementation of its "keep-alive" mechanism. At least, it is quite careful (with Linux default parameters): Only after 9 keep alive messages are lost, the connection is considered broken. >... [Dieter] >> I will implement an application specific keep alive mechanism. [Tim] >How will you do that? I attach the script (wrapped up as a Zope "product"). > Would it be appropriate for ZEO to do something >similar itself? Yes. It would be very nice. > If ZEO did, would you still need to bother? No. And I would be happy, because I am using internal undocumented implementation details that already once changed between ZEO releases. -- Dieter
# $Id: __init__.py,v 126.96.36.199 2005/06/27 08:37:21 dieter Exp $ '''ZEOKeepAlive. Our cluster checks made us aware that 'ZEOClientStorage' depends on the operating system to inform about connection breaks and that this cannot do this reliably during inactivity periods. This product starts a thread that periodically sends a keep alive message to all ZEO connections. Sending will fail when the connection is no longer intact triggering 'ZEOClientStorage's automatic reconnect. The thread is controlled by the environment variable 'ZEO_KEEPALIVE_PERIOD'. If not specified, the keep alive thread is not started. For use by non Zope ZEO clients, it defines the function 'startThread'. It has the optional argument 'period' specifying the period (default: use value given by environment variable 'ZEO_KEEPALIVE_PERIOD'). As in 'ZEOClientStorage', we assume that no communication will hang, not even under abnormal conditions. The solution would be much more complex had we not made this assumption. Logs We log as 'INFO' whether or not we start the keep alive thread. We log as 'WARNING' unready storages We log as 'BLATHER' send and response events for keep alive messages ''' from os import environ from time import sleep from thread import start_new_thread from zLOG import LOG, INFO, WARNING, BLATHER, ERROR, PANIC from App.config import getConfiguration from ZEO.ClientStorage import ClientStorage _period = None def startThread(period=None): global _period if _period: raise ValueError('KeepAlive thread already started') if period is None: period = environ.get('ZEO_KEEPALIVE_PERIOD') if period is not None: period = int(period) period = _period = period and period > 0 and period _log(INFO, period and 'Thread started with period %s sec' % period or 'Thread not started' ) if period: start_new_thread(_keepAlive, (period,)) def initialize(context): startThread() def _keepAlive(period): try: while True: sleep(period) dbs = getConfiguration().dbtab for db in dbs.opened.values(): S = db._storage if isinstance(S, ClientStorage): name = S.getName() if S._ready.isSet(): _log(BLATHER, "Sending keep alive for %s" % name) S._load_lock.acquire() try: try: S._server.get_info() _log(BLATHER, "Keep alive for %s successful" % name) except: _log(ERROR, "Keep alive for %s failed" % name, error=True) finally: S._load_lock.release() else: _log(WARNING, "Connection for %s is down" % name) except: _log(PANIC, "Exception in keep alive thread", error=True) def _log(*args, **kw): LOG('ZEO KeepAlive', *args, **kw)
_______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev