[Zope-dev] zc.async.queue.DispatcherAgents last_ping

2011-06-29 Thread Hanno Schlichting
Hi Gary, or any other zc.async developer,

we use zc.async in one site and I noticed there's a hell of a lot of
write transactions caused by it.

Every 30 seconds there's a zope.minmax.Maximum() by the name of
last_ping stored. It looks like this is coming from
zc.async.queue.DispatcherAgents and the 30 seconds are a result of a
class variable stating: ping_interval =
datetime.timedelta(seconds=30)

I tried to understand what this last_ping is used for and it seems to
have to do with detection of dead workers.

Is that indeed the only thing it's used for? If so can we change it in
our setup to be 15 minutes instead without any big problems (other
than potentially not having any active worker for 15 minutes)?

Ideally I'd like to get avoid storing such information in the ZODB at
all and would much prefer a volatile attribute, a thread local or
something on the filesystem. But maybe I'm missing something here were
this information needs to be shared across workers running on
different machines.

While the size of the transaction is only 140 bytes each, the overhead
of new transactions is massive.

Any insights appreciated,
Hanno
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )


Re: [Zope-dev] zc.async.queue.DispatcherAgents last_ping

2011-06-29 Thread Fred Drake
On Wed, Jun 29, 2011 at 7:57 AM, Hanno Schlichting ha...@hannosch.eu wrote:
 Ideally I'd like to get avoid storing such information in the ZODB at
 all and would much prefer a volatile attribute, a thread local or
 something on the filesystem. But maybe I'm missing something here were
 this information needs to be shared across workers running on
 different machines.

I do believe that's the case.  Increasing the period is probably safe.

Another strategy you can employ is to use a separate database for the zc.async
bookkeeping; that could have different pack policies or a completely different
storage implementation.


  -Fred

-- 
Fred L. Drake, Jr.    fdrake at acm.org
Give me the luxuries of life and I will willingly do without the necessities.
   --Frank Lloyd Wright
___
Zope-Dev maillist  -  Zope-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 https://mail.zope.org/mailman/listinfo/zope-announce
 https://mail.zope.org/mailman/listinfo/zope )