Re: [ZODB-Dev] ZEO ClientStorage's use of the asyncore main loop (was [Fwd: Re: [Zope3-dev] Twisted Publisher and Zope 2])

2005-12-09 Thread Shane Hathaway

Jim Fulton wrote:

Tim and I have discussed this for some time.  We think an
asynchronous I/O approach is still appropriate, to handle
asynchronous messages from servers to clients, but we need
to get away from expecting a server to provide the asyncore
main loop needed by ZEO. Rather, ZEO should provide its own
main loop running in a separate thread. (Or, possibly, each
client storage should have its own thread.) Among other things,
this would allow us to get rid of the horribly complex dual-mode
code we have now.


I think that's a good strategy.


(Note that we've had lots of frustration with asyncore.  It is
   tempting to get rid of that at the same time.  However, we still
   need an asynchronous I/O framework.  This would probably requite
   a switch to Twisted.  This might be worthwhile, but would almost
   certainly entail a lot of risk.  I'm not at all sure the benefit
   is worth it unless there were some able volunteers who wanted to
   work on it.)


Last I checked, Twisted could not support a private event loop.  Twisted 
assumes there is only one event loop for all threads.  I'd like to know 
if this has changed.  The asyncore module can support a private event 
loop, although it's ugly--you have to pass the 'map' argument all over 
the place.


Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] ZEO ClientStorage's use of the asyncore main loop (was [Fwd: Re: [Zope3-dev] Twisted Publisher and Zope 2])

2005-12-09 Thread Jim Fulton

Shane Hathaway wrote:

Jim Fulton wrote:


Tim and I have discussed this for some time.  We think an
asynchronous I/O approach is still appropriate, to handle
asynchronous messages from servers to clients, but we need
to get away from expecting a server to provide the asyncore
main loop needed by ZEO. Rather, ZEO should provide its own
main loop running in a separate thread. (Or, possibly, each
client storage should have its own thread.) Among other things,
this would allow us to get rid of the horribly complex dual-mode
code we have now.



I think that's a good strategy.


(Note that we've had lots of frustration with asyncore.  It is
   tempting to get rid of that at the same time.  However, we still
   need an asynchronous I/O framework.  This would probably requite
   a switch to Twisted.  This might be worthwhile, but would almost
   certainly entail a lot of risk.  I'm not at all sure the benefit
   is worth it unless there were some able volunteers who wanted to
   work on it.)



Last I checked, Twisted could not support a private event loop.  Twisted 
assumes there is only one event loop for all threads.  I'd like to know 
if this has changed.  The asyncore module can support a private event 
loop, although it's ugly--you have to pass the 'map' argument all over 
the place.


Historical note: asyncore didn't support private event loops either
until I added the extra argument to the constructor.  I'm not sure why
you need to pass the map to anything but the constructor.  Oddly. most,
but not all of the methods that take a map, default to the one set in the
constructor.  I don't really remember the details

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] ZEO ClientStorage's use of the asyncore main loop (was [Fwd: Re: [Zope3-dev] Twisted Publisher and Zope 2])

2005-12-08 Thread Jim Fulton


Here is a draft of a note I plan to send to the zodb list soon:

The ZEO client storage currently relies on a host application to
provide an ayncore main loop.  This causes two major headaches:

1. The ZEO client storage has to have two modes of operation,
depending on whether the host application's main loop has started.

2. The ZEO client storage doesn't work optimally if there isn't a
main loop.  In particular, without an asyncore main loop, a
ZEO client storage won't see messages from the ZEO server until
the client storage tries to read or write some data from or to
the ZEO storage server.

Starting in this month's release, Zope 3 is moving away from the
asyncore-based ZServer network (HTTP, FTP) server architecture.
I expect Zope 2 to follow in the June release.  This means that Zope
no-longer provides the main loop that the ZEO client storage expects.

Tim and I have discussed this for some time.  We think an
asynchronous I/O approach is still appropriate, to handle
asynchronous messages from servers to clients, but we need
to get away from expecting a server to provide the asyncore
main loop needed by ZEO. Rather, ZEO should provide its own
main loop running in a separate thread. (Or, possibly, each
client storage should have its own thread.) Among other things,
this would allow us to get rid of the horribly complex dual-mode
code we have now.

This should be a project for the next development cycle feeding
the June release.

In the short term, I'll probably have to add some Zope 3.2 code
to start an asyncore main loop for ZEO's use.

Thoughts?

(Note that we've had lots of frustration with asyncore.  It is
   tempting to get rid of that at the same time.  However, we still
   need an asynchronous I/O framework.  This would probably requite
   a switch to Twisted.  This might be worthwhile, but would almost
   certainly entail a lot of risk.  I'm not at all sure the benefit
   is worth it unless there were some able volunteers who wanted to
   work on it.)

Jim

--
Jim Fulton   mailto:[EMAIL PROTECTED]   Python Powered!
CTO  (540) 361-1714http://www.python.org
Zope Corporation http://www.zope.com   http://www.zope.org


---BeginMessage---

Stephan Richter wrote:

On Thursday 08 December 2005 08:29, Sidnei da Silva wrote:


On Thu, Dec 08, 2005 at 08:18:49AM -0500, Stephan Richter wrote:
| On Thursday 08 December 2005 07:06, Sidnei da Silva wrote:
|  Just one thing that struck me right now. ZServer uses medusa/asyncore
|  and twisted has it's own 'main loop'. How do they play together in
|  Zope 3? Or they dont?
|
| They don't. Either you use ZServer or you use Twisted, but not both. So
| if you want to use Twisted-specific packages, like the scheduler, you
| have to use twisted, otherwise you are out of luck. I think this is
| totally fine.

What's the situation with ZEO then? The ZEO 'zrpc.client' uses
ThreadedAsync.register_loop_callback(), which is a evil monkeypatch to
asyncore. I haven't seen that change recently, so I assume this has
been ignored?



Yep. Nobody has complained to us about this yet. I personally do not use ZEO, 
so I could not fix the issue anyways.




I don't expect other people for me to fix this for me, but I suggest 
that this be noted somewhere where it can be seen, because ZEO is quite 
important for us running larger sites.


Even though many people do not use ZEO themselves, when considering and 
comparing web-app frameworks, it is a major pro for Zope to be able to 
provide cluster capabilities out of the box.


So the capability to use an alternative to ZServer for ZEO sites should, 
I suggest, be considered as a desirable future feature by ye Zope3 Gods.


My 0.02 €

/dario

--
-- ---
Dario Lopez-Kästen, IT Systems  Services Chalmers University of Tech.
Lyrics applied to programming  application design:
emancipate yourself from mental slavery - redemption song, b. marley

___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/jim%40zope.com

---End Message---
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev