On Wednesday 31 January 2001 04:18, Donovan Baarda wrote:
On Mon, Jan 29, 2001 at 07:26:56PM +0100, Dieter Maurer wrote:
Donovan Baarda writes:
On Sun, Jan 28, 2001 at 09:31:07PM +0100, Dieter Maurer wrote:
Donovan Baarda writes:
I remember reading ages ago on a wishlist for ZServer that
someone was thinking of adding blocking support to producers.
This patch adds support for just that in a non-intrusive way. I
haven't tried applying this patch to ZServer, but it _should_
work.
What does "blocking support dor producers" mean?
Disclaimer: you must be moderately familiar with
ZServer/medusa/asynchat for the following to make sense;
Asynchat used by ZServer and medusa use producers as a way to "push"
data through an async interface. The current version of asynchat
requires that producers be able to produce data immediately on calling
their more() method, untill they are finished, when they return ''.
My patch adds support for producers being able to "block" by letting
asynchat know if they are ready via a ready() method. This allows you
to use the simple producer model to push data that may not be
immediately available. An example would be a http-fetch producer that
fetches data from upstream for a proxy.
This patch is only useful for people creating or modifying things like
medusa or ZServer.
Thank you for this good explanation.
By now, I think, it would not be usefull for ZServer, at least
not for standard operations.
ZServer is a multi-threaded server, where different requests
are serviced by different threads.
The thread may block and does simply not return from "more()".
On first thought, you patch might free some blocked threads
to do productive work. However, Zope's transaction system
is linked to thread management. If you patch would be used
for that, Zope's transaction subsystem would need to be
rewritten.
Hmmm, it's been a while since I looked at ZServer, I might have to take
another look. I thought that it pre-spawned a number of threads that each
ran their own asyncore.loop event handler. I thought this was a neat way to
get the benefits of async event handlers and distributing threads on
multiple processors. Unless it spawns new threads when it needs them, it
would still be possible for ZServer to get blocked if all the threads were
blocked.
disclaimer: insert here.
zserver has a main thread which binds and listens, passing requests to a pool
of zpublisher threads. the zpublisher threads communicate with the client via
the main threads dispatcher, ie output is funneled back into the main
thread's dispatcher so as not to tie up threads talking to distant clients.
reference: Zserver/medusa/select_trigger.py
python has a global interpreter lock, so multiple cpus have little utility.
However, my patch is only useful if you _have_ producers that can block. It
is usually possible to re-structure your system so that no producers block
(typicaly by using a dispatcher instead of a producer). The ready() block
feature is really a shortcut way of modifying an existing producer-based
design when you realise that the producer can block; ie an alternative way
of implementing it that may or may not be better.
i've taken a quick look at the patch, i find it interesting in that it
provides i think a way to implement a Proactor type server vs the standard
medusa Reactor style server.
reference: Patterns of System Architecture Vol II. Douglas Schmidt et al
reference: http://www.cs.wustl.edu/~schmidt/PDF/proactor.pdf
and a webserver that makes use of this pattern
http://www.cs.wustl.edu/~jxh/research/
kapil
In my case I wanted to create a cached-ftp-backend filesystem that medusa
could serve via ftp or http. Unfortunately the existing ftp and http server
implementations used producers to pull data from the abstract filesystem.
Since my "filesystem" could block pretty severly, I either needed to
completely re-write the http and ftp servers to use dispatchers instead of
producers, or implement blocking for producers (or use a seperate thread
for each request, but I thought that defeated the purpose of using an async
event loop).
I really must look at ZServer to figure out how it handles things... I
suspect that my patch by itself would not _hurt_ ZServer, but perhaps if
you started using the ready() feature it provides it might cause problems.
I'll have a look at ZServer again soon and give you an evaluation of how my
patch would interact with it.
Who knows, someone might find a need for blocking producers as ZServer
evolves, and if people know the patch exists, they might use it instead of
re-structuring to get around it :-)
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or
