On Tue, Dec 23, 2008 at 12:57 AM, Kottiyath <n.kottiy...@gmail.com> wrote: > Hi, > I have been looking at Twisted and lately Circuits as examples for > event driven programming in Python.
Wonderful! :) "circuits" that is :) > Even though I understood how to implement the code in these and > what is deferred etc, I have not yet understood the implementation of > deferred. I went through a lot of tutorials, but I guess most places > they expect that the user already understands how events are > generated. The tutorials mention that there is no more threads once > twisted is used. deferred to me is nothing more than scheduling "things" to happen "later". ie: Timers, or Timed Events. I clearly biased here as I'm the developer of circuits, so I can't comment on Twisted's design or architecture, but the way to achieve Twisted's so-called deferred (events?) is to use the Timer component, Timed Events. > My question is as follows: > I have not understood how the callbacks are hit without (a) > blocking the code or (b) having new threads. Again speaking in terms of circuits, but also in the event-driven paradigm: a) Event Handlers _can_ block - but ideally shouldn't (ihmo). b) Forking new threads/processes per event is mostly not necessary. (There are exceptions). > The usual example given is that of a program waiting for data coming > through a socket. In the tutorials, it is mentioned that -in an event > driven program, we schedule the code to hit when the remote server > gets back to us - . > Now, my question is - somebody has to still wait on that socket and > check whether the data is received, and once all the data is received, > call the appropriate callbacks. In the case of circuits' TCPServer component you simply poll it in your main event-loop. The pattern looks like this: from circuits.lib.sockets import TCPServer server = TCPServer(8000) while True: server.poll() server.flush() The TCPServer component has various builtin event handlers that do all the work for you and expose other more useful events to the application, such as: * connect * disconnect * read * error > Is twisted creating a micro-thread which just waits on the socket and > once the data is received, calls callFromThread for it to run on the > main loop? I can't comment - I tend to avoid threads where ever possible. > If so, Even though data locking etc is not a problem, are we not still > having threads? Will it not still cause scalability problems in high > traffic? Regarding scalability btw ... (in case you're interested) circuits comes with circuits.lib.web and several sets of Web Components. A lot of code was borrowed from the BaseHTTPServer from the python standard library and bits and pieces from CherryPy and the cgi module.... In terms of performance, it has a "raw" performance ~3k req/s on good hardware. Do have fun in your endeavours :) cheers James Circuits: http://trac.softcircuit.com.au/circuits/ -- http://mail.python.org/mailman/listinfo/python-list
