Re: [Python-Dev] microthreading vs. async io

2007-02-26 Thread Joachim König-Baltes
Adam Olsen wrote: > That would depend on whether Joachim's wait() refers to the individual > tasks' calls or the scheduler's call. I assumed it referred to the > scheduler. In the basic form it would literally be select.select(), > which has O(n) cost and often fairly large n. The wait(events, ti

Re: [Python-Dev] microthreading vs. async io

2007-02-26 Thread Adam Olsen
On 2/25/07, Armin Rigo <[EMAIL PROTECTED]> wrote: > Hi Adam, > > On Thu, Feb 15, 2007 at 06:17:03AM -0700, Adam Olsen wrote: > > > E.g. have a wait(events = [], timeout = -1) method would be sufficient > > > for most cases, where an event would specify > > > > I agree with everything except this.

Re: [Python-Dev] microthreading vs. async io

2007-02-26 Thread Joachim König-Baltes
Armin Rigo wrote: > I just realized that this is not really true in the present context. > If the goal is to support programs that "look like" they are > multi-threaded, i.e. don't use callbacks, as I think is Joachim's goal, > then most of the time the wait() function would be only called with a >

Re: [Python-Dev] microthreading vs. async io

2007-02-25 Thread Armin Rigo
Hi Adam, On Thu, Feb 15, 2007 at 06:17:03AM -0700, Adam Olsen wrote: > > E.g. have a wait(events = [], timeout = -1) method would be sufficient > > for most cases, where an event would specify > > I agree with everything except this. A simple function call would > have O(n) cost, thus being unac

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread dustin
On Thu, Feb 15, 2007 at 11:47:27AM -0500, Jean-Paul Calderone wrote: > Is the only problem here that this style of development hasn't had been made > visible enough? Yep -- I looked pretty hard about two years ago, and although I haven't been looking for that specifically since, I haven't heard an

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Adam Olsen
On 2/15/07, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote: > Is the only problem here that this style of development hasn't had been made > visible enough? Perhaps not the only problem, but definitely a big part of it. I looked for such a thing in twisted after python 2.5 came out and was unable

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Adam Olsen
On 2/15/07, Joachim König-Baltes <[EMAIL PROTECTED]> wrote: > Adam Olsen schrieb: > > I don't think we're on the same page then. The way I see it you want > > a single async IO implementation shared by everything while having a > > collection of event loops that cooperate "just enough". The async

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Phillip J. Eby
At 11:47 AM 2/15/2007 -0500, Jean-Paul Calderone wrote: >Is the only problem here that this style of development hasn't had been made >visible enough? You betcha. I sure as heck wouldn't have bothered writing the module I did, if I'd known it already existed. Or at least only written whatever

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Jean-Paul Calderone
On Thu, 15 Feb 2007 10:36:21 -0600, [EMAIL PROTECTED] wrote: > [snip] > >def fetchSequence(...): > fetcher = Fetcher() > yield fetcher.fetchHomepage() > firstData = yield fetcher.fetchPage('http://...') > if someCondition(firstData): >while True: > secondData = yield fetcher.fetchPage(

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread dustin
On Thu, Feb 15, 2007 at 04:51:30PM +0100, Joachim K?nig-Baltes wrote: > The style used in asyncore, inheriting from a class and calling return > in a method > and being called later at a different location (different method) just > interrupts the > sequential flow of operations and makes it harde

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Phillip J. Eby
At 11:00 PM 2/14/2007 -0600, [EMAIL PROTECTED] wrote: >Instead, I would like to concentrate on producing a small, clean, >consistent, generator-based microthreading library. I've seen several >such libraries (including the one in PEP 342, which is fairly skeletal), >and they all work *almost* the

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Joachim König-Baltes
Joachim König-Baltes wrote: > The problem solved by this approach is to allow a number of cooperating > threads to wait for an event without the need to busy loop or block by > delegating > the waiting to a central instance, the scheduler. How this efficient > waiting is > implemented is the resp

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Joachim König-Baltes
Adam Olsen schrieb: > I don't think we're on the same page then. The way I see it you want > a single async IO implementation shared by everything while having a > collection of event loops that cooperate "just enough". The async IO > itself would likely end up being done in C. > No, I'd like to

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Adam Olsen
On 2/15/07, Joachim König-Baltes <[EMAIL PROTECTED]> wrote: > Adam Olsen wrote: > >> I have implemented something like the above, based on greenlets. > > > > I assume greenlets would be an internal implementation detail, not > > exposed to the interface? > Yes, you could use stackless, perhaps even

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Joachim König-Baltes
Adam Olsen wrote: > I agree with everything except this. A simple function call would > have O(n) cost, thus being unacceptable for servers with many open > connections. Instead you need it to maintain a set of events and let > you add or remove from that set as needed. We can learn from kevent h

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Adam Olsen
On 2/15/07, Joachim König-Baltes <[EMAIL PROTECTED]> wrote: > [EMAIL PROTECTED] wrote: > E.g. have a wait(events = [], timeout = -1) method would be sufficient > for most cases, where an event would specify I agree with everything except this. A simple function call would have O(n) cost, thus bei

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Joachim König-Baltes
[EMAIL PROTECTED] wrote: [...] > microtreading: > Exploiting language features to use cooperative multitasking in tasks > that "read" like they are single-threaded. > > asynchronous IO: > Performing IO to/from an application in such a way that the > application does not wait for any IO ope

Re: [Python-Dev] microthreading vs. async io

2007-02-15 Thread Josiah Carlson
"Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > > [EMAIL PROTECTED] schrieb: > > Asyncore *only* implements asynchronous IO -- any "tasks" performed in > > its context are the direct result of an IO operation, so it's hard to > > say it implements cooperative multitasking (and Josiah can correct m

Re: [Python-Dev] microthreading vs. async io

2007-02-14 Thread Martin v. Löwis
[EMAIL PROTECTED] schrieb: > Asyncore *only* implements asynchronous IO -- any "tasks" performed in > its context are the direct result of an IO operation, so it's hard to > say it implements cooperative multitasking (and Josiah can correct me if > I'm wrong, but I don't think it intends to). I'm

[Python-Dev] microthreading vs. async io

2007-02-14 Thread dustin
I've steered clear of this conversation for a while, because it drifted a little bit off my original topic. But I did want to straighten a little bit of terminology out, if only to resolve some of my own confusion over all the hubbub. I don't pretend to define the words others use; these definiti