Re: [Python-Dev] epoll implementation

2006-05-29 Thread Barry Scott
On May 27, 2006, at 04:59, Alex Martelli wrote: > > I believe it's: kqueue on FreeBSD (for recent-enough versions > thereof), otherwise epoll where available and nonbuggy, otherwise > poll ditto, otherwise select -- that's roughly what Twisted uses for kqueue is not always faster. It depends on

Re: [Python-Dev] epoll implementation

2006-05-27 Thread Martin v. Löwis
Ross Cohen wrote: > The first thing any user of the poll interface does with the file descriptor > is map it to some state object. That's where the lookup can be saved, the > object can just be handed back directly. Problem being that when the fd is > unregistered, we won't get back the PyObject po

Re: [Python-Dev] epoll implementation

2006-05-27 Thread Greg Ewing
Alex Martelli wrote: > On May 26, 2006, at 6:27 PM, Steve Holden wrote: >>Of course that would mean establishing which *was* the best available >>which, as we've seen this week, may not be easy. > > I believe it's: kqueue on FreeBSD (for recent-enough versions > thereof), otherwise epoll where

Re: [Python-Dev] epoll implementation

2006-05-27 Thread Ross Cohen
On Sat, May 27, 2006 at 02:27:20AM +0100, Steve Holden wrote: > Greg Ewing wrote: > > Rather than adding yet another platform-dependent module, > > I'd like to see a unified Python interface in the stdlib > > that uses whichever is the best one available. > > > Of course that would mean establishi

Re: [Python-Dev] epoll implementation

2006-05-27 Thread Ross Cohen
On Sat, May 27, 2006 at 08:36:12AM +0200, "Martin v. Löwis" wrote: > Ross Cohen wrote: > > epoll also allows 64 bits of data to be tucked away and returned when events > > happen. Could be useful for saving a dict lookup for every event. However, > > there are some refcounting issues. Dict lookup p

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Alex Martelli wrote: >> Of course that would mean establishing which *was* the best available >> which, as we've seen this week, may not be easy. > > I believe it's: kqueue on FreeBSD ... Such a statement assumes they are semantically equivalent. However, I believe they aren't. A specific usage p

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Greg Ewing wrote: > There are many different select-like things around now > (select, poll, epoll, kqueue -- are there others?) and > random combinations of them seem to be available on any > given platform. This makes writing platform-independent > code that needs select-like functionality very aw

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Ross Cohen wrote: > epoll also allows 64 bits of data to be tucked away and returned when events > happen. Could be useful for saving a dict lookup for every event. However, > there are some refcounting issues. Dict lookup per event could be traded > for one on deregistration. All it needs is a sma

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ganesan Rajagopal
> Steve Holden <[EMAIL PROTECTED]> writes: >> Rather than adding yet another platform-dependent module, >> I'd like to see a unified Python interface in the stdlib >> that uses whichever is the best one available. >> > Of course that would mean establishing which *was* the best available > w

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Alex Martelli
On May 26, 2006, at 6:27 PM, Steve Holden wrote: > Greg Ewing wrote: >> Fredrik Lundh wrote: >> >> >>> roughly speaking, epoll is kqueue for linux. >> >> >> There are many different select-like things around now >> (select, poll, epoll, kqueue -- are there others?) and >> random combinations of t

[Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing: import epoll as select It was released under the Python license on sourceforge: http://sourceforge.net/proj

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Steve Holden
Greg Ewing wrote: > Fredrik Lundh wrote: > > >>roughly speaking, epoll is kqueue for linux. > > > There are many different select-like things around now > (select, poll, epoll, kqueue -- are there others?) and > random combinations of them seem to be available on any > given platform. This make

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Greg Ewing
Fredrik Lundh wrote: > roughly speaking, epoll is kqueue for linux. There are many different select-like things around now (select, poll, epoll, kqueue -- are there others?) and random combinations of them seem to be available on any given platform. This makes writing platform-independent code th

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 11:56:06PM +0200, "Martin v. Löwis" wrote: > I thought about this (even though I never used it), and I think there > are good uses for EPOLLET. There are, if the programmer wants to deal with it. Easy enough to add the flag and give them the choice. I'll put together a sele

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 11:43:05PM +0200, Ronald Oussoren wrote: > >Ok, I'm not familiar with intimate details of kqueue. However, if > >there > >were a select.poll implementation based on kqueue, it would still > >be an > >improvement, since there isn't *any* implementation on OS X right now.

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Ross Cohen wrote: > I've never been fond of the edge-triggered mode. It's a pretty minor > optimization, it saves scanning the set of previously returned fds to see > if the events on them are still relevant. Given that there are added > pitfalls to using that mode, it never seemed worth it. Any la

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ronald Oussoren
On 26-mei-2006, at 22:22, Ross Cohen wrote: >> >> AIUI, kqueue actually isn't implemented for PTYs on OS X, whereas >> poll(2) is. Given this, I don't think kqueue is actually strictly >> better. Although hopefully Apple will get their act together and >> fix this deficiency. > > Ok, I'm

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Ross Cohen wrote: > 1) Provide an epoll implementation which is used "silently" when the call is > available. > > 2) Expose both poll(2) and epoll(4) in python and build select.poll on top of > whatever is available. > > Ok, so 2 is only different in that it exposes the lower level APIs. I'd li

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 10:12:15PM +0200, "Martin v. Löwis" wrote: > That said, I would be in favour of having select.poll "silently" use > epoll where available. Of course, it would be good if a "cheap" run-time > test could be made whether epoll is available at run-time (although > just waiting f

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
> How about "not *only* as a separate set of calls"? If poll(2) and > epoll(4) are both available on the underlying platform, then they > should both be exposed to Python as separate APIs. Then, on top of > that, a relatively simple layer which selects the most efficient > mechanism can be expose

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 02:49:44PM -0400, Jean-Paul Calderone wrote: > On Fri, 26 May 2006 14:31:33 -0400, Ross Cohen <[EMAIL PROTECTED]> wrote: > > > >I agree that it should go into the select module, but not as a seperate > >set of calls. > > How about "not *only* as a separate set of calls"? I

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Ross Cohen wrote: > True, and as I mentioned before, the python API more closely matches epoll > in this case. The level-triggered mode of epoll is an almost perfect match. > Someone went to some lengths to hide the details of the system poll > interface. Ah right, I missed that point. That makes

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 08:48:42PM +0200, "Martin v. Löwis" wrote: > Ross Cohen wrote: > > I agree that it should go into the select module, but not as a seperate > > set of calls. epoll is strictly better than poll. kqueue is strictly > > better than poll. Windows has its own completion ports API.

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Jean-Paul Calderone
On Fri, 26 May 2006 13:31:33 -0400, Ross Cohen <[EMAIL PROTECTED]> wrote: >On Fri, May 26, 2006 at 01:10:30PM -0400, Jean-Paul Calderone wrote: >> Of course, if there is a volunteer to maintain and support an extension >> module, that's better than nothing. PyEpoll is missing a couple features I

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Jean-Paul Calderone
On Fri, 26 May 2006 14:31:33 -0400, Ross Cohen <[EMAIL PROTECTED]> wrote: >On Fri, May 26, 2006 at 08:03:12PM +0200, "Martin v. Löwis" wrote: >> Ross Cohen wrote: >> > Is there any interest in incorporating this into the standard python >> > distribution? >> >> I would like to see epoll support in

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Ross Cohen wrote: > I agree that it should go into the select module, but not as a seperate > set of calls. epoll is strictly better than poll. kqueue is strictly > better than poll. Windows has its own completion ports API. Why should > an application developer have to detect what platform they ar

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Jonathan LaCour wrote: > There is, in fact, an implementation of kqueue for Python already > available. I have not used it myself, but it is available here: > > http://python-hpio.net/trac/ > > maybe the needforspeed people could take a look at this? "take a look" is really quite involved.

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 08:03:12PM +0200, "Martin v. Löwis" wrote: > Ross Cohen wrote: > > Is there any interest in incorporating this into the standard python > > distribution? > > I would like to see epoll support in Python, but not in the way PyEpoll > is packaged. Instead, I think it should go

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Jonathan LaCour
Guido van Rossum wrote: > On a related note, perhaps the needforspeed folks should look into > supporting kqueue on systems where it's available? That's a really > fast BSD-originated API to replace select/poll. (It's fast due to the > way the API is designed.) There is, in fact, an implementation

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Christopher Weimann
On 05/26/2006-10:07AM, Guido van Rossum wrote: > > I don't know what epoll is. > > On a related note, perhaps the needforspeed folks should look into > supporting kqueue on systems where it's available? That's a really > fast BSD-originated API to replace select/poll. (It's fast due to the > way

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Martin v. Löwis
Ross Cohen wrote: > Is there any interest in incorporating this into the standard python > distribution? I would like to see epoll support in Python, but not in the way PyEpoll is packaged. Instead, I think it should go into the select module, and be named epoll_*. Likewise, if kqueue was ever su

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 01:10:30PM -0400, Jean-Paul Calderone wrote: > Including a wrapper for this functionality would be quite useful for many > python network apps. However, I think with ctypes going into 2.5, it might > be better to consider providing epoll support using a ctypes-based modul

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Fredrik Lundh
Guido van Rossum wrote: > I don't know what epoll is. > > On a related note, perhaps the needforspeed folks should look into > supporting kqueue on systems where it's available? That's a really > fast BSD-originated API to replace select/poll. (It's fast due to the > way the API is designed.) ro

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Sean Reifschneider
On Fri, May 26, 2006 at 01:13:44PM -0400, Ross Cohen wrote: >The NeedForSpeed wiki lists /dev/epoll as a goal for the twisted folks. I Just FYI, none of the Twisted items made it onto the list of items being worked. Thanks, Sean -- "You're thinking of Mr. Wizard." "[Emilio Lizardo's] a top sci

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
On Fri, May 26, 2006 at 11:47:43AM -0500, [EMAIL PROTECTED] wrote: > > Ross> I wrote an epoll implementation which can be used as a drop-in > Ross> replacement for parts of the select module > ... > Ross> Is there any interest in incorporating this into the standard > Ross> pyt

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Jean-Paul Calderone
On Fri, 26 May 2006 11:47:43 -0500, [EMAIL PROTECTED] wrote: > >Ross> I wrote an epoll implementation which can be used as a drop-in >Ross> replacement for parts of the select module >... >Ross> Is there any interest in incorporating this into the standard >Ross> python distribu

Re: [Python-Dev] epoll implementation

2006-05-26 Thread Guido van Rossum
I don't know what epoll is. On a related note, perhaps the needforspeed folks should look into supporting kqueue on systems where it's available? That's a really fast BSD-originated API to replace select/poll. (It's fast due to the way the API is designed.) --Guido On 5/26/06, Ross Cohen <[EMAIL

Re: [Python-Dev] epoll implementation

2006-05-26 Thread skip
Ross> I wrote an epoll implementation which can be used as a drop-in Ross> replacement for parts of the select module ... Ross> Is there any interest in incorporating this into the standard Ross> python distribution? Without going to the trouble of downloading epoll (always an

[Python-Dev] epoll implementation

2006-05-26 Thread Ross Cohen
I wrote an epoll implementation which can be used as a drop-in replacement for parts of the select module (assuming the program is using only poll). The code can currently be used by doing: import epoll as select It was released under the Python license on sourceforge: http://sourceforge.net/proj