[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: You're right, I just saw my mistake. Sorry for the noise (the extra performance arguement is not a good enough motivation to tune this dynaically). Closing. -- resolution: -> invalid stage: needs patch -> committed/rejected status: open -> cl

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: Of course it does, the write ends all get filled, so the number of ready writers drops to zero... That's not all at the problem I'm talking about (I may not be clear though, a script will make it more clear). -- __

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: > Yes, but the problem is that between two epoll_wait() calls, the > readiness of the FDs can have changed: and if that happens, you'll get > the same list over and over. If an fd *was* ready but isn't anymore then why would you want to know about it? Trying

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Here is a version which uses epoll to service a number of pipes which is larger than maxevents. (If NUM_WRITERS is too large then I get "OSError: [Errno 24] Too many open files".) All pipes get serviced and the output is: Working with 20 FDs, 5 maxevents [5

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: > That assumes that epoll_wait() is supposed to return *all* ready fds. But > that is not possible because maxevents is finite. If you want all events > then obviously you may need to call epoll_wait() multiple times. Yes, but the problem is that be

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: > The fact that that the FDs are duped shouldn't change anything to the > events reported: it works while the number of FDs is less than > FD_SETSIZE (epoll_wait() maxevents argument). That assumes that epoll_wait() is supposed to return *all* ready fds. But t

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: > The program does *not* demonstrate starvation because you are servicing the > resource represented by the "starved" duplicate fds before calling poll() > again. No. What the program does is the following: while all the write FDs have not been retur

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: > I actually wrote a script to reproduce this issue: The program does *not* demonstrate starvation because you are servicing the resource represented by the "starved" duplicate fds before calling poll() again. You are creating thousands of duplicate handles f

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Antoine Pitrou
Antoine Pitrou added the comment: > 3) use a simple heuristic: start with a reasonable value for > `maxevents` (FD_SETSIZE seems like a good candidate), and if > epoll_wait() ever returns `maxevents` events, double the value (that's > what libevent does, with a capping to 32000). What if some ev

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Charles-François Natali
Charles-François Natali added the comment: The implementation can't simply drain a queue, because it can be level-triggered (which is the case by default), so you want to keep events around. The kernel actually uses a red-black tree, but I didn't really check how it's done (there's probably very

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-06 Thread Richard Oudkerk
Richard Oudkerk added the comment: Is this actually a problem? If events are arranged in a queue and epoll_wait() just removes the oldest events (up to maxevents) from that queue then there would be no problem with using a small value for maxevents. I don't *know* if that is the case, but I w

[issue16873] increase epoll.poll() maxevents default value, and improve documentation

2013-01-05 Thread Charles-François Natali
New submission from Charles-François Natali: In issue #16853, it was noted that many several projects don't set epoll.poll() maxevents argument, which effectively limits the number of events retuend to FD_SETSIZE-1 (set in selectmodule.c). Also, the methode documentation can confuse users into